A Friendly Guide to “Ask-My-PDF” with LangChain (How Retrieval-Augmented Generation Works in Plain English)

Project Reference : https://www.youtube.com/watch?v=swCPic00c30&t=1366s

1 . Why you might care

• Problem : You have a long PDF full of facts; skimming it is painful.
• Dream : Type a natural-language question (“What’s scaled dot-product attention?”) and instantly get the answer, with citations.
• Solution : A retrieval-augmented-generation (RAG) pipeline built from a few open-source Lego bricks (LangChain, FAISS, an LLM, and a tiny Gradio front-end).

2 . The big idea in one breath

“Break the document into bite-sized chunks ➜ turn every chunk into a math vector ➜ when a user asks something, find the chunks whose vectors look similar ➜ feed those chunks, plus the user question, to an LLM ➜ show the answer.”

3 . Key parts, spoken like a tour guide

• Document Loader : Opens a PDF and hands you its pages as plain text objects.
• Text Splitter : Slices pages into ~1 000-character morsels so the model’s context window isn’t overloaded.
• Embeddings Model : Converts each morsel into a list of numbers (a “vector”) that captures meaning.
• Vector Store : A special database (FAISS) that can say “show me the chunks closest to this new vector.”
• Retriever : A polite façade around the vector store “give me K relevant chunks for query Q.”
• LLM (Large Language Model) : Reads the chunks + question and writes a human answer.
• Prompt Template : The instruction sheet the LLM follows (“Only use the context. Think step by step.”).
• Chain : Glue code that wires retriever → LLM in one call.
• Gradio UI : A two-widget webpage where you upload a PDF and ask questions.

4 . Think of it as a mini graph

• Nodes
• Loader node : emits raw pages
• Splitter node : emits chunks
• Embedding node : emits vectors
• Vector-store node : stores vectors, returns neighbors
• LLM node : emits answers
• Edges: plain Python function calls passing data along the arrow.
• State: the FAISS index on disk (so you don’t recompute every startup).

5 . Step-by-step roadmap (bullet style, no code yet)

1. Install libraries – pip install langchain faiss-cpu gradio ollama openai (swap or remove what you don’t need).
2. Pick your LLM -Local & free ? Use LLaMA 2 through Ollama. Cloud & bigger ? Use GPT-4o via OpenAI (OPENAI_API_KEY env var required).
3. Load your PDF – Feed the file path to PyPDFLoader.
4. Split it – Create a RecursiveCharacterTextSplitter with chunk_size=1000, chunk_overlap=20.
5. Embed – Make an OpenAIEmbeddings() (or OllamaEmbeddings() if local). Call FAISS.from_documents(chunks, embeddings).
6. Turn it into a retriever – retriever = db.as_retriever().
7. Write your prompt – Keep placeholders {context} and {input}.
8. Create a chain – document_chain = create_stuff_documents_chain(llm, prompt) retrieval_chain = create_retrieval_chain(retriever, document_chain)
9. Test in pure Python – retrieval_chain.invoke({“input”: “Your question”}) ➜ returns {“answer”: “…”, “context”: […]}.
10. Wrap in Gradio – Build a small def qa(pdf, text): … function and launch Interface.

6 . Most common “Wait, what about…?” questions

• “Do I need GPUs?” –No for OpenAI embeddings + remote LLM. Maybe yes if you run the LLM locally (but Ollama can stream CPU-only at small sizes).
• “Why split at 1 000 characters?” – Keeps each chunk well under model limits while holding a few paragraphs of context. Tune freely.
• “Can I store millions of chunks?” – Yes use a persistent vector DB (Chroma, Pinecone, Qdrant) instead of in-memory FAISS.
• “What about citations?” – The chain already returns the source chunks. Display result[‘context’] under each answer.
• “Is this secure for private docs?” – Use local embeddings + local LLM to keep data on-prem. Otherwise your text travels to OpenAI.

7 . Where this recipe shines

• Policy or contract chatbots for legal teams.
• Course handouts Q&A so students can query lecture PDFs.
• Technical manuals for field engineers with spotty internet (offline Ollama mode).
• Customer-support knowledge bases (swap PDF loader for Confluence or Notion loader).

8 . A super-minimal runnable example (20 lines)

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_community.llms import Ollama
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains import create_retrieval_chain
import gradio as gr

loader   = PyPDFLoader("myfile.pdf")
docs     = loader.load()
chunks   = RecursiveCharacterTextSplitter(
             chunk_size=1000, chunk_overlap=20
           ).split_documents(docs)
db       = FAISS.from_documents(chunks, OpenAIEmbeddings())
retriever= db.as_retriever()
llm      = Ollama(model="llama2")
prompt   = ChatPromptTemplate.from_template(
"""Answer from context only.
<context>{context}</context>
Q: {input}""")
doc_chain      = create_stuff_documents_chain(llm, prompt)
retrieval_chain= create_retrieval_chain(retriever, doc_chain)

def ask(q):
    return retrieval_chain.invoke({"input": q})["answer"]

gr.Interface(ask, gr.Textbox(label="Ask"), "text").launch()

Copy-paste, set OPENAI_API_KEY if you’re using OpenAI embeddings, change “myfile.pdf” to your file, and you have a personal Q&A bot in under a minute.

9 . Final takeaway

Building an “Ask My PDF” bot is mostly wiring together existing blocks:

• Loader ➜ Splitter ➜ Embeddings ➜ Vector store ➜ Retriever ➜ Prompt ➜ LLM ➜ Gradio

Once you grasp that sequence, you can swap any block (use a website loader, a different vector DB, a chart-drawing LLM, a React front-end, etc.) and produce a whole family of retrieval-powered apps. Happy hacking, and may your PDFs finally talk back!

10. RESULTS