Hey everyone, As you guys know recently I started learning about AI/ML in order to become an AI engineer and so today I’ll be sharing what I learned about RAG in hinglish.

Introduction

So sabse pehle 1st chiz jo humko karni chaiye wo hai ek environment banana taki jo bhi libraries hum download kar rahe hai or jis bhi version ki kar rahe hai hamara code usko hi use kare.

Uske baad hi humko koi bhi code ya kaam continue karna chaiye and the best way to manage environment in my opinion is using uv. uv use karke hum environment create kar sakte hai and isko set karna bhi kafi asan h.

So you can create environment using:

uv venv - for creating environment

source .venv/bin/activate (Mac/Linux) or .\.venv\Scripts\activate for activating that environment

Make sure to perform these on the same directory where you are working

and uske baad you can just perform uv pip install -r requirements.txt where requirements.txt consists of all relevant libraries which are required. Incase tum koi library bhul gye you can just add it using uv add <library-name>

Code

from pathlib import Path
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma
from langchain_openai import ChatOpenAI
import os

# Load all PDFs
docs = []
for pdf in Path(".").rglob("*.pdf"):
    pages = PyPDFLoader(str(pdf)).load()
    for p in pages:
        p.metadata["source"] = pdf.name
    docs.extend(pages)

print(f"Loaded {len(docs)} pages")

# Split into chunks
chunks = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200
).split_documents(docs)

print(f"Created {len(chunks)} chunks")

# Embeddings + Vector Store
embedding_model = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2"
)

vectorstore = Chroma(
    collection_name="pdf_chatbot",
    embedding_function=embedding_model
)

vectorstore.add_documents(chunks)

# DeepSeek
os.environ["DEEPSEEK_API_KEY"] = "xxxx"

llm = ChatOpenAI(
    api_key=os.environ["DEEPSEEK_API_KEY"],
    base_url="https://api.deepseek.com",
    model="deepseek-chat",
    temperature=0.1,
)

retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

def rag(query):
    docs = retriever.invoke(query)
    context = "\n\n".join(doc.page_content for doc in docs)
    return llm.invoke(
        f"""Answer only using the context below.

Context:
{context}

Question: {query}
Answer:"""
    ).content

Functioning

Ab me code ke blocks ko paragraph by paragraph show karte hue explain karunga also mene deepseek use kara hai but you can use any other model by tweaking code a little bit which I will also explain:

import

from pathlib import Path
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma
from langchain_openai import ChatOpenAI
import os

pathlib ka function hai path ko declare karna ki konsi file kahape hai. langchain_community ka function is it contains third-party integrations that implement the base interfaces defined in LangChain Core, making them ready-to-use in any LangChain. Is library me kafi kaam ki chize hai and jo hum use karenge wo hai PyPDFLoader jiska function hai pdf ka content read karna. Fir ata hai hamara langchain_text_splitters library jisme se hum RecursiveCharacterTextSplitter import karte hai ab iska kaam hota hai text ko chunks me divide karna - “RecursiveCharacterTextSplitter is the standard text-chunking method in ⁠LangChain. It takes a list of separators (defaulting to [”\n\n“, “\n”, “ “, “”]) and recursively slices text, prioritizing larger structural boundaries like paragraphs and sentences before falling back to words or characters to preserve semantic“ ye nearest separator ko dekhta hai and us hisab se jo input value doge (like chunk_size=500) usi hisab se chunks me content ko divide karega.