Module 3: Embeddings Deep Dive

Learning Objectives

• Understand how text embedding models work
• Compare embedding models and their tradeoffs
• Optimize embeddings for production performance
• Choose the right embedding strategy for your data

Why This Matters

If your embeddings are bad, your retrieval is bad, and your RAG answers are bad. No amount of prompt engineering fixes poor embeddings. This module teaches you to choose, optimize, and evaluate the component that determines 80% of your RAG quality.

Lesson Content

Embeddings are the bridge between text and vector search. An embedding model converts text into a fixed-size vector (list of numbers) where similar meanings produce nearby vectors. The quality of your embeddings directly determines the quality of your retrieval.

How Embedding Models Work

Embedding models are neural networks trained on massive text pairs (question-answer, paraphrase, similar documents). They learn to map semantically similar text to nearby points in vector space. At inference time, they convert any text to a vector in milliseconds.

Comparing Embedding Models

from sentence_transformers import SentenceTransformer

# Small, fast — good for prototyping
model_small = SentenceTransformer('all-MiniLM-L6-v2')  # 384 dims, 22M params

# Large, accurate — good for production
model_large = SentenceTransformer('all-mpnet-base-v2')  # 768 dims, 109M params

# Domain-specific options:
# nomic-embed-text — strong general purpose
# voyage-3 — high quality, API-based
# text-embedding-3-large — OpenAI, 3072 dims

Embedding Optimization

• Dimensionality: Higher dims capture more nuance but cost more storage and compute
• Batch processing: Embed documents in batches for throughput
• Caching: Cache embeddings — do not re-embed unchanged documents
• Quantization: Reduce vector precision (float32 to int8) for 4x storage savings
• Matryoshka embeddings: Models that work at variable dimensions (truncate for speed)

Common Mistakes

• Using the cheapest/fastest embedding model without benchmarking quality
• Re-embedding entire corpus on every update instead of incremental embedding
• Mixing embedding models — query and document MUST use the same model
• Not normalizing vectors before cosine similarity calculation

Key Terms

Embedding Model

Neural network that converts text to fixed-size vectors

Dimensionality

Number of values in the vector (e.g., 384, 768, 1536)

Quantization

Reducing vector precision to save storage (float32 → int8)

Matryoshka Embeddings

Models that produce useful embeddings at variable dimensions

Hands-On Labs

1. Generate and Compare Embeddings

   Explore how different models embed the same text.

   30 min - Beginner

   • Embed identical sentences with 3 different models
   • Compare vector dimensions and similarity scores
   • Measure latency and throughput per model
   • Visualize embedding clusters with t-SNE

   View lab files on GitHub

2. Embedding Model Selection

   Choose the right model for your use case.

   25 min - Intermediate

   • Benchmark retrieval quality on a test dataset
   • Compare small vs large models on precision/recall
   • Measure latency at different batch sizes
   • Document model selection decision for production

   View lab files on GitHub

Key Takeaways

• Embedding quality directly determines retrieval quality
• Smaller models (MiniLM) are fast but less accurate; larger models (mpnet) are better but slower
• Batch processing and caching are essential for production throughput
• Quantization reduces storage 4x with minimal quality loss
• Choose your embedding model based on benchmarks on YOUR data, not general leaderboards