Module 3 of 16

Embeddings Deep Dive

Embedding models, optimization strategies, and choosing the right model for your use case

3 hours2 labsFree

Watch as Slides Course overview Lab code

Start here

Learning objectives

Understand how text embedding models work
Compare embedding models and their tradeoffs
Optimize embeddings for production performance
Choose the right embedding strategy for your data

Embeddings are the bridge between text and vector search. An embedding model converts text into a fixed-size vector (list of numbers) where similar meanings produce nearby vectors. The quality of your embeddings directly determines the quality of your retrieval.

How Embedding Models Work

Embedding models are neural networks trained on massive text pairs (question-answer, paraphrase, similar documents). They learn to map semantically similar text to nearby points in vector space. At inference time, they convert any text to a vector in milliseconds.

Comparing Embedding Models

from sentence_transformers import SentenceTransformer

# Small, fast — good for prototyping
model_small = SentenceTransformer('all-MiniLM-L6-v2')  # 384 dims, 22M params

# Large, accurate — good for production
model_large = SentenceTransformer('all-mpnet-base-v2')  # 768 dims, 109M params

# Domain-specific options:
# nomic-embed-text — strong general purpose
# voyage-3 — high quality, API-based
# text-embedding-3-large — OpenAI, 3072 dims

Embedding Optimization

Dimensionality: Higher dims capture more nuance but cost more storage and compute
Batch processing: Embed documents in batches for throughput
Caching: Cache embeddings — do not re-embed unchanged documents
Quantization: Reduce vector precision (float32 to int8) for 4x storage savings
Matryoshka embeddings: Models that work at variable dimensions (truncate for speed)

Common mistakes

What usually breaks

Using the cheapest/fastest embedding model without benchmarking quality
Re-embedding entire corpus on every update instead of incremental embedding
Mixing embedding models — query and document MUST use the same model
Not normalizing vectors before cosine similarity calculation

Key terms

Vocabulary used in this module

Embedding Model

Neural network that converts text to fixed-size vectors

Dimensionality

Number of values in the vector (e.g., 384, 768, 1536)

Quantization

Reducing vector precision to save storage (float32 → int8)

Matryoshka Embeddings

Models that produce useful embeddings at variable dimensions

Labs

Hands-on labs

30 minBeginner

Generate and Compare Embeddings

Explore how different models embed the same text.

Embed identical sentences with 3 different models
Compare vector dimensions and similarity scores
Measure latency and throughput per model
Visualize embedding clusters with t-SNE

View lab on GitHub

25 minIntermediate

Embedding Model Selection

Choose the right model for your use case.

Benchmark retrieval quality on a test dataset
Compare small vs large models on precision/recall
Measure latency at different batch sizes
Document model selection decision for production

View lab on GitHub

Recap

Key takeaways

Embedding quality directly determines retrieval quality
Smaller models (MiniLM) are fast but less accurate; larger models (mpnet) are better but slower
Batch processing and caching are essential for production throughput
Quantization reduces storage 4x with minimal quality loss
Choose your embedding model based on benchmarks on YOUR data, not general leaderboards

Related resources