Embeddings and vector databases
Monday, 10:10: a guest searches Lisbon reviews for "quiet room to work remotely" and gets zero results. The perfect review says "peaceful, fast wifi, great desk", and 38% of review searches end the same way.
Embeddings turn meaning into map coordinates, so "find something similar" becomes "find what's nearby".
- Problem
A guest searches reviews for “quiet room to work remotely”. The best review says “peaceful, fast wifi, great desk”.
- Why the simple system fails
Keyword search finds no shared words and returns nothing useful. Synonyms, paraphrases and other languages all defeat exact matching.
- Engineering concept
Represent each text as a vector whose position encodes meaning; similar meanings land close together, so search becomes nearest-neighbour lookup.
- Technology
Embedding models, cosine similarity, ANN indexes (HNSW), vector databases or pgvector.
- Trade-offs
Approximate search trades a little recall for huge speed. A dedicated vector DB adds scale features but also another system to run and sync.
The problem
2 min- Maya · Product managerA guest typed "quiet room to work remotely": 0 results. And 38% of all review searches return nothing.
- Maya · Product managerMeanwhile this review exists: "Peaceful, fast wifi, great desk. Perfect for a work week."
- You · Engineer at StayHubOur search matches words, and they share none. Add a synonym list?
The big idea
4 minSimilar meaning, nearby vectors: search becomes finding the nearest points.
Picture a library shelved by topic, so a book on quiet work retreats sits next to one on peaceful coworking cafés, though their titles share no word. You walk to where your question would be shelved and look at the nearest books.
How it works
5 min- 1Embed every review, once
An model turns each text into a vector of 1,536 numbers, trained so similar meanings point in similar directions. All 4.2M reviews, stored next to the text. It's a small model, not a chat model.
- 2Embed the query
"quiet room to work remotely" becomes a vector, with the same model. Each model draws its own map.
- 3Measure closeness
: 1 = same meaning, 0 = unrelated. "Peaceful, fast wifi, great desk": 0.84. "Quiet pool area, loud bar at night": 0.41, despite sharing "quiet".
- 4Find the nearest, fast
A uses an (approximate nearest-neighbour) index like : ~20 ms instead of ~1.5 s for checking all 4.2M, finding ~97% of the true top 10.
- 5Filter and return
Top 10, filtered by city and rating. A multilingual model puts "silencioso, ótimo wifi" next to its English twin. That's .
Watch a decision being made
4 minLina adds semantic review search
Find reviews by meaning across 4.2M reviews in 9 languages, filtered by city and rating, fast enough for a search box.
- 1One model, versioned
A multilingual model, 1,536 dimensions, for both reviews and queries. Store its name and version with every vector.
Why: Mixing models silently breaks every distance.
- 2Size it before building
4.2M × 1,536 numbers × 4 bytes ≈ 26 GB raw, plus index overhead.
Why: Memory decides where the index can live. 26 GB fits in a 256 GB replica.
- 3Index on a replica
Embed all reviews once in a batch; new reviews on insert. HNSW with cosine distance in .
sqlALTER TABLE reviews ADD COLUMN embedding vector(1536); ALTER TABLE reviews ADD COLUMN embedding_model text; CREATE INDEX ON reviews USING hnsw (embedding vector_cosine_ops);Why: Documents are embedded once, queries per search. The replica keeps vector load away from bookings.
- 4Check recall, then ship
500 sample queries vs exact search: ~97% of the true top 10 found. Searches with no result: 38% → 4%.
Why: ANN is approximate by design; measure how approximate before guests do.
In real life
3 minSpotify: its own nearest-neighbour libraries
- Open-sourced Annoy, and later Voyager, ANN libraries built for its recommendations
- Huge catalogue and traffic, a team to build and run it
Fits when vector search is core, at enormous scale.
Supabase: vectors inside Postgres
- Promotes pgvector so users store embeddings in the Postgres they already have
- Filtered and joined in SQL; no second system to sync
Fits millions, not billions, of vectors.
What goes wrong
3 minDual writes, sync lag, deleted reviews still showing, another bill.
pgvector if you run Postgres; move when measurements justify it.
"room 312" or "LIS-2280" carry little meaning; embeddings blur them.
Keyword search for exact terms, vectors for meaning: .
It averages every topic and matches nothing well.
Split into passages and embed each one.
Trade-offs
3 min| Search | Strong at | Weak at |
|---|---|---|
| Keyword | Codes, names, numbers; cheap and explainable | Synonyms, paraphrases, other languages |
| Semantic (vectors) | Meaning, descriptive queries, across languages | Exact identifiers; approximate recall |
| Hybrid (both, merged) | Both kinds of query | Two indexes; merging to tune |
- Maya · Product managerSemantic search is so much better. Let's replace keyword search everywhere.
- Omar · On-call SREGuests also search "room 312", "wifi code", "Hotel Alfama". Embeddings blur exact things.
- Maya · Product managerTwo search systems means twice the work.
Your job as tech lead
4 minClaude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.
- Which embedding model, and that queries and reviews always use the same one, with the model id stored per vector.
- Where vectors live: a column in the Postgres you run, or a second database to keep in sync.
- What happens when the model changes: re-embed everything into a new column, then switch both sides together.
38% of review searches return nothing because we only match keywords. Add semantic search over the 4.2M reviews from the last 3 years using pgvector in our Postgres: a `vector(1536)` column, the model id per row, an HNSW cosine index, and a batch job embedding new and edited reviews. Queries use the same model and keep the hotel filter in the same SQL. No new database; don't touch booking tables. Plan first with backfill time and cost, then show top 5 for "quiet room to work remotely" in Lisbon, p95 latency, and recall@10 on 200 queries.
- The migration has a 1,536-dimension column, a cosine HNSW index and a model-id column.
- The Lisbon example puts "peaceful, fast wifi, great desk" near the top.
- p95 around 20 ms and recall@10 around 0.97, measured, not claimed.
- It adds a separate vector database with a sync job when pgvector covers 4.2M vectors.
- It upgrades the embedding model for queries only.
- It reports "search works" from one query, with no latency or recall numbers.
Check yourself
2 minNext episode: duplicate-review detection flags "the room was clean" and "the room was not clean" as the same review, using a 0.8 threshold copied from a blog post.
Mark it complete to update your skill map.
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
Technical deep dive
Lina adds semantic review search
Make review search find reviews by meaning, for 4.2M reviews in 9 languages, filtered by city and rating, fast enough for a search box.
- 1Pick one embedding model
A multilingual model, 1,536 dimensions, used for both reviews and queries. Store its name and version with every vector.
Why: Each model has its own space. Mixing models silently breaks every distance.
- 2Size it before building
4.2M × 1,536 numbers × 4 bytes ≈ 26 GB of raw vectors, plus index overhead.
Why: Memory decides where the index can live. 26 GB fits in a 256 GB replica; 2 TB would not.
- 3Embed everything once
A batch job embeds all 4.2M reviews (a few hours, a few hundred euros at illustrative prices); new reviews are embedded on insert.
Why: Documents are embedded once; only queries are embedded per search, which keeps search cheap.
- 4Index for speed
Add an HNSW index with cosine distance on a read replica.
sqlCREATE EXTENSION IF NOT EXISTS vector; ALTER TABLE reviews ADD COLUMN embedding vector(1536); ALTER TABLE reviews ADD COLUMN embedding_model text; CREATE INDEX ON reviews USING hnsw (embedding vector_cosine_ops);Why: Exact search takes ~1.5 s per query; HNSW ~20 ms. The replica keeps vector load away from bookings.
- 5Check recall, then ship
On 500 sample queries, compare HNSW results with exact search: ~97% of the true top 10 found. Searches with no result: 38% → 4%.
Why: ANN is approximate by design. You measure how approximate before guests do.
| Option | What it is | Strengths | Costs / limits |
|---|---|---|---|
| pgvector | Postgres extension (HNSW, IVFFlat) | No new system; SQL filters and joins; never out of sync | Shares resources with other queries; very large indexes need lots of RAM |
| Pinecone | Fully managed vector database service | No operations; scales to large collections | Another vendor and data copy to sync; usage-based cost |
| Weaviate | Open-source vector database (self-host or managed) | Built-in hybrid search, multi-tenancy | Another cluster to run or pay for |
| Qdrant | Open-source vector database (self-host or managed) | Fast filtered search, quantisation | Another system to run and keep in sync |
Interactive simulation
Visual diagram
Semantic search over hotel reviews, with pgvector inside the existing Postgres.
Analogy
- A giant map of every sentenceFinding a sentence's coordinatesEmbeddingsEmbedding a text
- A giant map of every sentenceNeighbourhoods of similar topicsEmbeddingsClusters of similar meaning
- A giant map of every sentence"Show me what's near here"EmbeddingsSemantic search
- A giant map of every sentenceSame direction from the centreEmbeddingsCosine similarity
- A giant map of every sentenceHighways first, then local streetsEmbeddingsAn HNSW index
Where the analogy breaks: a real map has two dimensions. Embeddings have hundreds or thousands, and the fast index can occasionally miss a house on a side street. That's why it's called approximate.
Check your understanding
Practice ladder
Rank three reviews by hand
Estimation Warm-upA toy 2D embedding model gives:
- Query "family-friendly": (2, 1)
- Review A "great for kids, big pool": (4, 2)
- Review B "romantic, adults only": (−1, 3)
- Review C "spacious rooms for families": (3, 3)
Compute the cosine similarity of the query with each review, and rank them. Use cos = (a·b) / (|a| × |b|).
Finish the review search query
Code PracticeComplete Lina's pgvector query: the top 5 reviews similar to the query vector $1 for hotel $2, written in the last 12 months, with their similarity score. Fill in the three blanks.
SELECT r.id, r.text,
___ AS similarity -- (1) turn cosine distance into similarity
FROM reviews r
WHERE r.hotel_id = $2
AND ___ -- (2) last 12 months
AND r.embedding_model = 'reviews-embed-v1'
ORDER BY ___ -- (3) nearest first
LIMIT 5;Size the policy search
Estimation PracticeNext, the concierge needs to search hotel policy documents by meaning. 9,000 hotels × about 220 passages each. The embedding model has 1,024 dimensions (float32). Searches are always filtered by hotel. Expected load: 300 searches a second at peak. StayHub runs Postgres replicas with 256 GB RAM.
Estimate the raw vector size, decide where the vectors should live, and say what you'd measure before launch.
Upgrade the embedding model?
Decision StretchThe provider's embedding model v2 scores better on benchmarks. Upgrading means re-embedding all 4.2M reviews: about 6 hours of batch work and €400 at illustrative prices. Options:
- A. Re-embed into a new column, run 500 real queries on both, switch queries and documents together if v2 wins.
- B. Switch tonight: re-embed in place and switch queries when the job finishes.
- C. Stay on v1.
Write a short decision: "We chose X over Y because…, accepting the risk of…".
Explain it yourself
Explain to a backend engineer how “quiet room to work remotely” can find a review that says “peaceful, fast wifi, great desk”, and how you'd store and query it.