Skip to content
Episode 106 · Clean, not clean, 0.91

Similarity metrics: cosine, dot product and Euclidean distance

Tuesday, 9:05: the new duplicate-review detector hid 47% of last week's reviews. It thinks "the room was clean" and "the room was not clean" are the same review, thanks to a 0.8 threshold copied from a blog post.

There are three common ways to say how close two vectors are. They can disagree, and knowing when saves you from slow, wrong searches.

30 min intermediateCosine vs dot product vs EuclideanVector normalization (unit length)Similarity thresholds and calibrationCosine similarityEmbeddingsApproximate nearest neighbour indexes
  1. Problem

    Search, deduplication and RAG all reduce to one question: how similar are these two vectors, and is that similar enough?

  2. Why the simple system fails

    Teams pick a metric at random, mix normalized and unnormalized vectors, query with an operator the index wasn't built for, and copy a '0.8 means relevant' threshold from a blog post written for a different model.

  3. Engineering concept

    Cosine measures direction, dot product measures direction times length, Euclidean measures straight-line distance. On unit-length vectors all three rank identically. Thresholds are model-specific and must be calibrated on labelled data.

  4. Technology

    L2 normalization at write and query time, pgvector operators (<=>, <#>, <->) with matching index operator classes, score histograms on labelled pairs, rerankers for relevance decisions.

  5. Trade-offs

    Dot product is cheapest but only safe on normalized vectors or models trained for it; cosine is the safe default; Euclidean matters for some clustering and non-text embeddings. Fixed thresholds are simple but brittle.

01

The problem

2 min
Tuesday, 9:05. The duplicate-review detector has been live for a week.
  1. Omar · On-call SRE
    It should catch copy-paste review farms: about 3% of new reviews. Last week it hid 47%.
  2. Maya · Product manager
    It hid "The room was not clean" as a duplicate of "The room was clean".
  3. You · Engineer at StayHub
    The rule hides a review if cosine ≥ 0.8 with an earlier one. I found 0.8 in a blog post.
  4. Lina · Senior engineer, your mentor
    Guess first: what's wrong with it?
02

The big idea

4 min
What a similarity score really says
The big idea

A similarity score isn't a probability: a threshold belongs to one model and one kind of data, so you measure it on labelled pairs, never copy it.

Cosine was never the problem. With our model, unrelated reviews of the same hotel score around 0.71, so the blog's 0.8 sits inside the "unrelated" range. Embeddings capture what a text is about far more than a single "not".

03

How it works

5 min

An embedding is a list of numbers (a vector). To ask "how alike are two texts?" you compare their vectors. There are three common ways, and two rules that keep them honest.

Comparing vectors, from metric to threshold
  1. 1
    Cosine

    compares direction only: 1 means same direction, 0 unrelated, −1 opposite. The safe default for text.

  2. 2
    Dot product

    The is direction × both lengths. On raw vectors, long vectors score higher whatever they mean.

  3. 3
    Euclidean distance

    is the straight-line gap between two points. Smaller means more similar.

  4. 4
    Normalize

    Normalization scales every vector to length 1 (). Then all three metrics give the same ranking. Do it when you store and when you query.

  5. 5
    Calibrate the threshold

    A is the score above which you act. Score labelled pairs from your own data and pick the number that meets your goal: high precision for automatic actions, high recall for suggestions.

  6. 6
    Match operator and index

    In pgvector each metric has its own operator: <=> cosine, <-> Euclidean, <#> inner product. An index built with vector_cosine_ops is only used by <=>. Any other operator scans every row.

04

Watch a decision being made

4 min
Worked example

Lina fixes the duplicate detector's threshold

The detector hides 47% of new reviews with a copied 0.8 threshold. Make it hide only true copy-paste duplicates, with a safe path for borderline cases.

  1. 1Label real pairs

    "Before any number, data." The moderation team labels 800 pairs of reviews from the same hotel, duplicate or not, including tricky ones like clean / not clean.

    Why: A threshold is a property of our model on our data. Only labelled pairs from that data can set it.

  2. 2Read the two distributions

    Not duplicates: 0.62–0.80 (median 0.71), with a tail of negation pairs up to 0.92. True duplicates: 0.93–0.99.

    Why: Where the two groups overlap tells you which mistakes any threshold will make.

  3. 3Pick the number for the goal

    "Hiding a genuine complaint is the costly error." So aim for 98% precision: auto-hide at 0.94. Scores 0.90–0.94 go to a moderator.

    Why: Auto-actions need high precision; a borderline band gets human judgement.

  4. 4Version it with the model

    Store threshold = 0.94 next to the model id, and recalibrate on every model change.

    text
    embedding_model: v3   duplicate_threshold: 0.94   review_band: 0.90-0.94

    Why: A new model moves the whole score range; an untied threshold breaks silently.

The whole thing in one line: Hidden reviews went from 47% to 3.4%. The fix was not a better metric but a measured threshold, a human band, and a rule to recalibrate.
05

In real life

3 min
Acting on "these two are the same"

YouTube Content ID: act automatically

  • Matches uploads against rights holders' files and applies their policy (block, track or monetise)
  • Volume far too large for humans to check each match
  • Mistakes are handled afterwards through disputes

Fits huge volume where wrong matches can be appealed.

Google Photos: ask when unsure

  • Groups photos of the same face, and sometimes asks "same or different person?"
  • Borderline cases go to the owner
  • Slower, but avoids confident mistakes

Fits when a wrong merge is costly and a person is available.

06

What goes wrong

3 min
Copying a threshold from a blog or another model

Each model spreads scores differently; 0.8 can mean "identical" or "same hotel".

Instead

Calibrate on a few hundred labelled pairs and store the threshold with the model id.

Ordering by an operator the index wasn't built for

Same results, full scan, latency explodes.

Instead

Match operator and index class, and check vector queries with EXPLAIN.

Reading similarity as agreement

Embeddings capture topic more than negation: clean / not clean scores 0.91.

Instead

Use similarity for "same topic"; add a reranker or classifier when meaning details matter.

07

Trade-offs

3 min
ChoiceFits whenYou pay
CosineText embeddings, unsure about normalizationSlightly more work per comparison
Dot product on unit vectorsYou normalize everywhere, or the model was trained for itWrong rankings if one vector isn't normalized
Calibrated threshold + human bandA wrong automatic action is costlyLabelled data, recalibration, a queue to staff
Friday, 10:00. Setting the final threshold.
  1. Maya · Product manager
    Review farms are real. Auto-hide at 0.90 and catch more fakes; moderators can restore mistakes.
  2. Omar · On-call SRE
    0.90 hides complaints like "not clean". Auto-hide at 0.94, send 0.90–0.94 to moderators.
  3. Maya · Product manager
    Then more fakes stay visible until someone looks.
08

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own
  • One metric for the whole system (cosine for reviews), in one shared constant, so index, queries and thresholds can't drift apart.
  • Every threshold comes from labelled pairs for the current model (800 pairs → 0.94 for 98% precision), stored with the model id.
  • What a match does: auto-hide, or go to a moderator first.
Tell Claude Code
Our duplicate-review detector hid 47% of new reviews; we expected about 3%. Its 0.80 cosine threshold came from a blog about another model. Using the 800 labelled pairs in `data/duplicate-pairs.csv`, find the lowest threshold with 98% precision and show the score ranges for unrelated pairs and true duplicates. Store it in config with the model id. Don't touch the HNSW index or the `<=>` operator. Until I approve, send matches to moderation instead of hiding them.
Check before you accept
  • The calibration report shows both score ranges, the chosen threshold (about 0.94), its precision and the share flagged (about 3.4%).
  • EXPLAIN on the search query shows the HNSW index, not a sequential scan.
  • "The room was clean" vs "The room was not clean" is not hidden.
Red flags
  • A new threshold picked by feel ("0.90 looks better") with no labelled pairs or precision number.
  • The ORDER BY operator changed "for speed" without checking the query plan.
  • A cosine score described as a probability ("0.8 = 80% likely").
09

Check yourself

2 min
Q1/3Scenario
After switching embedding models, your 'duplicate if cosine ≥ 0.80' rule flags 47% of new reviews instead of about 3%. What should you do?
How confident are you?

Next episode: the concierge's knowledge index grows to 60 million vectors that must be filtered per hotel and fresh within minutes. Karim asks whether Postgres can still carry it.

Finished?

Mark it complete to update your skill map.

Next: Choosing a vector database
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
10

Technical deep dive

Three metrics, one index

Three metrics, three rankings. Query q = (3, 4), length 5. Stored: A = (6, 8) (same direction, length 10), B = (4, 3) (slightly off, length 5), C = (0.6, 0.8) (same direction, length 1).

  • A: dot 50, cosine 1.00, distance 5.00
  • B: dot 24, cosine 0.96, distance 1.41
  • C: dot 5, cosine 1.00, distance 4.00

Dot product ranks A > B > C (C is last only because it's short). Cosine: A = C > B. Euclidean: B > C > A (A is last only because it's far). Normalize everything to length 1 and all three agree: A = C > B, since on unit vectors dot = cosine and Euclidean² = 2 − 2 × cosine.

Worked example

You fix the duplicate detector

The detector hides 47% of new reviews with a copied 0.8 threshold. Make it hide only true copy-paste duplicates, with a clear path for borderline cases. Lina asks one question per step.

  1. 1Normalize on both sides

    Lina: "Are our vectors unit length?" You check: the model returns unit vectors, but the old import didn't re-normalize after averaging. Normalize at write time and at query time.

    ts
    const normalize = (a: number[]) => {
  const n = Math.sqrt(a.reduce((s, x) => s + x * x, 0));
  return n === 0 ? a : a.map((x) => x / n);
};

    Why: On unit vectors the three metrics agree, so a later operator choice can't change the ranking.

  2. 2Label real pairs

    The moderation team labels 800 pairs of reviews from the same hotel: duplicate or not, including tricky ones like "clean" / "not clean".

    Why: A threshold is a property of our model on our data. Only labelled pairs from that data can set it.

  3. 3Look at the two distributions

    Score all 800. Not duplicates: 0.62–0.80 (median 0.71), with a tail of negation pairs up to 0.92. True duplicates: 0.93–0.99.

    Why: Seeing where the two groups overlap tells you which errors any threshold will make.

  4. 4Pick the threshold for the goal

    Hiding a genuine review is the costly error, so aim for 98% precision: threshold 0.94 for auto-hide. Scores 0.90–0.94 go to a moderator.

    python
    for t in np.arange(0.85, 0.99, 0.01):
    flagged = scores >= t
    precision = labels[flagged].mean()
    print(round(t, 2), round(precision, 3), flagged.mean())

    Why: Different goals need different thresholds: auto-hiding wants high precision; "suggest similar reviews" would want high recall.

  5. 5Version it with the model

    Store threshold = 0.94 next to the model id. Re-run the calibration on every model change. Result: 3.4% of new reviews hidden, and no genuine complaint in a week's sample of 200.

    Why: Swapping the model moves the whole score range; a threshold that isn't tied to its model breaks silently.

The whole thing in one line: You normalized the vectors, labelled 800 real pairs, read the two score distributions, chose 0.94 for 98% precision with a moderator band below it, and tied the threshold to the model: hidden reviews went from 47% to 3.4%.
MetricMeasuresUse whenWatch out
Cosine similarityAngle onlyDefault for text embeddingspgvector's <=> is a distance (1 − cosine); sort the right way
Dot productAngle × both lengthsUnit vectors (fastest, equals cosine), or models trained for itOn raw vectors, long vectors win regardless of meaning
Euclidean (L2)Straight-line distanceClustering, models trained with L2Matches cosine's ranking only after normalization
Use the metric the embedding model was trained for. With unit vectors, any of the three gives the same ranking, but each still needs its own index in pgvector.
11

Interactive simulation

12

Visual diagram

Click a component

Where the metric choice lives in a semantic search pipeline, and the three places it must agree: the model, the stored vectors and the index.

13

Analogy

  • Three hikers leaving a hotelAna walks 2 km north-eastSimilarity metricsA short vector
  • Three hikers leaving a hotelBen walks 10 km north-eastSimilarity metricsA long vector, same direction
  • Three hikers leaving a hotelComparing compass bearings: Ana = BenSimilarity metricsCosine similarity
  • Three hikers leaving a hotelAna is far from Ben on the groundSimilarity metricsEuclidean distance
  • Three hikers leaving a hotelBen "agrees" most because he went farSimilarity metricsDot product favours long vectors
  • Three hikers leaving a hotelEveryone walks exactly 1 kmSimilarity metricsNormalised vectors: all three agree

Where the analogy breaks: hikers move in two dimensions. Embeddings have hundreds, where nearly everything feels far apart, so scores bunch together in a narrow band.

14

Check your understanding

Q1/3Reasoning
Two unit-length vectors have cosine similarity 0.82. What is the Euclidean distance between them?
How confident are you?
15

Practice ladder

Step 1 · Trace

Three metrics by hand

Estimation Warm-up

Query q = (1, 0). Stored vectors:

  • P = (2, 0)
  • Q = (0.8, 0.6)
  • R = (0.5, 0)

Compute the dot product, cosine similarity and Euclidean distance between q and each one, and give the ranking (best first) for each metric. Then say what changes if P and R are normalized first.

Step 2 · Complete

Match the operator to the index

Code Practice

The team wants to use the dot product for speed (vectors are normalized). Complete the migration and the query so the new index is actually used, and the score shown to the UI is a similarity where higher is better.

-- 1. Build an index for inner product
CREATE INDEX reviews_emb_ip ON reviews USING hnsw (embedding ___);

-- 2. Query: top 10, with a similarity score (higher = better)
SELECT id, ___ AS score
FROM reviews
WHERE hotel_id = $2
ORDER BY ___
LIMIT 10;
Step 3 · Solve

Calibrate the concierge's "no answer" threshold

Estimation Practice

The concierge searches hotel policies and should say "I'm not sure, let me check" when nothing relevant is found. On 400 labelled question–passage pairs, the best passage's cosine score was:

score bandrelevantnot relevant
≥ 0.851505
0.80–0.856025
0.75–0.802070
< 0.75565

Pick a threshold for "answer vs say I'm not sure", explain the trade-off, and estimate precision and recall at your choice.

Step 4 · Decide

A semantic cache threshold

Decision Stretch

The concierge's semantic cache (generic FAQs per hotel) returns a stored answer when a new question's cosine with a cached question is above a threshold. On 600 labelled pairs ("same question?"):

  • At 0.92: 99% precision, 40% of repeat questions served from cache.
  • At 0.88: 95% precision, 65% served.
  • "Is breakfast included?" vs "Is breakfast included for kids?" scores 0.90.

Write a short decision: "We chose X over Y because…, accepting the risk of…".

16

Explain it yourself

Explain to a teammate why 'we use dot product because it's fastest, and 0.8 means similar' could be wrong, and what you'd do instead.

0 words