Embedding models misalign with physical measurements across 24 models, study finds
A closer look at how AI representations relate to real-world quantities.
- Publication
- arXiv
- Stage
- Preprint
- What we read
- Summary of the paper
- Authors
- Juri Opitz, Andrianos Michail
- Universities and research institutions
- Not yet supplied in verified metadata; the Brief does not guess.
What they did and found
The study evaluated 24 common embedding models and visualized how they express physical units like meters, kilograms, liters, and seconds. Results show weak alignment with actual physical distances and a troubling pattern of irregularities, driven largely by surface text similarity rather than true numeric meaning.
Why it matters
Practically, relying on embeddings to reason about measurements can be unreliable; users should not assume numeric order or unit conversions are preserved. For example, similar-looking numbers may cluster together even if their real values differ.
Across many models, representations of units and measurements were weak and inconsistent, with patterns that don’t reliably track physical distance or size. The strongest alignments were not consistent across model types or sizes, suggesting limits to model-driven intuition about quantities.
A practical takeaway is that embedding-based tools may still help with text similarity, but they should not be trusted to preserve numerical semantics or unit conversions without explicit, targeted training or checks.
limitation-counting-this-study-only
Original sources · 1
- Embedding Models Measure in Peculiar Ways ↗arXiv · 2026-09-17
Check the original paper for its authors, methods, version and access terms.
What remains uncertain
The evidence comes from 24 models on specific benchmark setups; results may not generalize to all models or real-world tasks.
What is your take?
Ask a question, add useful context or share a different perspective. Keep the conversation respectful and grounded.