Hyperbolic CLIP embeddings help with compositional search but don't beat CLIP at broad search or classification

What we checked

• What concrete evidence would resolve the remaining editorial uncertainty for this item? partly confirmed. Most of the uncertainty is settled. The blog's SugarCrepe numbers (0.7954 vs 0.7306) match the hyper3-clip-v1 model card, and the Hyper3-CLIP paper and code exist. What remains open is that the ranking results come from a single vendor's runs, partly on a test the authors designed themselves, and nobody has replicated them independently. [1][2][3]
• Is the model Milvus tested (Hyper3-CLIP V1) the same model as the one in the Hyper3-CLIP paper? refuted. No. The Milvus post says it covers V1, not the paper's model. The GitHub repo says its released Hugging Face artifact is 'not the paper model' and that the paper checkpoint 'will be published'. Neither the blog nor the paper's own claims test the other. [1][4]
• Can hyperbolic (Lorentz) ranking run on a standard inner-product vector index without a custom metric? confirmed. Yes, according to part 1 of the series. If you negate the query's first ('height') coordinate, a plain inner-product search returns exactly the hyperbolic ranking; the largest difference measured was about 4e-14. Approximate indexes need an extra fix: subtract the corpus-mean height from documents at ingest. That fix does not help IVF_PQ. [5]

What we found

• Compositional matching is V1's clearest win, and the numbers check out. Milvus reports SugarCrepe scores of 0.7954 for V1 and 0.7306 for CLIP ViT-B/16, and says its harness reproduced the model card exactly. The model card lists 79.54% vs 73.06% macro accuracy, using cone scoring. [1][2]
• CLIP clearly wins zero-shot classification: 68.64% vs 51.06% top-1 on ImageNet. The authors point out a training-data mismatch: OpenAI's CLIP saw about 400M pairs, while the paper's baselines were trained from scratch on GRIT. The model card reports no ImageNet results. [1][2]
• The evidence on broad-term search conflicts. On Milvus's own 132-term ImageNet test, CLIP leads by 6.4 AP points (95% CI 2.1 to 10.8), mostly because V1 struggles with rare words. Changing 'weapon' to 'armament' dropped V1 from 79 to 0 AP. When Milvus reran hyper³labs' COCO test (three broad queries, 2,953 images), V1 led by 6.3–6.6 points with bare queries but only by 0.9–1.0 with a 'a photo of' prefix. [1]
• Holding the model fixed, Lorentz scoring beats cosine on top-1 accuracy (51.06% vs 50.14%, gain CI 0.72–1.11 points) and nudges up all five hierarchy metrics slightly. But it is 2.2 AP worse on broad-term search and loses on 97 of 132 terms. When cosine is wrong, its mistakes land slightly closer in the WordNet tree (5.598 vs 5.629 steps). In other words, this test did not show the expected hierarchy benefit from the geometry. [1]
• The model tested is not the paper's model. The Hyper3-CLIP paper (Mahmood et al., arXiv:2608.29313, ECCV 2026 Beyond Euclidean Workshop) reports gains on COCO/Flickr R@5/R@10 and calls its hierarchy-metric results 'competitive'. The GitHub repo, however, says the released Hugging Face artifact is 'not the paper model' and that the paper checkpoint will be published later. Milvus says its hierarchy comparison does not test the paper's claim. [3][4][1]
• Serving is straightforward. Part 1 shows that negating the query's height coordinate makes inner-product search exact for Lorentz ranking. Centering document heights brings HNSW (ef=128) recall@10 to 0.996 and IVF_FLAT from 0.900 to 0.990 on V1/ImageNet. Centering does not rescue IVF_PQ (0.115 raw, 0.096 centered). [5]

What it means

Don't adopt a hyperbolic CLIP model expecting general wins. Based on this evidence, Hyper3-CLIP V1 is worth an A/B test if your queries are descriptive captions or depend on how objects, attributes and relations combine ("a dog on a sofa"). It is not a drop-in replacement for OpenAI CLIP in zero-shot classification. For broad-category search it is unproven and sensitive to wording, so evaluate on your own vocabulary, including rare words and synonyms, before switching. Switching means re-encoding your whole collection. Once you have, you can compare Lorentz and cosine on the same vectors at little cost: Lorentz gave about one point of top-1 accuracy here, but cosine did better on broad-term search.

On the infrastructure side, you don't need a special database. Use the sign-flip trick with a centered height coordinate on HNSW or IVF_FLAT, and avoid PQ compression, where the ranking signal collapses. Treat every number above as one vendor's measurements on one checkpoint. The paper's model has not been released, so the hierarchy claims in the paper have not been tested by anyone outside the authors.

Still open

• Will the Hyper3-CLIP paper model, once its checkpoint is published, beat CLIP on broad-term retrieval or on hierarchy metrics?
• How would a hyperbolic model trained on data comparable to CLIP's ~400M pairs do on ImageNet zero-shot?
• Do the broad-term results hold on test sets not designed by Milvus or hyper³labs, and on vocabularies outside WordNet/ImageNet?
• Does height centering generalize to other Lorentz-form models such as MERU or HyCoCLIP and to other corpora? Milvus measured it only on V1 and one image collection.