Hakken is a 66-page arXiv preprint from Sony AI researchers, submitted on 3 September 2026. The 23 authors include Tarek R. Besold, Peter Stone, Hiroaki Kitano and Michael Spranger. The paper describes a domain-agnostic system for what it calls "knowledge prediction": predicting relationships between scientific concepts that are not yet documented, and explaining each prediction. It combines three parts. THiGERLLM is a transformer trained on temporal sequences of knowledge graphs mined from publications and fused with an LLM's semantic knowledge. PHELInE is an explanation layer that surfaces supporting literature paths. A GUI lets scientists review the hypotheses. In biomedicine the authors report a new benchmark for time-aware multi-label relation prediction, about 1.5 million aging-related hypotheses above a confidence threshold, and two wet-lab confirmations.
This matters to builders because "AI that proposes discoveries" is now a crowded claim. Hakken is one of the more complete attempts: it has a temporal benchmark, explanations, released code and lab follow-up. That makes it a useful reference for how to evaluate such systems, and for how far the evidence actually goes. The idea itself is not new. Literature-based discovery and concept-network forecasting (for example Krenn et al.'s Science4Cast) came first, so the real question is what Hakken adds.
What we checked
- If Hakken's mechanism generalizes beyond the reported setting, do comparable primary studies under different models or tasks show the same direction? partly confirmed. Yes for the basic direction. Predicting future links from time-sliced concept networks beats naive baselines in AI research (Science4Cast, ten methods) and in OpenAlex technology and biomedical networks (LightGBM, ROC-AUC 0.954–0.967). However, those studies find that hand-crafted or structural features are what drive performance, not learned end-to-end or LLM signals. No independent study has replicated Hakken's own LLM-fusion model. [4][5][1]
- (Added) Does fusing LLM semantic knowledge improve on the authors' KG-only model? partly confirmed. Only on some metrics. THiGERLLM beats THiGER on macro recall (60.72 vs 46.13) and macro F1 (51.16 vs 50.98). It loses on macro precision (53.77 vs 60.20), weighted F1 (73.73 vs 77.97) and nDCG (90.30 vs 92.19). [1][2]
- (Added) Do the wet-lab results establish a real-world hit rate for Hakken's hypotheses? refuted. No. Experts picked three hypotheses out of roughly 1.54 million and tested them in vitro at a contract research organization. Two were supported (TP53→BAMBI, RAF1→TNF) and one was not (SOAT1→STAT3). The failed hypothesis had a higher confidence score than one of the successes, so the scores did not predict which would hold up. [1][2]
What we found
- The knowledge graph is built from PMC Open Access full texts, MEDLINE abstracts and a commercially licensed dataset. After cleaning it has 254,806 entities, 7,127,960 triples, 23 relation types and 20 macro-domains, down from roughly 249 million raw triples. The benchmark trains on relations first seen on or before 2020 and tests on relations first seen afterwards. Back-tests use cutoffs of 1990, 2000 and 2010. [1]
- On the 2020-cutoff benchmark, both of the authors' models clearly beat the external baselines. For example, TNodeEmbed reaches a weighted F1 of 63.47 and the MLP 61.52, against 73.73–77.97 for the THiGER models. The LLM-fused THiGERLLM, however, beats the KG-only THiGER only on macro recall and macro F1. It trails on precision, weighted F1 and nDCG, so the paper does not show a clear overall gain from adding the LLM. [1][2]
- The authors list several limitations themselves. Precision is 'fundamentally ill-defined' in this temporal setting, because a prediction not yet in the literature may simply be undiscovered rather than wrong. Residual temporal leakage 'cannot be fully excluded' because the LLM's pretraining cutoff is unspecified. The graph also lacks quantitative or conditional information and records only the first occurrence of each fact. [1]
- Three expert-selected hypotheses were tested in vitro at a contract research organization. TP53→BAMBI and RAF1→TNF were supported; SOAT1→STAT3 showed no effect at the transcript or protein level. The reported confidence scores (0.854, 0.831 and 0.842 respectively) did not separate the successes from the failure, and an independent write-up stresses that three human-picked tests say nothing about the system's overall hit rate. [1][2]
- Sony released HakkenOSS under the BSD 3-Clause license. It includes a Neo4j and PostgreSQL pipeline, knowledge-graph-embedding models (ComplEx, DistMult, RotatE, GNN), agents and a React UI. The paper says the open model is trained only on PubTator3 data, while the full-scale model depends on licensed data that is not released. The license for the model weights is marked as pending management confirmation. The repo is small: 6 commits at the time it was checked. [3][1]
- Prior and parallel work points the same way, but credits simpler features. Science4Cast, built on a network of about 64,000 AI concepts, found that the best forecasters used curated network features rather than end-to-end learning. A June 2026 OpenAlex study reached a ROC-AUC of 0.954–0.967 across four domains using 59 auditable features and LightGBM. Adamic-Adar similarity and degree-based features drove most of the accuracy. [4][5]
What it means
For builders of scientific-discovery systems, Hakken is a good evaluation template but weak proof that adding an LLM pays off. The parts worth copying are the temporal holdout (train on facts before a cutoff, score on facts that appear later), back-testing at several cutoffs, and explanations built from paths through the literature. Before attributing gains to an LLM, compare against strong structural baselines such as Adamic-Adar and gradient-boosted features: in Hakken's own table the KG-only model wins most metrics. If any component has an unknown pretraining cutoff, treat leakage as a live risk and report it, as the authors do. Report precision only as a lower bound, because a missing fact may simply not have been discovered yet.
For anyone weighing Hakken-style tools for R&D triage, the claim of 1.5 million hypotheses is a measure of volume, not a validated hit rate. The lab evidence is two successes out of three hand-picked in vitro tests, with confidence scores that did not tell the success and failure apart. The usable workflow is: rank candidates, read the explanation paths, then spend lab budget sparingly. The open-source release lets you rebuild the pipeline on PubTator3. It will not reproduce the paper's full-scale results, which depend on licensed data.
Still open
- Is Hakken better calibrated or more precise than structural-feature baselines such as Adamic-Adar or LightGBM on the same temporal biomedical split?
- How much of THiGERLLM's recall gain comes from leakage, given that the LLM's pretraining cutoff is unspecified?
- What is the hit rate on a larger, blinded, randomly sampled set of above-threshold hypotheses?
- How well does the PubTator3-only open model perform compared with the licensed-data model?