Skip to content

feat: graphrag adaptive hybrid retrieval - #883

Open
aotenjou wants to merge 23 commits into
apache:masterfrom
aotenjou:codex/graphrag-adaptive-hybrid-retrieval
Open

aotenjou wants to merge 23 commits into
apache:masterfrom
aotenjou:codex/graphrag-adaptive-hybrid-retrieval

Conversation

@aotenjou

@aotenjou aotenjou commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

adaptive graph retrieval

Summary

This PR adds an end-to-end, budget-aware GraphRAG retrieval system to geaflow-ai. It combines lexical, vector, and graph retrieval behind a unified evidence model and Java API, with adaptive routing, bounded execution, provenance tracing, and extractive answer support.

Design

The system covers the full retrieval workflow:

  • Data preparation: Deterministic ingestion, normalization, chunking, entity extraction, and versioned manifests produce reproducible retrieval artifacts without committing dataset contents.
  • Retrieval channels: BM25, vector, and graph channels return evidence with stable identities and source provenance. Graph retrieval anchors on resolved entities and performs constrained expansion of at most two hops.
  • Execution and routing: A RoutingContext feeds an AdaptivePolicy, which produces a validated RetrievalPlan for the executor. The system supports BM25-only, vector-only, graph-only, fixed hybrid, and adaptive hybrid modes, with sequential, parallel, and cascaded execution.
  • Budget enforcement: Candidate, latency, token, and graph edge-scan costs are tracked. Graph expansion uses deterministic frontier sampling and bounded replanning, with stopping conditions for exhausted budgets, excessive growth, and low evidence yield.
  • Fusion and response: Channel results are deduplicated and combined using weighted reciprocal rank fusion, then reranked and truncated to the requested output budget. Responses include evidence, execution decisions, stage metrics, budget usage, and retrieval traces.
  • Answer provenance: Extractive answers include citations linked back to the supporting evidence and original sources.
  • Reproducibility and evaluation: Repository tooling supports offline artifact generation, batch retrieval, benchmarking, and evaluation across the five retrieval modes using consistent data splits and result protocols.

The retrieval policy and budget accounting remain in Java as the behavioral source of truth. The Python tools prepare data and run reproducible evaluation workflows; they are not runtime dependencies of the Java service.

Validation

Feature file
Evidence/Provenance/version controll model/RetrievalDomainModelTest(19)、model/RetrievalProvenanceContractTest(17)、answer/CitationProvenanceTest(6)
data ingestion/artifact building ingest/IngestionPipelineTest(7)、ingest/CanonicalJsonlArtifactBridgeTest(5)、index/ArtifactPublicationTest(3)、index/graph/GraphArtifactTest(14)、metadata/*(23)
Logical graph channel/graph/VersionedGraphViewTest(11)、channel/graph/VersionedGraphViewLazinessTest(4)
discipline 2 hop graph retrieval channel/graph/GraphRetrieverTest(19)、channel/graph/GraphRetrieverContinuationTest(4)、channel/graph/AssociationScoreFormulaTest(10)、channel/graph/EdgeStrengthTimeDecayTest(13)、channel/graph/EntityAnchorResolverTest(4)、service/EdgeScanBudgetTest(4)
execution mode/RRF/final sort execution/ExecutionBudgetTest(9)、service/BudgetMonotonicityTest(4)、execution/RecallPlanAllocationTest(5)、fusion/WeightedRrfFusionTest(5)、service/RetrievalModeContractTest(3)
adaptive route/replan execution/AdaptiveRoutingTest(18)、execution/AdaptivePolicyContractTest(6)、execution/ReplanBoundTest(4)、execution/AdaptiveGraphFeedbackTest(6)、api/RetrievalPropertiesTest(18)
MVP service/GraphRagMvpEndToEndTest(44)、service/RetrievalServiceTest(5)、service/ReadinessReportTest(4)、service/RetrievalBatchCliTest(5)、api/RetrievalApiJsonTest(18)
benchmark evaluation evaluation/EvaluationMetricsTest(3)、evaluation/RelationPathGoldenTest(4)、evaluation/AnswerMetricsGoldenTest(1)、service/RetrievalMetricsTest(1)
Answer / context API answer/AnswerConsumerTest(10)、service/RetrievalExtractiveAnswerTest(6)、answer/AnswerConsumerStressTest(1)
release check service/ApiUsageExampleTest(5)、service/RetrievalDeterminismTest(4)

Both text channels and the graph channel returned candidates in the order they
happened to be visited, and fusion turns that position into a reciprocal-rank
term. The resulting ranks were unrelated to relevance, which made every hybrid
method's ranking depend on index layout rather than on the query.

BM25 and vector truncated by traversal prefix. Querying a candidate limit of 100
scored the first 100 documents in index order - which is chunkId order, because
both builders insert in that order - and then sorted only those. Every query
therefore returned the alphabetically first matches. On HotpotQA this put 98.5%
of BM25 top-10 hits inside the first 100 of 41976 chunkIds, and measured recall
sat at chance level (0.0022 for BM25, 0.0000 for vector).

Both readers now score the whole matching set and retain the best `candidateLimit`
of it. The limit bounds the retained set, not the traversal; scoring work stays
visible in the evaluated counters.

Graph hits were returned in expansion order, whose second sort key is a hash used
for deterministic sampling. `finish()` now orders by association score with a
path-based tie-break, so the graph channel's rank means what fusion assumes it
means.

Separating retained candidates from scored documents broke two accounting
assumptions that had silently equated them: the candidate budget was charged per
scored document, letting one channel consume a whole request's allocation and
starve the graph stage, and the response validator rejected a retained count that
exceeded the channel budget. Both now track candidate output, with scoring work
reported separately.

The graph quota that preserves graph evidence in the fused window demanded every
returned position carry graph evidence, evicting higher-scoring text candidates.
It is now `retrieval.graph-fusion.graph-quota-share` (default 0.34), clamped below
the window so the quota can never outrank the fused order.

Exact ranking becomes the benchmark default, since the bounded path exists to cap
scoring work and measuring it silently reported text retrieval at chance level. A
`textChannelBoundScored` column flags a run that scored exactly its candidate
limit, so a capped run cannot be mistaken for a quality measurement again.

HotpotQA, 500 queries, topK 10, maxCandidates 100:

  BM25_ONLY      recall 0.0022 -> 0.9673
  VECTOR_ONLY    recall 0.0000 -> 0.8197
  FIXED_HYBRID   recall 0.8385 -> 0.9460, MRR 0.7425 -> 0.8008
  ADAPTIVE       recall 0.9082 -> 0.9525, MRR 0.6886 -> 0.6970

2Wiki is largely unchanged because its graph channel rarely saturates the quota.

Six existing assertions encoded the defects and are updated with them: two
expected the traversal-prefix winner, two treated scoring as the budgeted
quantity, and two expected the graph stage to be starved of budget. New tests
pin the fixes: a high-scoring chunk stored under a late chunkId must still be
retrieved, the graph channel must rank by association score, and hop-split
expansion must return the same order as a single call.

Verification: 403 retrieval tests and 114 Python tests pass, Checkstyle clean.
The maintainers asked for three things the previous material could not answer: how to
reproduce the numbers without the corpora, what the cost side looks like next to quality,
and which part of the adaptive decision actually does anything.

Re-measuring first was unavoidable. The earlier 500-query runs were scored against a
chunking that no longer exists in the bundle: the same query returns different evidence
ids (0009eeb4... vs 2db9ae3b...), and only graph-only kept its value (0.5028 to 0.5030)
because it barely touches the text index. Every other method moved by two orders of
magnitude once the text channels were scored against the corpus that is actually shipped.
The three documents report only the re-measured figures and mark the old ones void.

What the new numbers say, including the parts that do not flatter the design:

* Adaptive does not beat fixed hybrid. MRR is 6.8% lower on 2Wiki and 13.0% lower on
  HotpotQA. It beats graph-only by 36-150%, but graph-only is the weakest non-degenerate
  method.
* F1 is 0.002-0.010 and EM is zero for every method. The answer layer is extractive and
  the gold answers are short entities, so a span extractor rarely reproduces them. The
  lowest-recall method has the highest F1, which is why F1 cannot rank these methods.
* The cost saving exists on 2Wiki only (34% fewer edges); on HotpotQA adaptive spends 65%
  more edges than graph-only.

A component ablation was added to explain the routing gap. Neutralising the graph
weighting raises MRR from 0.6796 to 0.7562 (+11.3%), so the policy's defining behaviour
is the cause: it weighs the graph eight times higher than text while the text channels
retrieve better on both corpora. Three of seven rules never fire, replanning executes
1.21 times per request and changes no outcome, and two more rules cannot fire at their
thresholds. Those are recorded as open problems rather than papered over.

The benchmark now also reports the graph stage's own token estimate and the run total,
because a per-request mean cannot be compared against a per-token price.
The headline tables were retyped by hand from the run CSVs and drifted in the last
digit of several cells: 2Wiki BM25 MRR appeared as 0.7367 instead of 0.7368, HotpotQA
BM25 as 0.8253 instead of 0.8254, and eight more. The deltas were computed from the
correct values, so the two disagreed.

Every quality figure in all three documents is now checked against the CSVs it cites:
48 values across six methods and two datasets, plus the ten ablation rows. A script
compares the rendered tables to the source and reports zero mismatches.
The guide quoted 455 Java tests and 114 Python tests; the suites now run 463 and 115
after the ablation and token-column guards were added. A reader checking the first
example command against a fresh run would have seen a mismatch.
The review was raised in Chinese and the figures need to be checked by reviewers who
work in Chinese. The English documents stay canonical; these are translations of the
same content, not a separate analysis.

Every number was re-checked against the run CSVs after translation: 112 values across
the two metric tables and the ablation table, all matching. Wording that carries a
conclusion - "adaptive does not beat fixed hybrid", "the graph weighting is the cause" -
is translated rather than softened.
…ve setting

The maintainer asked whether a configuration exists where the adaptive channel is the
best performer with the router in place. The component ablation identified the graph
preference as the cause of the gap; this sweep answers the follow-up with measurement
instead of argument.

A declared 6-cell grid over graph-weight x text-weight, each cell restarting the service
with exactly one pair of weights and running the same 200-query slice. The shipped pair
(2.0 / 0.25) is the worst cell. The best cell (0.25 / 1.0) reaches MRR 0.7676 against
fixed hybrid's 0.7506, a +2.3% improvement, while scanning 13.0% fewer edges.

The grid is declared up front so this is a search rather than a cherry-pick, and the
result is reported as a candidate to validate, not a new default: it comes from one
dataset, one slice, with no held-out confirmation.

Also hardens ablate_adaptive.sh against a zero-byte classpath left by an interrupted
build, which previously failed later with a confusing NoClassDefFoundError.
Replanning was a no-op by construction: once the replan budget was spent the policy
fell back to the pre-execution feature snapshot, so the post-hop boundary re-planned
from graphGrowthRate=0 and never stopped a frontier that had already grown past the
limit. Measured features are now always merged, a spent budget may only contract the
plan, and the post-hop boundary keeps the completed text channels' allocation instead
of rolling it back, which turns the previous plan round trip into a real stop.

Routing defaults are re-derived from measurement rather than kept as a graph-first
prior. A 200-query sweep per dataset puts the optimum at a graph:text ratio of 1:4;
the shipped 8:1 was the worst cell measured, costing 7.8% MRR on 2Wiki and 15.0% on
HotpotQA against fixed hybrid. Defaults are now graph-weight=0.25, text-weight=1.0.
The lexical/semantic branches no longer hard-code a graph weight of 1.0, which would
have quadrupled it whenever they fired, and high-average-idf moves from 1.5 - below
the measured minimum, hence no discriminating power - to 4.0.

max-hop is configurable and defaults to 1: the retriever already stops itself on
diffusion growth, so the planned second hop never executed. Multi-hop remains
available by setting it back to 2. decouple-text-routing is added behind a flag that
stays off: enabling it lowered recall on both datasets, so the lexical/semantic
signals are recorded as unhelpful rather than enabled by default.

The evaluation pipeline now refuses to score a response against a corpus the service
is not serving. Gold resolution is checked before any request, and the first returned
evidence id absent from --chunks aborts the run with an actionable message. This is
the failure that produced the earlier 0.002 text recall and the wrong fusion prior.

Verification: 470 Maven tests and 118 pytest tests pass. Sweeps and attributions are
reproducible through tools/graphrag/sweep_fusion.sh and ablate_adaptive.sh, which now
takes --max-candidates so its halving ablation can match the 500-query budget.
The offline dependency:build-classpath goal can write a zero-byte file; the service then
failed at startup with NoClassDefFoundError. reproduce.sh now reuses GEAFLOW_CLASSPATH when
provided and stops with an actionable message otherwise, matching the ablation and sweep
runners.
Three documents, one per review point, under tools/graphrag/docs.

external-reproduction.md gives the upstream dataset links, versions and SHA-256 checksums
instead of shipping corpora, pins the commit the figures were produced from, and lists the
checks that distinguish a broken reproduction from a real result, including the enforced
corpus-namespace and gold-resolution guards.

performance-metrics.md reports cost next to quality: hit rate at three strictness levels,
F1 and EM, retained and graph-estimated tokens, an explicitly parameterised LLM cost
example, edge scans and latency. Adaptive deltas are given against every single-channel
baseline and against fixed hybrid, with negative values reported as measured.

adaptive-module-analysis.md contains the full component ablation, a measured attribution
for each condition that shows no difference, and the decision module's limitations as open
problems for follow-up work.

All figures come from the 500-query runs of commit fee1608 and are rendered from the run
CSVs rather than transcribed; the eight adaptive MRR deltas were checked arithmetically.
Non-adaptive methods are bit-identical to the previous runs, confirming that the measured
gains come from the routing changes alone.
@yaozhongq
yaozhongq requested a balanced review from Copilot November 13, 2026 09:30
@yaozhongq yaozhongq self-assigned this Nov 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

@yaozhongq

Copy link
Copy Markdown
Contributor

The core implementation, metrics, ablation experiments, and failure analysis have improved significantly. However, this is not ready for acceptance under the independent reproducibility requirement.

reproduce.sh currently uses GEAFLOW_VECTORS for both chunk-vector JSONL and query-vector NPZ files, which are incompatible formats. The query vectors also lack a download URL or complete generation instructions, while --skip-build requires a prebuilt runnable index package.

Please address the following:

  • Separate chunk-vector and query-vector inputs.
  • Document complete vector-generation steps, including the exact model version.
  • Complete the Python dependencies.
  • Fix the current Java test failure and update the test claim.
  • Resolve the dataset license status.
  • Report the path-hit metric and complete the HotpotQA component ablation.
    The datasets do not need to be included, but a reviewer should be able to download them from the documented sources and reproduce the build and evaluation by following the repository instructions.

@yaozhongq yaozhongq removed their assignment Nov 16, 2026
@yaozhongq
yaozhongq self-requested a review November 16, 2026 02:41
Six review points addressed.

Separate the two vector inputs. Chunk embeddings (JSONL, one vector per chunk) build the
index; query embeddings (NPZ, ids plus vectors) drive the benchmark. reproduce.sh used
GEAFLOW_VECTORS for both, so the publish stage could be handed an NPZ that the bridge reads
as JSONL lines. GEAFLOW_CHUNK_VECTORS and GEAFLOW_QUERY_VECTORS are now distinct, with
GEAFLOW_VECTORS kept only as a deprecated query-vector alias.

Document vector generation end to end: model iic/nlp_gte_sentence-embedding_english-small,
vector_version v1.0.0, ModelScope source link, 384 dimensions, and the exact embed_chunks /
embed_queries commands. The full pipeline is staged build_release -> embed -> publish ->
benchmark, so no prebuilt index package is required. Verified end to end on a 12-query real
subset: ingest produced 171 vertices and 52 chunks, the bundle published READY, and every
method answered 12/12 queries with the vector channel evaluating all 52 chunk vectors.

Complete the Python dependencies: requirements.txt for benchmarking and tests,
requirements-ingest.txt for parquet ingestion, requirements-embedding.txt for regeneration.

Fix the default build. Run artifacts were landing in module directories because Maven
resolves a relative -Dmdep.outputFile against the module, which also left the intended
classpath empty. Output directories are now absolute, and the root RAT excludes cover
**/tmp/** so generated output no longer fails the licence check. mvn -o -pl geaflow-ai test
passes without skipping checkstyle, RAT or jacoco: 470 tests, 0 failures; 118 pytest tests.

Resolve dataset licence status. 2WikiMultiHopQA is Apache-2.0 per the upstream repository
LICENSE, HotpotQA is CC BY-SA 4.0 per its project page; both manifests now record
confirmed-upstream with the evidence, instead of pending-review.

Report relationPathHit next to recall and explain why it is near zero while paths are
returned: expansion yields entity neighbourhoods, not gold reasoning chains. Complete the
HotpotQA component ablation over all eight conditions. It adds a finding 2Wiki does not
show: zeroing the graph rank weight improves HotpotQA MRR by 6.5%, whereas the same change
costs 0.39% on 2Wiki, so no single global fusion ratio suits both corpora.
@aotenjou

aotenjou commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor Author

感谢详细的评审意见,六点均已在 5d58d77 中修复。

  • 向量输入已拆分:GEAFLOW_CHUNK_VECTORS(JSONL,用于构建索引)与 GEAFLOW_QUERY_VECTORS(NPZ,用于基准评测)分离;GEAFLOW_VECTORS 仅保留为查询向量的废弃别名。
  • 向量生成步骤已补齐:模型 iic/nlp_gte_sentence-embedding_english-small、vector_version=v1.0.0、ModelScope 链接、384 维,以及 embed_chunks.py / embed_queries.py 的完整命令。构建流程分阶段为
    build_release → 嵌入 → 发布 → 基准,不再需要预构建索引包。已用 12 条真实查询子集做端到端验证:摄取 → 嵌入 → READY bundle → 12/12 请求成功,向量通道评估了全部 52 个 chunk 向量。
  • Python 依赖已补全:基准/测试、摄取、向量再生成分别对应三个依赖文件。
  • 构建失败已修复:根因是 Maven 将相对的 -Dmdep.outputFile 解析到模块目录,既导致 classpath 为空,又让运行产物落入 RAT 检查范围。现已改为绝对路径并加入 /tmp/ 排除。mvn -o -pl geaflow-ai
    test 在不跳过任何检查的情况下通过:470 个测试 0 失败;118 个 pytest 测试通过。
  • 数据集许可已确认:2WikiMultiHopQA 为 Apache-2.0、HotpotQA 为 CC BY-SA 4.0,均已对照上游来源核实,manifest 标记为 confirmed-upstream。
  • path-hit 指标已报告,HotpotQA 组件消融已补齐:relationPathHit 现已与召回并列报告,并注明其接近零的原因是图扩展产出实体邻域而非 gold 推理链。

数据集仍不随仓库分发;评审者可从文档中的来源下载、核对校验和,并仅凭仓库内的说明完成构建与分数复现。

Measured on all 48851 re-embedded chunks: text channels reproduce exactly, but the rebuilt
graph has 834797 edges against the reference 640275 because the ingest and graph code changed
after the reference bundle was built. Documents that reproducing the reported scores requires
the published reference bundle, and that from-scratch graph figures additionally require the
same build code revision. Also documents the required GRAPH_RAG_CACHE variable.
A from-scratch rebuild reproduces the text index exactly but not the graph snapshot: the
reference 2Wiki bundle has 640275 edges while a rebuild with current code has 834797, because
the ingest and graph code changed after the reference was built. Reproducing graph figures
therefore requires the same build code, not just the same data and model.

publish_bundle.py now records `git rev-parse HEAD` as build_code_revision in READY.json, with
an explicit --build-code-revision override for reproducible or offline builds, and the Java
bridge writes it into the bundle metadata. A reviewer can check out that revision and rebuild
the same graph snapshot instead of a newer, incompatible one. Older bundles without the field
remain valid and are still the reference for the earlier reported figures.

Verified by publishing a real 12-query subset: READY.json carries the exact HEAD revision.
CanonicalJsonlArtifactBridgeTest asserts the field is written; 470 Maven tests pass with
checkstyle and RAT enabled, and 118 pytest tests pass.
…sums

The reported scores are tied to the reference bundles, which are not in the git repository and
whose build revision predates build_code_revision recording. Publishing them is therefore the
way to keep those exact figures reproducible without forcing a rebuild that would change the
graph snapshot.

package_reproduction.sh assembles one archive per dataset containing the READY bundle, the
query vectors, the reference six-method CSV and a REPRODUCE.json descriptor, and writes
SHA256SUMS.txt plus the gh release command for whoever has release permissions. The
reproduction guide now leads with downloading or regenerating these packages and points at
the extracted bundle and vectors, while warning that rebuilding them with newer code will not
reproduce identical graph figures.

Verified by unpacking: READY.json version and every artifact checksum are identical to the
source bundle.
@aotenjou
aotenjou marked this pull request as ready for review November 16, 2026 10:58
… can reproduce

The reported figures were tied to a reference bundle that is not in the repository and whose
graph snapshot cannot be rebuilt from current code (640275 edges then, 834797 now because the
ingest and graph code changed afterwards). That made the numbers impossible to reproduce from
the remote contents alone.

Both datasets were rebuilt and re-measured with the current code, and the reports now use those
figures. Rebuilding twice yields the same bundle version and edge count, so the new baselines
are reproducible from the repository contents alone; READY.json also records the build revision.

Conclusions are unchanged in direction and small in magnitude. Adaptive MRR is 0.7850 on 2Wiki
(+2.3% over fixed hybrid) and 0.8096 on HotpotQA (+1.2%), against 0.7867 and 0.8067 before,
i.e. under 1% drift even though the graph edge count moved by 30%. Single-channel methods stay
bit-identical. The 1:4 fusion ratio remains optimal on both datasets, and the HotpotQA
no-graph-fusion result is literally identical at 0.8397, so the finding that graph evidence is
mostly ranking noise there is robust to the snapshot change. The three zero-delta ablation
conditions are unchanged as well.

Also documents the reproducibility checks: the 12-query end-to-end build, the determinism of
the full rebuild, and why graph snapshots drift without affecting the headline conclusions.
…ntents alone

Freshly regenerated all query vectors and re-ran both 500-query benchmarks against the
rebuilt bundles. 2Wiki matches the report bit for bit on every method; HotpotQA matches except
for a 0.04% MRR wobble on the vector channel caused by float-level differences when vectors
are regenerated. This closes the last open question on whether the figures can be reproduced
from the remote repository alone.
@yaozhongq

Copy link
Copy Markdown
Contributor

@aotenjou
Thanks for the update. Reproducibility has improved, but I do not think this is ready for final acceptance yet:

  • The from-scratch workflow is incomplete: query JSONL generation is undocumented, placeholders remain, and the HotpotQA file/path/checksum conflict with the manifest.
  • The Java test still fails on JDK 17 due to an exact floating-point assertion.
  • Adaptive path-hit remains 0 o

Document the missing query-JSONL step with a real command. materialize_queries.py turns the raw
dataset (zip or parquet) into the canonical query records the benchmark and embed_queries.py
consume; the generated id and question sequences are identical to the files used for the report,
so the evaluation is unchanged. The HotpotQA section now names the actual parquet cache and its
SHA-256, which is what the manifest references - the earlier JSON path and checksum contradicted
it. Angle-bracket placeholders in the build, embed and publish commands are replaced by a
copy-pasteable 2Wiki example with a HotpotQA substitution table.

Make floating-point assertions tolerant instead of exact. RankingMetrics.ndcgAtK was asserted
to sixteen digits, but Math.log is only specified to one ulp, so that assertion can fail on
other JDKs even though the computation is correct. All exact double assertions in the touched
tests now compare with a 1e-9 tolerance. Verified by running the whole module on a downloaded
Temurin JDK 17.0.21 as well as on JDK 21: 470 tests pass with checkstyle and RAT enabled.

Explain the adaptive path-hit of zero as a graph-construction limitation rather than a routing
tuning issue: edges come from entity co-occurrence and carry no directed relation semantics,
so the gold reasoning chain is absent from the graph and no fusion weight can produce it.
Improving path-hit requires relation extraction at ingest or relation-constrained traversal,
which is now stated as the follow-up work.
@aotenjou

Copy link
Copy Markdown
Contributor Author

Thanks for the careful re-review — all three points are addressed in 6a4c379.

From-scratch workflow. The missing step is now a real command: tools/graphrag/materialize_queries.py turns the raw dataset (zip or parquet) into the canonical query JSONL, and I verified
the generated id and question sequences are identical to the files used for the report, so the evaluation is unchanged. The HotpotQA section was indeed wrong — it documented
hotpotqa-distractor.json while the manifest references distractor-validation.parquet; it now names the parquet cache, its actual SHA-256, and the upstream HuggingFace URL with the commit
hash. All angle-bracket placeholders in the build/embed/publish commands are replaced by a copy-pasteable 2Wiki example plus a HotpotQA substitution table.

JDK 17 test failure. The root cause was an exact sixteen-digit assertion on RankingMetrics.ndcgAtK. Math.log is only specified to one ulp, so that assertion can legitimately fail on another
JDK even though the computation is correct. All exact double assertions in the touched tests now compare with a 1e-9 tolerance. I also downloaded a Temurin JDK 17.0.21 and ran the whole
module there as well as on JDK 21: 470 tests pass with checkstyle and RAT enabled in both. (On 17.0.21 the exact NDCG assertion also passes, so the failure you saw likely comes from a
different JDK 17 build — the tolerance removes that class of failure rather than depending on which Math.log a JDK ships.)

Adaptive path-hit of 0. This is a graph-construction limitation, not a routing-tuning issue. Edges come from text-cooccurrence-entity-v2 — "two entities appear in the same chunk" — and
carry no directed relation semantics, so the gold reasoning chain is simply absent from the graph and no fusion weight can produce it. Improving path-hit requires relation extraction at
ingest (typed, directed edges) or relation-constrained traversal, which I've documented as explicit follow-up work in the reports rather than claiming it can be tuned away.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants