Specialist Research Process — End-to-End

When a clinician (or another agent) poses a clinical question to AiMOneHealth, the answer is not a single LLM call — it's a structured pipeline that retrieves, reasons, adversarially re-checks, and records every step. This page walks through the full flow as an interactive stepper, then lays out the decision points with balanced pros/cons so a reviewer can judge whether the production defaults are right for their setting.

Controls — ◀ / ▶ buttons step manually · ▶ Play autoplays (default 3.5s/frame; pick from the Speed menu, slowest is 6s) · click any dot to jump · keyboard ← / → / Space when the panel has focus.

Walkthrough — the full pipeline, agent by agent

The full picture starts dimmed. Each step lights up the agent that is currently active.

Question → Retrieval → Researcher → Critic → Memory — step by step

1 / 6SpeedSlowest (6s)Slower (4.5s)Normal (3.5s)Faster (2.5s)Fastest (1.5s)

Frame 1 of 6 — The full picture (dimmed). Roadmap for the walkthrough; we light up one agent per step.

Prefer an auto-looping GIF? Download the animated GIF. Use the buttons above (or ←/→/Space) to scrub frame-by-frame at your own pace.

The full ReasoningFlow — every retrieval, prompt, classification, supersession, freshness check, verdict — is recorded. Patients can later replay it on the patient-facing reasoning view.

Decision point 1 — Retrieval mode

The hybrid retriever can be steered toward graph-only, text-only, or balanced hybrid. Default is hybrid; specialty workflows occasionally override.

Mode	Pros	Cons	When to choose
Graph-only (CKG)	Precise relationships ("first-line for X" / "contraindicated with Y"); easy to cite the source guideline; fast	Misses claims not yet in the curated graph; brittle to entity-resolution misses; weak on free-text patient narratives	Drug–drug, drug–condition, guideline-driven queries with a clean entity set
Text-only (RAG over journals)	Highest recall on novel evidence (NEJM, JAMA, Cochrane); fluent narrative answers	Can hallucinate without a graph anchor; freshness depends on the embedding refresh job; harder to cite cleanly	Open-ended literature surveys; "what's new in X" queries
Hybrid (graph + text + agg) — default	Graph anchor keeps citations clean; text fills the recall gap; aggregation queries enable population-level prompts	Higher latency (parallel fan-out + late merge); ranking the merged set is more complex (the composite_k score does this)	Default. Most clinical questions sit between the two extremes

Implementation pointer: the dispatcher lives at apps/research/memory-store/services/retrieval/hybrid_retriever.py; per-mode classifiers in classifier.py route the call. Spec 060 (Hybrid Retrieval Core).

Decision point 2 — Critic dispatch strategy

When the Critic rejects a claim, the consumer can route the rejection one of two ways. The strategies are mutually exclusive at runtime, selected via CRITIC_DISPATCH_STRATEGY.

Strategy	Pros	Cons	When to choose
mark_for_review (default)	Human-in-the-loop. Curator catches subtle semantic errors that re-research can't fix. Lower LLM spend per rejection.	Curator workload scales with rejection volume; queue can grow if staffing is thin. Pending-review entries have to be drained manually.	Critic rejections are mostly semantic — the claim is wrong, not just under-supported. Re-research would reproduce the defect.
re_research	Fully automated. The dispatcher synthesises a research-task input that includes the rejection reason, then re-runs the Researcher with the loop-break marker dispatched_from_backlog=True. No curator round-trip.	Burns LLM tokens on every reject. If the original retrieval set was the bottleneck, re-research will just produce another weak claim (the loop voluntary-fails with still_weak_after_critic_reject:k=...).	Critic rejections are evidence-quality — low support, low novelty, weak composite_k. Fresh research with rejection context yields stronger evidence.

Switching between them is reversible — see research/loop-activation. Existing pending-review entries from a prior mark_for_review run stay in the triage queue (not auto-converted).

Decision point 3 — Source tier mix

Retrieval ranks across multiple source tiers. Operators tune the weight given to each tier (config: apps/research/research-engine/services/config/trusted_sources.yml).

Source tier	Pros	Cons	Weight in default composite_k
NCCN guidelines	Authoritative; clinically actionable; revised on a known cadence	Slow to incorporate brand-new evidence; US-centric	High (1.0)
NEJM / JAMA / JBJS (publisher AI partners)	Peer-reviewed; high signal per article; citations are clean	License-gated; embedding refresh latency; breadth limited to subscribed journals	High (0.95)
Cochrane (systematic reviews)	Strongest summary of effect sizes; ideal for "does X work"	Updates infrequently; narrow topic coverage	High (0.95)
JOR / JBMR-B / Spine / Acta / J. Arthroplasty (specialty journals)	Deep coverage of orthopaedic / specialty subdomain	Narrower than NEJM; smaller audience	Med (0.8)
PubMed (broad)	Highest recall; freshest preprints	Mixed quality — needs Critic re-check; some predatory sources	Med (0.6)
ClinicalTrials.gov	Authoritative for what trials exist; no PHI outbound	Doesn't tell you outcomes — only protocols and status; needs joining to NEJM/JAMA results	Med (0.7) — boosted for "is there a trial for X" queries
ClinVar (variants)	Authoritative for genetic variant interpretation; public NIH	Niche; only relevant when query touches genetics	High (0.95) when query is on-topic; otherwise weight=0
DrugBank	Comprehensive drug–drug + drug–target catalog	Commercial license (no PHI outbound); doesn't carry guideline strength	High (0.9) for pharmacology queries

Rule of thumb: a strong claim cites at least one NCCN/NEJM/JAMA/Cochrane anchor plus one corroborating source from a different tier. The Critic flags single-tier claims as low_support even when composite_k would otherwise pass — that's intentional defense in depth.

Where the safety rails live

The pipeline is bracketed by hard rules (these never turn off in production):

• PHI in / PHI out — apps/ml/services/sanitize.py (G-01..G-20 prompt-injection patterns) + deidentify.py run on every prompt. The <<<CLINICAL_DATA>>> delimiter scopes user content as data, not instruction. Output passes Pydantic schema validation before reaching the HCP.
• BAA registry — every LLM provider, journal source, and embedding service must be entered in apps/api/src/config/baa-registry.yml. Production fails-closed if any vendor has baa_signed: false (FR-032).
• Audit chain — the ReasoningFlow records the what with opaque IDs only; llm_call_logs holds the encrypted prompt + output content, joined back by trace_id. Auditors have everything; logs and audit payloads have no PHI narrative.
• FDA / Cures-Act exemption posture — every AI response carries the FR-020 metadata block: { disclaimer, llm_call_id, model_provider, model_name, generated_at, fda_cds_exemption_ref }. The HCP confirms; the system never auto-acts.

Where this is implemented

Stage	Code	Spec
Orchestrator	apps/research/research-engine/services/orchestrator/	050
Hybrid Retriever	apps/research/memory-store/services/retrieval/	060
Researcher	apps/research/research-engine/services/agents/researcher/	050
Critic	apps/research/research-engine/services/agents/critic/	050
Backlog + consumer + triage queue	apps/research/research-engine/services/workers/ + memory-store routes	066
Reasoning flow recorder	apps/research/research-engine/services/recorders/	066
Source-tier weighting	apps/research/research-engine/services/config/trusted_sources.yml	056, 059

Related pages

• Memory Concepts → — three-tier model + how data moves.
• Reasoning Flow → — ReasoningFlow / ReasoningStep schema + the self-improvement loop.
• Self-Improvement Loop — Activation Runbook → — operator-facing flag-flip order, including the Critic-strategy switch.
• Hybrid Retrieval → — retriever internals.
• Agents → — Researcher / Critic / other agent details.