An AI Clinical Governance Assistant for Ensuring Compliance of Cancer Treatments with the NCCN Guidelines
Bringing clinical guidelines into the loop: an AI assistant that checks treatment plans, explains its reasoning, and points oncologists back to the evidence.
Results reported in the paper for treatment compliance evaluation against oncologist-labelled cases (Table 4).
The question
Can a language model help oncologists assess whether a planned cancer treatment follows the NCCN guidelines? Our work explores an assistant integrated with an oncology electronic medical record system, supporting guideline questions and treatment compliance checks with source references.
The approach
We compared eight retrieval strategies and three language models. The selected system combines BM25 search with fine-tuned mxbai embeddings through reciprocal rank fusion. Parent-child retrieval preserves surrounding guideline context, which is passed to Mixtral-Small to produce a reasoned response. The design keeps patient records and model inferences on-premise.
The evaluation
A panel of oncologists labelled 102 anonymized patient scenarios spanning breast, lung, prostate, and colon cancers, plus multiple myeloma. The assistant achieved an F1 score of 0.863 and accuracy of 0.864 in checking treatment compliance. Oncologists also assessed response clarity, accuracy, brevity, and relevance.
What comes next
These findings reflect a carefully selected cohort of 102 scenarios. The paper identifies a larger patient dataset and broader evaluation as next steps, alongside exploring guideline-based support for other chronic diseases.
A closer look.
The original paper, page by page.