A prompt-guided language model Speeds Systematic Reviews of digital health trials in two case studies, according to research published in PLOS Digital Health. Taylor B. Harrison, Dian Hu and colleagues tested a retrieval-to-abstraction framework that screened publications and extracted information from full texts without task-specific fine-tuning.
The work focuses on randomized controlled trials (RCTs) involving digital health technologies. These trials use tools such as wearable activity trackers, smartphone interventions, telehealth platforms and remote monitoring sensors. The researchers examined whether an unmodified pretrained large language model could help review teams assess reporting in this expanding research area.
In the study, the model achieved F1 scores of at least 0.9 across screening and information-extraction tasks. The authors described that performance as comparable to inter-annotator agreement, a measure of how consistently trained human reviewers make the same judgments. The result came from two case studies, not a validation across all medical research.
How the Framework Speeds Systematic Reviews
The framework divides review work into retrieval and abstraction. During retrieval, the model screens candidate articles against a review question’s inclusion criteria. During abstraction, it preprocesses text and tables, reviews full texts and extracts requested data elements.
The team did not fine-tune the underlying model on annotated medical literature. Instead, its workflow used prompts and staged tasks to guide a general-purpose model through specialized review work. Text and table preprocessing helped convert the varied formats of published studies into inputs the model could handle.
Two Reporting Questions Put to the Test
One case study assessed how trials reported technology-related outcomes, including participant engagement with a digital intervention. The other examined digital health equity reporting: whether trial reports described how technologies performed across diverse populations, including groups historically underrepresented in digital health research.
These areas can be difficult to compare across studies. Reporting guidelines such as CONSORT establish common elements for trial reports, but digital health studies may describe outcomes in different ways. A trial might present engagement measures in a different format from another study, or leave technology-adherence details in supplementary material.
That variation can require reviewers to search through additional material or exclude a study when a needed detail is unavailable. The researchers argue that automated assessment of reporting adherence could help make such gaps visible across larger sets of trials.
Reporting Quality Still Shapes the Evidence
Systematic reviews inform clinical practice guidelines, regulatory decisions and health technology assessments. Inconsistent reporting makes it harder to locate and compare common data across primary studies. The study presents machine-assisted screening and abstraction as a way to organize that work more quickly, while keeping the focus on defined review questions and requested data.
The authors’ case demonstrations also point to possible uses for funders, guideline developers and journal editors. Automated checks could help these groups monitor reporting of technology engagement or equity-related items and identify where guidance may need strengthening. The study does not establish that reporting rules have changed or that the framework has been adopted by those groups.
The findings are bounded to the two digital health reporting case studies. The authors frame the work as support for scalable evidence synthesis, not as proof that human reviewers are unnecessary. Review teams still define the question, resolve ambiguous cases and interpret clinical significance.
For related Meditech Today coverage, read about AI in medical imaging, AI in global health and AI in patient monitoring.
