DrugCon maps contradictory randomized trial claims at scale
Contradictory claims across randomized controlled trials can leave clinicians with two studies pointing in opposite directions while both remain scientifically credible. In practice, the disagreement is rarely random; it often comes from shifts in who was studied, how the intervention was delivered, or which endpoint was chosen to define success. DrugCon was built to make those fault lines visible.
The framework uses a large language model to extract study conclusions from RCT abstracts, connect claims when they refer to the same drug-disease association, and determine whether the claims conflict. It then goes one step further by assigning explanations through a ten-part taxonomy of study-level factors, including population, dosage, exposure duration, and outcome measures. The point is not simply to label two findings as contradictory, but to show why the contradiction exists.
Applied to 224,756 RCT abstracts, DrugCon produced DrugCon-Bench, an exhaustively annotated benchmark containing 542 contradictory claim pairs drawn from 819 abstracts. That scale matters because the volume of clinical literature now exceeds what manual reconciliation can handle, and the consequence is familiar to any clinician reading the same therapy through different trial lenses. One paper looks favorable. Another does not.
Why trial disagreements are so difficult to automate
Standard automated methods usually stop at sentence-level comparison. They can identify whether two abstract sentences appear to disagree, but they often miss the clinical structure behind the disagreement. A study of an antihypertensive in older adults is not interchangeable with a study in younger adults with different baseline risk, and a blood-pressure endpoint is not the same thing as a composite cardiovascular endpoint even when both involve the same drug.
DrugCon was designed to address that gap with counterfactual reasoning. Instead of asking only whether two claims sound similar or opposite, the system evaluates whether changing a study-level factor would plausibly reverse the conclusion. In other words, it tries to distinguish a true contradiction from a context-driven divergence. In clinical research, that distinction determines whether a conflict reflects a flaw in evidence or a difference in trial design.
The benchmark itself captures how often those disagreements are layered. Most contradictory pairs involved more than one study-level factor, 75.5% in total. Single-variable explanations were not the dominant pattern. The evidence disagreed because the trials differed in multiple ways at once.
Outcome measures drove most contradictions
Outcome selection emerged as the most common source of conflict, appearing in 64.0% of contradictory claim pairs. That finding is clinically intuitive. If one trial defines success by symptom relief and another by hospitalization reduction, the same intervention can appear helpful in one setting and neutral in another. The biology may not have changed; the lens used to judge it has.
Population differences were nearly as common, accounting for 51.3% of contradictions. This is where bedside medicine and trial interpretation often collide. Age, baseline disease severity, comorbidity burden, and prior treatment exposure can all shift effect size or dilute a signal that looks clear in a narrower cohort. A treatment that performs well in a selected trial population may look far less convincing once broader patients are included.
Dose and exposure duration also played a major role, present in 35.6% of contradictions. Pharmacology rarely behaves as a fixed binary. Underdosing may suppress efficacy, while longer exposure can reveal delayed benefit or delayed toxicity. A short follow-up window can miss effects that only emerge after sustained treatment, and a higher dose can change both efficacy and tolerability in ways that alter the final abstract conclusion.
Performance across detection and explanation tasks
DrugCon achieved an F1 score of 0.744 for contradiction detection, outperforming rule-based, supervised, and natural language inference baselines. For a task that depends on comparing clinical claims rather than isolated words, that level of performance suggests the model is capturing more than surface-level opposition. It is learning the structure of medical disagreement.
The explanatory task was harder. DrugCon reached an F1 of 0.482 for identifying study factors that account for the contradiction. That drop is unsurprising. Explanation is a more demanding clinical task than classification because it requires mapping language to the hidden architecture of a trial: who was enrolled, what endpoint was measured, how long the intervention lasted, and which protocol details could reasonably account for divergence.
Even so, the system offers a meaningful advance over prior approaches. A binary contradiction label tells a reader that two abstract conclusions conflict. An explanation layer tells the reader where to look next. In evidence synthesis, that is the difference between a flag and a working hypothesis.
What scalable explainability could change in evidence review
The study positions contradictory evidence as a computable problem rather than a purely manual one. That shift has obvious implications for systematic review, guideline development, and trial surveillance. Large literature corpora can be screened for conflicting claims far faster than a human team can read them one by one, and the resulting disagreements can be organized by the factors most likely to drive them.
For clinicians, the value is practical. If a new abstract appears to reverse a prior conclusion, DrugCon’s framework points toward the likely reason: a different patient mix, a different dose, a different endpoint, or a different treatment window. Those are not abstract distinctions. They decide whether one study should alter practice, refine practice, or be set aside because it answers a different question.
The benchmark also raises a useful caution. Contradiction is often a property of framing, not merely of results. Two valid randomized trials can disagree because they studied different diseases within the same disease label, or because one used a surrogate outcome and the other used a clinical endpoint with direct patient relevance. The model’s taxonomy makes those hidden mismatches explicit.
Limits of the benchmark and the clinical next step
DrugCon-Bench is built from abstracts, not full texts, so it works within the constraints of what trial authors choose to compress into summary form. Abstracts do not always expose every design nuance that may explain a discrepancy, especially when subgroup structure, protocol amendments, or secondary analyses drive the final interpretation. The framework therefore reflects the limits of summary evidence as much as the power of automation.
Even with that constraint, the dataset is unusually valuable because it is exhaustively annotated and derived from a very large RCT corpus. The result is a reference point for future work on evidence reconciliation, literature mining, and explainable clinical natural language processing. The unresolved question is not whether contradictory claims exist. It is how quickly and reliably they can be organized into the trial-level reasons that matter for interpretation at the bedside and in guideline committees.
DrugCon shows that contradiction detection can move beyond sentence matching toward clinically legible explanation. The remaining challenge is extending that reasoning from abstract-level summaries to the broader evidentiary context in which treatment decisions are actually made.