LLM Drug Recommendations Gravitate Toward a Narrow Set of Product Names
Five large language models showed striking product-name convergence when asked to recommend FDA-approved drugs containing the same active ingredient. Across 177,120 synthetic clinical vignettes built from 123 disease-drug pairs and 1,440 sociodemographic profiles, the models repeatedly returned the same products rather than the broader set of available equivalents. In many cases, the top product accounted for nearly all first-choice recommendations.
The pattern matters because the choice of product name is not a cosmetic detail. Within the same active ingredient class, different manufacturers and formulations can carry very different acquisition costs, and a model that repeatedly elevates one brand or product over others may shape downstream spending even when the clinical target is unchanged.
Near-Perfect Agreement on Which Product to Recommend
The five systems evaluated were GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Flash, Llama 4 and MedGemma. Each vignette asked the model to recommend up to three FDA-approved products by name. Physicians then standardized product names and checked them against FDA approval status before the outputs were analyzed.
Across the 123 disease-drug pairs, the most frequently recommended product captured a mean of 97.5% of all first-choice recommendations. In practical terms, the models rarely behaved as if multiple clinically acceptable products existed. For most pairs, they behaved as though a single answer dominated the field.
Across all five models, the same dominant product accounted for more than 90% of recommendations in 96 of 123 disease-drug pairs. Agreement on which product to choose was almost perfect, with Fleiss’ κ reaching 0.895 (95% CI, 0.855–0.931). Claude Sonnet 4.6 aligned most closely with the pooled dominant drug at 96.6%, while MedGemma aligned least often at 90.6%.
This degree of concentration is clinically notable because the average disease-drug pair had 5.50 FDA-approved products available. The output pattern therefore reflects a strong narrowing of choice, not a shortage of therapeutic options.
Real-World Prescribing Looked Very Different
The LLM outputs were then compared with prescription distributions from the Medical Expenditure Panel Survey, a nationally representative US survey of prescribed medicine use. Of the 123 disease-drug pairs, 100 had corresponding products in the 2019–2023 MEPS data and were included in the comparison.
The mismatch was substantial. On average, the most frequently prescribed drug in MEPS matched the models’ first-choice recommendation in only 29.7% of cases, with a standard deviation of 4.5%. When the top three recommendations were considered, concordance rose to 58.8% on average, but even then the overlap remained incomplete. Weighted by prescription volume, first-choice concordance fell further to 17.0%.
A single model output can look internally consistent while still being disconnected from how clinicians actually prescribe.
Total variation distance sharpened that point. Across models, median TVD values ranged from 0.93 to 0.98, indicating very limited overlap between LLM-generated and MEPS-derived product distributions. On a scale where 0 means identical distributions and 1 means no overlap, these values sit close to the upper end of divergence. The models were not simply choosing among acceptable alternatives in roughly the same proportions as practice patterns. They were recommending a very different set of products.
Cost Differences Tilted Upward in Most Active Ingredients
The analysis moved from recommendation patterns to acquisition costs using National Average Drug Acquisition Cost data, which reflect invoice-based prices from retail community pharmacies. Of the 101 unique active ingredients represented by the 123 disease-drug pairs, 18 were excluded because MEPS prescription records were unavailable and 32 because complete NADAC pricing was unavailable for all relevant products. The remaining 51 active ingredients were included in the cost analysis.
Among those 51 ingredients, 38, or 74.5%, showed an increase in cost when LLM-recommended product distributions were compared with MEPS-derived distributions. Across the included ingredients, the projected annual acquisition cost difference was estimated at US$55.6 million, equivalent to a 1,019% increase. The paper’s cost modeling linked distributional divergence to spending, showing that recommendation bias is not merely an abstract informatics problem but a payer-level and system-level expense issue.
The mechanism is straightforward. If a model repeatedly favors higher-cost originator products or highly visible brands despite the presence of therapeutically equivalent options, the output can steer users toward more expensive prescribing behavior. In a setting where LLMs increasingly influence medication choices, even small recommendation skews can scale into large financial effects.
Visibility, Originator Status and Approval Timing Shaped Recommendations
Product characteristics were strongly associated with recommendation frequency. Drugs with greater online visibility were favored more often: a dedicated Wikipedia page was associated with higher recommendation odds, with OR 3.00, and higher Google search counts were also associated with increased recommendation frequency, with OR 1.70. Originator products were far more likely to be recommended than other products, with OR 20.5.
By contrast, products with later FDA approval and products more commonly prescribed in practice were recommended less often, with ORs of 0.76 and 0.87, respectively. Those associations point to a model behavior pattern shaped by text prominence and training-data visibility rather than by prescribing frequency alone.
This is exactly the kind of bias that can hide in plain sight. A model may appear clinically fluent while still inheriting a popularity effect from web text, where more discussed products are easier for the system to surface than lower-profile but equally valid options.
Prompts Helped, but Did Not Break the Pattern
Modified prompts shifted recommendations in the intended direction, yet the models still favored a narrow subset of products. The prompt intervention changed outputs, but it did not eliminate concentration. Even when instructed differently, the systems continued to collapse toward a limited set of product names.
That residual narrowing is important for deployment. If prompt engineering can only partially correct product choice bias, then the issue is not just instructional phrasing. The underlying model behavior needs testing, benchmarking and mitigation before medication-related use becomes routine in clinical or consumer-facing settings.
The study establishes a benchmark for evaluating regulatory concordance and commercial neutrality in LLM-generated drug recommendations. It also raises a direct question for health systems adopting these tools: whether a model that is technically accurate about the active ingredient can still be commercially misleading in the product name it favors.
The central limitation is not uncertainty about the signal; the signal is clear. The remaining challenge is operational—how to evaluate, constrain and audit product-name recommendations before they influence prescribing, cost exposure and pharmaceutical market behavior at scale.