BMJ data sharing policy met a stubbornly low compliance rate
An observational study of the British Medical Journal archive found that open data had not yet become routine practice, even under a strong journal policy designed to encourage it. Across a random sample of 160 BMJ research articles published between 2009 and 2015, only 7 of 157 eligible research papers ultimately shared their raw data sets, a rate of 4.5% with a 95% confidence interval of 1.8% to 9%. The gap between policy and practice was not subtle.
The study asked a practical question with direct relevance to reproducibility, trial verification, and secondary analysis: when authors say their data are available, can outside researchers actually get them? The answer was mixed at best. Three articles contained the data directly in the paper. Of the remaining 157, 50 articles, or 32%, indicated that their data sets were available. Twelve of those relied on publicly available data, while 38 required an email request to the investigators. Yet only one publicly available data set could actually be accessed, and only 6 of the 38 requested by email were shared in response. Availability on paper did not translate into access in practice.
What the archive showed about real-world access
This was an observational study drawn from the BMJ research archive, not a trial of an intervention, so the outcome was less about efficacy than about behavior. Even so, the signal was clear. A policy can exist, be prominent, and still fail to produce meaningful data sharing if the mechanics are loose or the incentives are weak. In this sample, many authors appeared to acknowledge some level of openness, but very few delivered usable data when asked.
The numbers matter because raw data are the foundation for verification. Without access to participant-level data, external investigators cannot easily reproduce analyses, probe inconsistencies, or test alternative assumptions. That becomes especially important in clinical research, where data sets often underpin treatment claims that influence guidelines, prescribing patterns, and patient care. When data sharing stalls at the request stage, the scientific record becomes harder to audit.
The study also examined a narrower group of 21 clinical trials that were bound by the BMJ data sharing policy. In that subset, 24% shared data, with a wide confidence interval of 8% to 47%. Even there, compliance was incomplete. A policy directed at trials, where the case for transparency is especially strong, still left substantial room for noncompliance.
Data availability is not the same as data accessibility.
Policy wording, inbox failures, and weak incentives
Several explanations for the low sharing rate emerged from the analysis. The first was policy language itself. The BMJ policy may have left room for individual interpretation and possible loopholes, allowing authors to claim openness without clearly defining the conditions under which data would actually be released. In practice, vague language can produce optimistic statements that look compliant while remaining difficult to operationalize at the bench or in a secondary analysis group.
Email-based access created another obvious point of failure. Requests may have landed in spam folders, been overlooked, or simply gone unanswered. For a process that depends on individual correspondence, the burden of access can shift entirely to the person seeking the data. If the corresponding author is inactive, unresponsive, or difficult to reach, a nominally shared data set may remain inaccessible indefinitely.
The study also pointed to a deeper structural problem: researchers are rarely rewarded for sharing their data. In academic medicine, publications, grants, and citations remain the primary currencies of advancement. Preparing a clean, usable data set for external use takes time, carries perceived risk, and may expose errors or invite scrutiny without bringing clear professional benefit. Under those conditions, even well-intentioned authors may default to reluctance.
Why transparency still struggles in clinical research
The barrier here is not just administrative. It is cultural and economic. Raw data sharing requires curation, documentation, and a willingness to cede control over downstream analyses. Many investigators worry about misuse, misinterpretation, or being scooped on secondary papers. Others may simply prefer to preserve the labor advantage embedded in their own data sets. Those concerns are familiar, but they do not erase the fact that clinical research depends on trust, and trust depends on the ability to inspect the underlying evidence.
For readers of medical journals, the practical consequence is straightforward. A journal policy that appears strong on paper may still produce sparse real-world access unless it is specific, enforceable, and paired with incentives that matter to researchers. Compliance cannot rest on aspirational language alone. It has to survive the friction of day-to-day academic behavior.
The BMJ dataset in this study also suggests a divide between statements and substance. Articles could declare data availability without ensuring that a stranger could actually obtain the data. That gap weakens the interpretability of published work and complicates any attempt to build cumulative science from previous studies. In an era when reproducibility is central to credibility, a low sharing rate is more than a process problem; it affects the reliability of the evidence base itself.
A policy that asks for openness must make openness usable
The findings argue for a more operational data sharing framework. Clearer language would reduce ambiguity about what counts as sharing, when access should be granted, and how requests should be handled. Better infrastructure could replace ad hoc email exchanges with a system that records requests, preserves contact pathways, and makes archived data easier to retrieve. Incentives also need attention, because a policy without academic credit or practical support often produces compliance in name only.
For clinical researchers, the message is direct. Transparency cannot depend on goodwill alone, especially when participant-level data are at stake. For journals, the challenge is to convert policy into behavior. And for readers who rely on published studies to shape clinical reasoning, the most important detail may be the most basic one: declared availability still does not guarantee delivery.