Temporal Resampling Distorts Clinical Reinforcement Learning Before It Reaches the Bedside
In offline reinforcement learning, the timing of clinical decisions is not a cosmetic detail. It shapes the state-action sequence the model learns from, and when those naturally irregular intervals are forced into neat bins, the learning signal can shift in ways that do not match real care. This simulated trial tested that problem directly and found a substantial gap between what resampled data suggested and what models could actually do when placed back into a realistic decision environment.
The work began with intensive care data from patients receiving continuous insulin infusions in the MIMIC-IV database. Temporal binning did more than simplify the data stream. It reversed the apparent causal order for over 70% of paired glucose measurements and insulin adjustments within the same interval, masked significant dysglycaemic events, and merged separate treatment decisions into a single artificial record. In a field that depends on correctly preserving cause and effect, those distortions are not minor preprocessing artifacts. They alter the clinical story the algorithm is being taught to believe.
Why Irregular Clinical Timing Matters for Learning Algorithms
Clinical care does not happen on a clock that waits for the nearest 10-minute, 2-hour, or 4-hour boundary. A nurse titrates insulin when glucose changes, an intensivist reacts to a trend, and a bedside team responds to the pace of physiology rather than to a resampling grid. When records are collapsed into fixed intervals, two decisions that were separated in time can appear simultaneous, and the model can no longer distinguish whether a glucose correction preceded the insulin change or followed it.
That problem cuts directly into the logic of reinforcement learning. These models are built to infer which actions lead to better downstream rewards, but they depend on accurate sequencing. If the timing has been rewritten, the agent may optimize an artificial timeline instead of a real one. In diabetes care, where the difference between drifting hyperglycemia and abrupt hypoglycemia can be clinically meaningful, temporal distortion can easily become a safety issue rather than a statistical nuisance.
A Virtual Trial in Type 1 Diabetes Exposed Large Performance Losses
Investigators then moved to an in silico trial using 30 virtual patients with type 1 diabetes from the FDA-approved UVA/Padova simulator. The simulator was modified so that decisions occurred at stochastic intervals, preserving the unpredictability of clinical timing rather than smoothing it away. From there, a training dataset was built for offline reinforcement learning, and three ORL algorithms were trained on the unprocessed data and on equivalent datasets resampled at 10-minute, 2-hour, and 4-hour intervals.
The difference after deployment back into the simulated environment was stark. Temporal resampling reduced model performance by up to 60% relative to unprocessed data, and 4-hour binning pushed every agent below the dataset baseline. One sentence captures the central clinical danger: the model looked better on paper after resampling, but performed worse where it had to act.
That failure pattern is especially troubling because the degradation did not present as a subtle nudge in output quality. It was large enough to overturn the practical value of the learned policy. In a real clinical workflow, a model that seems safe during retrospective validation but loses as much as 60% of performance after deployment could recommend actions that are simply too blunt for bedside use.
Retrospective Validation Missed the Harm
The most consequential finding may be the one that would be easiest to overlook during model development. Retrospective evaluation on resampled data actively obscured the performance loss, predicting returns that were 1.5 to 3 times better than the agents actually achieved in simulation. In other words, the very test used to reassure researchers about safety and utility became part of the problem.
This mismatch exposes a familiar vulnerability in healthcare AI: a model can appear stronger when judged against a sanitized dataset than when it encounters the messier timing of actual clinical care. If the evaluation pipeline erases the same irregularity that the real environment contains, it will reward the wrong behavior. Here, the consequence was not just overoptimism. It was a false safety signal.
The biological and operational logic is straightforward. Glucose control depends on response timing, insulin kinetics, and the pace of physiologic change. Compressing those dynamics into fixed bins can hide the very episodes that should teach a model to act cautiously, and it can merge distinct interventions into a single state transition. The result is a policy that learns from a simplified version of medicine rather than medicine itself.
Clinical Implications for Diabetes AI and Beyond
The findings carry immediate relevance for type 1 diabetes modeling, but the methodological warning extends much further. Any setting with irregular interventions, delayed responses, or event-driven decision-making can suffer when natural timing is flattened into regular intervals. Intensive care, sepsis management, ventilation, and chemotherapy all rely on sequences in which the order of events can define the meaning of the data.
The study does not argue against offline reinforcement learning. It argues for better fidelity in the datasets used to train and judge it. Natural clinical timings between decisions appear to preserve critical information that resampling removes, and the results here suggest that preserving those timings may be necessary before these systems can be used safely in patient care. The public release of simulated datasets and code also gives others a route to reproduce the analysis and test alternative preprocessing strategies.
The work leaves one important practical challenge in place: current clinical AI pipelines still favor regularity because standard algorithms are easier to apply to evenly spaced data. That convenience comes at a cost when the true sequence of care is the signal the model is supposed to learn. For developers building treatment recommender systems, the next step is less about inventing a new objective function than about respecting the raw timing of bedside decisions. Once that timing is altered, the model may already be learning the wrong medicine.