Small Language Models Translate Ophthalmic Narrative Into HPO Terms
A locally operated AI pipeline can pull standardized Human Phenotype Ontology terms from routine ophthalmic documentation with a level of agreement that begins to look clinically usable. In this study, the system combined text segmentation, negation detection, and dense retrieval powered by a small language model, PHI-4, to process free-text descriptions drawn from routine medical records. The work focused on a practical problem that ophthalmology has carried for years: clinicians write richly detailed narrative notes, while research databases and interoperable systems need structured, standardized terminology.
The core challenge is not simply finding words in a chart. Ophthalmic language is packed with synonyms, shorthand, and context-dependent phrasing, and a term that appears in one note may be meaningless unless the system can detect whether it is affirmed or negated. The pipeline addressed this by using a multilingual HPO catalog expanded with synonyms during training, then applying automated extraction to anonymized consecutive physician letters. That design matters because a system built only on exact matches would miss much of the language clinicians actually use at the slit lamp and fundus exam.
Manual Annotation Set the Benchmark for Free-Text Extraction
To test the pipeline, investigators manually annotated 175 anterior segment and fundus descriptions from randomly selected medical records and used those labels as the ground truth. Manual review identified 342 HPO terms across the sample, with an average of 2.53 terms per document. The automated pipeline retrieved 341 terms, averaging 2.52 terms per document. The near-identical term counts do not by themselves guarantee correctness, but they do show that the model was operating at the right scale rather than overcalling large volumes of spurious phenotypes.
Performance held up across several standard retrieval metrics. Median Jaccard similarity reached 0.67, precision was 0.83, recall was 0.82, and F1 score was 0.80. In plain clinical terms, the system found most of the right findings, made relatively few false-positive additions, and preserved enough balance between sensitivity and specificity to support downstream structured data use. For ophthalmology, where a note may include multiple subtle abnormalities in a single paragraph, that balance is essential.
The performance profile is especially relevant in record environments where manual coding slows research and burdens clinical workflow. If a tool can convert narrative notes into structured phenotypes with this degree of accuracy, the downstream effect is not abstract. It can reduce the labor needed to build searchable cohorts, standardize phenotype extraction across cases, and make routine clinic letters more usable for research without forcing clinicians to change how they write.
Why Negation Detection and Synonyms Matter in Eye Care Documentation
Ophthalmic documentation often hinges on what is absent as much as what is present. A clinician may describe no corneal edema, no vitreous hemorrhage, or no retinal detachment, and a naive text-mining system can misread those phrases as positive disease signals. The pipeline incorporated negation detection specifically to limit that failure mode. That step is not decorative; it is central to preserving clinical meaning in a domain where a negated lesion can be as important as a documented one.
Synonym handling was another practical strength. The HPO catalog was augmented with multilingual synonyms during training, which improved the system’s ability to map narrative language to standard terms. Ophthalmology often uses multiple expressions for the same sign, depending on training, institution, and language background. A structured system that recognizes those variants can bridge the gap between local documentation habits and broader data harmonization efforts.
This is where the study becomes more than a technical exercise. Standardizing phenotypes using HPO creates a shared language for interoperability, but it also makes data more portable across institutions and more suitable for computational analysis. For clinicians, the immediate payoff is less manual coding. For researchers, the advantage is cleaner cohort assembly and a lower risk of missing cases because a relevant finding was buried in free text.
Practical Fit for Clinical Information Systems
The pipeline was designed to run locally, which is important in medical settings where privacy, infrastructure, and deployment constraints often limit the use of cloud-based tools. A locally operated model can be integrated into clinical information systems without forcing routine patient data into an external environment. That architecture may appeal to ophthalmology practices that want automation but remain cautious about where protected data travel.
The study also points to a realistic implementation path. Rather than replacing clinical judgment, the system functions as a text-processing layer that extracts phenotype terms from physician letters and converts them into a structured vocabulary. That kind of assistive automation is likely to be more durable than attempts to make clinicians enter every exam finding into rigid templates. Eye care notes are often narrative for good reasons; the issue is how to preserve their richness while making the information computationally useful.
A single sentence captures the operational value: the pipeline retrieved almost exactly the same number of standardized phenotype terms that human annotators identified.
What the Performance Metrics Mean for Ophthalmic Research
The reported metrics suggest a system that is already approaching a useful threshold for automated phenotyping, though not one that should be treated as a finished replacement for human review. A precision of 0.83 indicates that most extracted terms were correct, while a recall of 0.82 suggests that most relevant terms were found. The F1 score of 0.80 reflects a meaningful compromise between those two dimensions. Median Jaccard similarity of 0.67 adds another layer of support, showing substantial overlap between human and machine-derived term sets.
For studies that depend on retrospective chart review, those figures can make a real difference. The time required to hand-code thousands of notes often limits sample size long before scientific questions are exhausted. An automated HPO extraction pipeline could ease that bottleneck and expand the number of records that can be screened for ophthalmic phenotypes. In a field where disease subtypes, laterality, severity, and exam findings often determine research eligibility, structured extraction has obvious value.
Still, the study remains a validation exercise on a modest set of 175 descriptions, and broader deployment would need testing across more institutions, more note styles, and a wider mix of ophthalmic conditions. Performance in a controlled validation sample is encouraging, but it does not eliminate the need to verify generalizability in real-world workflows. The next step is not simply scale; it is stress-testing the pipeline against different documentation habits, more diverse language patterns, and the messy variability of everyday clinical practice.