Why do language models miss domain context in scientific archives?

[JosephW]

I’ve been trying to use a large language model to help parse and categorize decades of old lab notes, but I’m hitting a wall where its interpretations of our shorthand and diagrams feel superficial. It’s like the model lacks the domain-specific context to make meaningful connections, even with careful prompting. Has anyone else run into this gap between the tool’s general capability and the nuanced understanding required for real scientific archival work?

[ZacharyH]

Yep, ran into that wall. We’ve got decades of notes with shorthand that only a few people can read. We built a tiny domain glossary and used it in a few-shot prompts, plus a small labeled subset to calibrate. On a 50-page test batch, the model’s top category matches rose from about 15% to 28% after the glossaries, but it still misreads core abbreviations and diagram cues. It felt like we swapped one surface issue for another and everything else stayed fuzzy.

[Hannah27]

We ended up with a human in the loop. The model drafted tags, we corrected them, and fed the corrections back. It helped a little, but roughly half the auto tags were wrong and the process was too slow for the archive size we’re dealing with. We wound up running two pipelines: one for structured metadata, one for free‑form notes, just to keep moving.

[Jeffrey_C]

A tangent I kept circling back to: maybe the real bottleneck is the feed quality. A lot of diagrams and handwriting get garbled by OCR; the model then reasons on shaky inputs. We re-OCR’d and even asked a few physicists to recheck sketches, but the fidelity never fully recovered. Still, the core context gap remained even with cleaner data.

[George21]

Do you think the bottleneck is the labeling scheme or the model’s inference ability for tacit knowledge you already have?