Most of us academics don't worry about "threat models" like the security folks who map out attack surfaces and the blast radius when something goes wrong. But I think it's the right framing for what just happened to scholarly publishing.
Citation omission was always a soft-failure mode: lazy or self-serving, sometimes pernicious, but mostly corrected by the slow social mechanisms of peer review and reputation. LLM-hallucinated citations looked at first like more of the same: a new occupational hazard layered onto an imperfect system.
What changed is that there are now consequences. Yesterday, Tom Dietterich announced that arXiv will ban authors for a year if they submit papers containing hallucinated citations. That's a step-function increase in severity. Hallucinations aren't just a nuisance, they're a real threat vector and the "attacker" was invited into your workflow. This can do real damage not just to the academic endeavor in the abstract but to your reputation and career.
Like any threat model, the answer isn't to stop using the tool, it's to add controls. This is one of the reasons we build Valency Bond (app.valency.io), to give frontier LLMs access to a real corpus of 50M+ papers and preprints so our workflows can stay tethered to verifiable citations. One must still sign off on the work, but the failure mode shifts from "model invents a plausible-sounding paper that doesn't exist" to "model output points you at a real one you can read."
My full post: https://lnkd.in/g47yHn_x
