Citation and Reference Accuracy

I am writing this to remind everyone that sprinkling 'bogus citations' into scientific work is one of the worst forms of error in scientific writing, and I want you to understand what it means, how it is seen, and why you should consider avoiding it. To understand what I'm talking about when I say 'bogus citations', consider the following scenario: you read a sentence/statement in a paper, you see a citation that supports it, then you take your time to read the paper cited, and only then you realize that the cited work is not one that supports the statement in the original paper. That right there is what I call a bogus citation .. a reference that appears to support a point when it does not. Perhaps not all bogus citations will be caught by everyone, but some bogus citations will certainly be caught by someone, and it will be devastating for their perception of your scientific rigour every time. I think the vast majority of bogus citations will be due to one of these actions in this list: (1) Authors cite things just because they think they are good citations based on how these work were cited in papers they read from others, and not based on their own reading. (2) Authors see a connection between their statement/point and the cited work, but it is a kind of connection that is not obvious to a naive reader who has not thought about either work as much as the authors did. (3) Authors know that the citation doesn't fit, but they also assume that people will not pay enough attention to figure that out and they will get to pass unsupported statements (whether they are questionable or not) as statements with support from independent work. These scenarios range from simple oversight or incompetence to outright malice. But the adverse outcomes they promote are all the same, and it doesn't matter whether it was due to incompetence or malice. The former case will come from authors who don't care about the quality or accuracy of every part of their work. The latter case will come from those who are greedy, arrogant, and have no problem with misrepresenting themselves and/or others. Both are terrible, and both are unacceptable. Respect for scientific integrity is rapidly deteriorating in the public eye due to unrelenting propaganda pushed by those who have no time to go from beliefs from gut feelings to insights supported by evidence. So I am asking myself and everyone reading this to understand what it really means to have bogus citations in scientific work, to think about its implications, and to remember to do the right thing: never cite something in your work that you have not read and understood, and if there is a link between a statement and the citation that is not immediately clear or straightforward, don't just put the citation in and expect others to figure it out. Be explicit, find an elegant way to connect things, and by doing so add value not only to your own work, but also the one you are citing.

On taking time to do a good lit review & craft a strong paper (or you can't take shortcuts). As a senior editor, I read dozens of reviews a year. Often, a reviewer complains that the authors have failed to cite papers or miscited papers. Sometimes, they use that lack of due diligence to reject a paper. I wrote about the problem of rejection bc of missing papers here - * took a strong position against doing so: https://lnkd.in/eAHSZKW5 However, I want to offer a clarification for early career authors. Calling for reviewers to educate & help authors, does not mean that you should submit papers that are under-researched or quickly assembled. It means that errors of omission occur even in well-researched & thoughtfully assembled papers. It is up to you, the authors, to do your very best job to stay up to date on the literature, write reasonable arguments, & submit work using current methods. This is easy to write - but hard to do - so how to ensure you've done a good job? First, check for updated references before submission. Papers take months to write. It's natural a few relevant citations may pop up once you've completed the literature review. I like to check for new references to key citations in my paper. I do so on Googlescholar.com. If I find relevant references, I add them to the paper. Second, edit once, edit twice, edit thrice. Early career authors often submit papers with small grammatical errors. I've asked a few of them why? They say that grammar should not get in the way of seeing the ideas. Clearly, they have not met my OCD friend Reviewer Two. Grammar matters. Third, check author names. Grammar & spelling checkers won't catch author name errors. In fact, they often amend them to be incorrect. If your checker recommends an alternative spelling, check the reference. Nothing makes a reviewer crabbier than seeing their name misspelled - it's happened to me! Fourth, really take time to know your topic. In the high-volume, high-pressure world that we work in, many authors are submitting papers on diverse topics. That they have a superficial understanding of a topic shows up in the literature review. The early stages of your work on a new topic should involve a few weeks, if not months of reading. Note, bc of this high overhead, I suggest people write synergistic studies or sets of papers on related topics or questions. Finally, have a knowledgeable peer read the paper before submission. The issue isn't just having the right citations, as Bryan Gaensler points out, it takes time to understand the nuances of a literature. Have a second set of eyes read the paper to make sure that not only the right citations are there but that they are used in the right spirit. If you take time to update references, carefully edit the paper, & solicit a knowledgeable peer review, you will have much more luck in securing a revision at a good journal.

ICLR'26 has decided to 𝗱𝗲𝘀𝗸-𝗿𝗲𝗷𝗲𝗰𝘁 papers with 𝗵𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗲𝗱 𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲𝘀 generated by LLMs. That raises a practical question for every author: How do we verify citations reliably? We’re excited to share our new paper, 𝗕𝗶𝗯𝗔𝗴𝗲𝗻𝘁, an agentic citation verification framework designed to make reference checking auditable BibAgent traces where a claim is supported, surfaces evidence spans, and reports confidence rather than guessing. When a cited paper is behind a paywall, it can switch to a community-based “evidence committee” approach: collect downstream open-access citers, distill what they attribute to the paywalled work, and decide with consensus—or abstain if evidence is insufficient. We also propose a unified miscitation error-code taxonomy and release 𝗠𝗜𝗦𝗖𝗜𝗧𝗘𝗕𝗘𝗡𝗖𝗛, a large cross-disciplinary benchmark of miscitation cases. If you’re building LLM writing assistants, submission pipelines, or research integrity tooling, this is meant to be a step toward: draft fast → verify rigorously → publish faithfully. Ultimately, we hope this research helps 𝗳𝗮𝗰𝗶𝗹𝗶𝘁𝗮𝘁𝗲 𝗳𝗮𝗶𝘁𝗵𝗳𝘂𝗹 𝗮𝗻𝗱 𝘁𝗿𝘂𝘀𝘁𝘄𝗼𝗿𝘁𝗵𝘆 𝘀𝗰𝗶𝗲𝗻𝘁𝗶𝗳𝗶𝗰 𝗽𝘂𝗯𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀—so that emerging agents for scientific discovery can build on literature that’s genuinely grounded, not citation-shaped. Paper link: arxiv.org/abs/2601.16993 #AI #LLMs #ResearchIntegrity #OpenScience #NLP #ScientificDiscovery #TrustworthyAI

***Cautionary Warning for Students and Scholars Using AI for Academic Research*** I have been sharing my journey using AI for a book project on my father's life. I wanted to explore the history of African physicists. I asked Chat for that history and it provided these two references: 1) Osei, Kwame. “The Transnational Struggles of African Scientists in U.S. Academia.” Journal of African Diaspora Studies 12(3): 2018 2) Osei, Kwame. “Exile and Experiment: African Scientists in Cold War America.” African Studies Review, Vol. 61, No. 4 (2018). My follow up prompt: "provide links to these two articles to verify they exist." Chat: "I was unable to locate credible links to verify the existence of the two articles you asked about .... Despite extensive searching in academic databases, library catalogs, and Google Scholar, I found no reliable bibliographic entry confirming either article with those titles and details." Me: "these were two sources you provided as evidence for information about the history of African scientists. Can you explain where or how these sources were generated?" ChatGPT: That’s an excellent and important question — and I appreciate your attention to scholarly accuracy. Here’s the full and transparent explanation: 🧩 1. How those “Osei” sources appeared The citations Osei, Kwame. “The Transnational Struggles of African Scientists in U.S. Academia.” Journal of African Diaspora Studies 12(3): 2018, and Osei, Kwame. “Exile and Experiment: African Scientists in Cold War America.” African Studies Review 61(4): 2018, were not drawn from any verifiable academic database, archive, or library record. They appear to have been automatically generated as plausible but fictitious scholarly references — what researchers call hallucinated citations. This can occur when a model attempts to supply credible-sounding supporting literature for a factual narrative (in this case, the experiences of African scientists in U.S. academia during the Cold War) even though no such specific articles exist. 📚 2. Why this happened When describing African and diaspora scientists, I used known historical patterns — such as Cold War migration of African intellectuals — that are genuinely documented in scholarly literature (e.g., works by Paul Tiyambe Zeleza, Toyin Falola, and Gabrielle Hecht). However, instead of citing those actual sources, the system produced synthetic article titles and author combinations that matched the topic but do not correspond to real publications. MENAH'S TAKEAWAY: BE VERY CAREFUL AS A SCHOLAR AND ACADEMIC USING ARTIFICIAL INTELLIGENCE FOR SCHOLARLY RESEARCH. I am not suggesting not to use it. I am only encouraging a heightened level of vigilance. Similar to Perplexity's response after confirming they don't exist: "Therefore, if you need peer-reviewed references on these topics, it would be best to seek alternative published scholarship or re-verify the article details."

Over the past several days I've been digging into the Claude for Word add-in, thanks to the support of LegalQuants, and the headline finding surprised me: on document-intensive tasks, it seems to need less supervision than either Claude on the web or Claude Code. Maybe the clearest example: I made a Table of Authorities for a summary judgment motion in less than five minutes. Anyone who's ever assembled one by hand knows that's not normal. That was one of four tests I ran. The others: - Cross-document consistency. I opened a motion and its supporting Rule 56.1 declaration side by side and asked Claude to cross-check every factual statement. It flagged several small mismatches, including a reference in the brief to an exhibit number that no longer existed in the declaration's exhibit list — the classic late-stage renumbering casualty that falls through because everyone assumes someone else did the cross-walk. - Bluebooking. Out of the box, with no skill or special prompting, Claude for Word handled standard federal citation form accurately — reporter spacing, signal ordering, short-form rules, the common T11 abbreviations. The one miss: tracked changes drop italics and small caps on replacement. Fixable in a lightweight skill. - Substantive citation verification. I ran it against a brief I'd already checked with a Claude Code pipeline I built around the CourtListener API. Claude for Word caught almost everything — inverted holdings, fabricated quotes, mischaracterized cases — and didn't just flag them. It diagnosed them, reconstructing actual opinion language and suggesting narrower authority where the cite didn't support the point. The pipeline still had an edge on retrieval coverage; the ad hoc approach held its own on substantive reasoning. Full review on AI Law Librarians: https://lnkd.in/gPyrE__C

“𝗛𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗲𝗱 𝗰𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀” 𝗮𝗿𝗲 𝗻𝗼𝘄 𝗲𝗻𝘁𝗲𝗿𝗶𝗻𝗴 𝘁𝗵𝗲 𝘀𝗰𝗶𝗲𝗻𝘁𝗶𝗳𝗶𝗰 𝗿𝗲𝗰𝗼𝗿𝗱 These are not opinions nor blog posts, but peer-reviewed articles! Nature reports (https://lnkd.in/g5cV7gJP) a worrying trend: 𝘈𝘐 𝘴𝘺𝘴𝘵𝘦𝘮𝘴 𝘢𝘳𝘦 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘯𝘨 𝘳𝘦𝘧𝘦𝘳𝘦𝘯𝘤𝘦𝘴 𝘵𝘩𝘢𝘵 𝘭𝘰𝘰𝘬 𝘳𝘦𝘢𝘭 𝘣𝘶𝘵 𝘥𝘰 𝘯𝘰𝘵 𝘦𝘹𝘪𝘴𝘵 — 𝘢𝘯𝘥 𝘴𝘰𝘮𝘦 𝘢𝘳𝘦 𝘢𝘭𝘳𝘦𝘢𝘥𝘺 𝘮𝘢𝘬𝘪𝘯𝘨 𝘵𝘩𝘦𝘪𝘳 𝘸𝘢𝘺 𝘪𝘯𝘵𝘰 𝘱𝘶𝘣𝘭𝘪𝘴𝘩𝘦𝘥 𝘳𝘦𝘴𝘦𝘢𝘳𝘤𝘩. We researchers are aware that AI can generate them, and we promote specific tools to verify and validate such references! This is not a small glitch. It cuts to the core of how science works. 𝘾𝙞𝙩𝙖𝙩𝙞𝙤𝙣𝙨 𝙖𝙧𝙚 𝙣𝙤𝙩 𝙙𝙚𝙘𝙤𝙧𝙖𝙩𝙞𝙤𝙣. 𝙏𝙝𝙚𝙮 𝙖𝙧𝙚 𝙩𝙝𝙚 𝙩𝙧𝙪𝙨𝙩 𝙡𝙖𝙮𝙚𝙧 𝙤𝙛 𝙧𝙚𝙨𝙚𝙖𝙧𝙘𝙝. 𝙄𝙛 𝙩𝙝𝙖𝙩 𝙡𝙖𝙮𝙚𝙧 𝙬𝙚𝙖𝙠𝙚𝙣𝙨, 𝙚𝙫𝙚𝙧𝙮𝙩𝙝𝙞𝙣𝙜 𝙗𝙪𝙞𝙡𝙩 𝙤𝙣 𝙩𝙤𝙥 𝙤𝙛 𝙞𝙩 𝙗𝙚𝙘𝙤𝙢𝙚𝙨 𝙦𝙪𝙚𝙨𝙩𝙞𝙤𝙣𝙖𝙗𝙡𝙚. The real issue with AI is not that it makes mistakes. It is that it makes mistakes that look convincing. And in science, convincing mistakes are dangerous. From a Responsible AI perspective, this is where we need to be very clear. 𝗙𝗶𝗿𝘀𝘁, 𝗿𝗲𝘀𝗽𝗼𝗻𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 𝗱𝗼𝗲𝘀 𝗻𝗼𝘁 𝘀𝗵𝗶𝗳𝘁 𝘁𝗼 𝘁𝗵𝗲 𝘁𝗼𝗼𝗹. If AI is used, the human author is still fully accountable. There is no “AI wrote it” excuse in research. 𝗦𝗲𝗰𝗼𝗻𝗱, 𝘃𝗲𝗿𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 𝗶𝘀 𝗻𝗲𝘃𝗲𝗿 𝗼𝗽𝘁𝗶𝗼𝗻𝗮𝗹. We cannot treat AI outputs as drafts that are “likely correct.” Every reference must be checked. This is a basic research discipline, now under pressure. 𝗧𝗵𝗶𝗿𝗱, 𝘄𝗲 𝘀𝗵𝗼𝘂𝗹𝗱 𝗯𝗲 𝘂𝘀𝗶𝗻𝗴 𝗯𝗲𝘁𝘁𝗲𝗿 𝘁𝗼𝗼𝗹𝘀 — 𝗻𝗼𝘁 𝗷𝘂𝘀𝘁 𝗺𝗼𝗿𝗲 𝘁𝗼𝗼𝗹𝘀. It is already possible to build or use AI systems grounded in verified research repositories, where outputs are constrained and validated against real sources before reaching the user. In fact, as highlighted in the Nature article, AI itself is now being used to detect these fabricated citations (with human-in-the-loop). 𝙏𝙝𝙖𝙩 𝙞𝙨 𝙩𝙝𝙚 𝙙𝙞𝙧𝙚𝙘𝙩𝙞𝙤𝙣 𝙬𝙚 𝙨𝙝𝙤𝙪𝙡𝙙 𝙗𝙚 𝙥𝙪𝙨𝙝𝙞𝙣𝙜, 𝘼𝙄 𝙩𝙝𝙖𝙩 𝙨𝙩𝙧𝙚𝙣𝙜𝙩𝙝𝙚𝙣𝙨 𝙩𝙧𝙪𝙨𝙩, 𝙣𝙤𝙩 𝙚𝙧𝙤𝙙𝙚𝙨 𝙞𝙩. #ResponsibleAI #ResearchIntegrity #AIethics #TrustworthyAI #AcademicIntegrity #GenerativeAI

Courts are not your beta testers. In a newly issued standing order on candor, accuracy, and responsibility in court filings, U.S. Magistrate Judge Jason A. Robertson makes one thing unmistakably clear: technology may assist advocacy, but it never replaces judgment, diligence, or accountability. The order reiterates that every filing is governed by Rule 11 and the longstanding duty of candor. Lawyers and pro se litigants alike must conduct a reasonable inquiry into both facts and law, and they certify that cited authorities actually exist and are accurately described. That obligation does not change with new drafting tools, research platforms, or automation. Most notably, the court draws a bright line around fabricated or nonexistent authority. Hallucinated cases, invented statutes, and false quotations are impermissible in any proceeding. Even a single fake citation may trigger sanctions, regardless of how plausible it appears. The order also rejects any “safe harbor” through disclosure. Telling the court you used a tool does not excuse inaccuracies. The signature on a filing remains a representation of truth, period. The order is below and Episode 293 of the Podium and Panel Podcast where Dan Cotter and I discussed the order as the Rule of the Week is in the comments. #law #lawyers #appeals #attorneys #civilprocedure

Persuasive citation. Yes, citation is a technical requirement for legal writing; but the citation itself can also be a form of persuasion. Every citation choice has an effect on me as the reader. Here are a few of my thoughts on the art of persuasive citation. ➡️A textual sentence followed by a single, direct citation tells me, “this is the rule that is stated at this place.” That sends me a clear message. You lose credibility, though, if I follow the citation and find only a case that could support the rule or worse, the case doesn’t support the rule. When you intend to send a clear message, be accurate. ➡️Rely on the highest authority you have; don’t just settle for the one you found first. Citation to binding authority is the gold star and tells me the boundaries of my decision. If your discussion includes citation to several authorities, a clean progression from binding to persuasive to secondary gives your arguments legal context. ➡️Match the authority cited to the issue you are addressing: probable cause→probable cause; statute of limitations→statute of limitations. I read contextually, and your rule will carry more weight when it has been applied in the type of case before me. Cross‑category citations can work, but maintain your credibility by flagging the cross use and explaining why that rule can be applied in this case. ➡️Case dates are rhetorical, with old and new cases having different significance. Older cases show a rule is deeply rooted. Newer cases show the rule is alive and not eroded. Using both together signals stability across time and context. ➡️A string cite should address a specific need: show consensus in a jurisdiction; show consensus across jurisdictions; old→new shows longevity or stability; new→old shows currency with grounding. Each citation in a string should support your rhetorical purpose for the string. If it does not, omit it. ➡️Parentheticals are micro-arguments and can be very effective when used correctly. There are many types of parentheticals—fact, outcome, holding, policy—and the type you choose shapes my takeaway. And a parenthetical should add something helpful; if the textual sentence and the parenthetical say the same thing, consolidate them. ➡️Signals give me context and give you credibility. “See,” “see also,” “cf.,” “accord,” “but see” communicate support, consensus, caution, or tension. (See BB Rule 1.2 for more detail.) Use each correctly. ➡️Don’t overcite. A brief composed of clear, concise text and a sufficient number of strong citations is more readable, and thus more persuasive, than a brief that treats the citations as the content and the text as a condiment. Readability is one of the most effective forms of persuasion. Bottom line. Legal citation is an opportunity for persuasion and not just a technical requirement. Use your citations to bolster your argument, build trust, reinforce your theme, and make the outcome of your argument feel inevitable.