![\"\"](\"https://thebibliomagician.wordpress.com/wp-content/uploads/2026/09/image.png?w=770\")
Nathalie Corn\u00e9e & Kat Hart
\n\n\n\nCitations aren’t the only way research shows up in policy. Sometimes it’s a named researcher, a nod to “the evidence,” or a reference tucked into the text with no formal citation at all. Kat Hart and Nathalie Corn\u00e9e dug into Overton’s data to find out how often that happens, and what it means for how we understand research’s reach into policy.
\n\n\n\nWhen we want to know if research has influenced policy, we usually turn to citations. They’re easy to spot and easy to count, so it makes sense that they’ve become the go-to signal for research influence.
\n\n\n\nBut citations are only part of the story. Policymakers don’t always follow academic conventions. Sometimes they name a researcher directly, reference a report rather than the paper behind it, or they simply gesture at “the evidence” without pointing to anything specific at all.
\n\n\n\nSo, we set out to ask a simple question: if we look past formal citations, what else might be going on? Using Overton’s data, we compared the scale of formal citations against these more informal mentions and tried to get a sense of what the gap between them might look like. This is an exploratory pilot rather than a definitive study, as the sample is far too small to draw firm conclusions. We hope its value lies more in how it reframes the question of what counts as 'research engaging with policy' (and vice versa) in the first place.
\n\n\n\nDefinitions of research impact are shifting. A recent Australian study published on LSE's impact blog made a useful point on this: institutional definitions of impact often don’t match how researchers actually work (O'Connor et al., 2026). Instead of asking “did this have impact?”, the authors suggest asking “what value does this create, and for whom?” It’s a small shift in framing, but it moves the conversation from proving impact to understanding it.
\n\n\n\n\n\n\n\n\nIf citation counts alone don’t capture the full picture, what other signals should we be paying attention to?
\n
We used the Overton Index as our starting point. At the time of the study, Overton indexed over 27 million policy documents from more than 190 countries and 3,100 sources, spanning everything from intergovernmental organisations like the UN and WHO to NGOs, think tanks, national governments, as well as regional and local bodies like Canadian provinces and UK devolved nations.
\n\n\n\nFor the pilot itself, we kept things deliberately focused. We took a random sample of 50 UK government policy documents from 2025 and used Overton’s AI classification to pull out document type, length, and authorship. Then came the manual part: we read and coded every single document by hand.
\n\n\n\nThis was a conscious choice. Automated tools are good at catching formal citations, but they miss the less obvious ways research shows up in policy text. We wanted to capture all of it, so we built a coding framework covering two broad categories:
\n\n\n\nFigure 1. Workflow for identifying and classifying attention to actors and objects in UK government policy documents.
\n\n\n\nFor each one, we recorded whether it was a citation (a formal, traceable reference) or a mention (an informal acknowledgement with no formal citation attached). Where a document referenced several things together, we logged each one individually rather than lumping them into a single entry. Across just 50 documents, that added up to over 2,200 individual references, which gave us plenty to work with despite the small sample size.
\n\n\n\nAcross our sample, we identified 2,244 objects and actors in total. Policy actors (think tanks, government bodies, IGOs) and policy evidence (past reports, transcripts, consultation outputs) were by far the most common categories in our data, together with in-house research accounting for around 75% of everything referenced.
\n\n\n\nPeer-reviewed publications, by comparison, made up just under 5% of all references. Generic, non-specific nods to “experts” or “previous research” outnumbered them, at around 9%. In other words, plenty of documents acknowledge that evidence exists somewhere, without pointing to anything specific.
\n\n\n\nPeer-reviewed articles were the type of evidence most likely to receive a full, formal citation. Commissioned research told a different story: it often got absorbed into the narrative (“the data suggests”, “interviews found”) with no formal attribution at all. Policy outputs sat somewhere in between, appearing as both citations and mentions in meaningful numbers.
\n\n\n\nAcross the whole sample, mentions were consistently more common than citations.
\n\n\n\nReferencing patterns varied a lot depending on who authored the document. Our five most prolific authors in the sample were the Department for Education, the Law Commission, UK Government more broadly, the Ministry of Justice, and the Department for Environment, Food and Rural Affairs (DEFRA) together with the Animal and Plant Health Agency (APHA). Each had a distinct pattern of what they referenced and how often.
\n\n\n\nA working paper from the Foreign and Commonwealth Office, written by commissioned academics, leaned heavily on peer-reviewed citations, as you’d expect. A similar paper from Technopolis, a consultancy commissioned by the Department for Science, Innovation and Technology (DSIT), leaned instead on funder acknowledgements and commissioned research. Same document type, same broad government context, very different citation behaviour.
\n\n\n\nWe also tested the intuitive assumption that longer documents would contain more citations. They don’t, at least not cleanly. Some of the highest citation counts did appear in longer documents, but plenty of short documents cited heavily too, and plenty of long ones cited very little. There’s no straightforward relationship between page count and citation volume in this sample.
\n\n\n\nMost documents in our sample didn’t have a formal bibliography at all, relying on footnotes instead. And even where a reference qualified as a formal citation, it only had roughly a 50/50 chance of actually appearing in a bibliography rather than just the body text. Citation habits clearly vary a lot from one policymaker to another.
\n\n\n\nThe research being cited mostly comes from a small number of places
\n\n\n\nOf the peer-reviewed research we identified, half came from UK universities alone, 43% came from international institutions, and the rest reflected UK and international collaborations.
\n\n\n\nA few things stood out consistently across the sample:
\n\n\n\nTaken together, these early findings suggest that citation counts alone may only capture a part of how research reaches policy.
\n\n\n\nWhere we think this work is genuinely useful is in how it reframes the question. Right now, most tools only ask whether a policy document cited a piece of research or not. Splitting that into citations and mentions lets us ask a more useful question: how is research actually showing up, even when there’s no formal reference at all? That feels like a real gap worth exploring.
\n\n\n\nThe coding framework itself, sorting references into different types of evidence and different types of actors, and tagging each one as a citation or a mention, is a first step towards making that visible. It gives us a way to describe the kinds of engagement that citation counts alone may miss, so it’s a starting point for more work.
\n\n\n\nThis also fits with a broader pattern often noted in research impact literature: that scholarly expertise tends to reach policy indirectly, through reviews, consultations, and expert commentary, rather than through direct citation. That raises a genuinely open question: how much of what we’re seeing is a passing reference, and how much reflects real engagement with the research itself?
\n\n\n\n\n\n\n\n\n\n\nBeing able to tell the difference matters if we want to understand not just whether research is being used, but how and why.
\n
This was a small, exploratory pilot, and we’re upfront about its limits. Fifty documents from one country over one year is a start, not a conclusion.
\n\n\n\nNext, we’d like to scale up the sample across a wider, more representative set of countries using an LLM to help with the coding. We’d also like to track whether the balance between citations and mentions is shifting over time and compare document types more systematically to understand whether citation behaviour is driven more by document function or by the authoring body. Beyond that, we want to develop ways to assess the depth of engagement behind a reference, not just whether it exists, and look at whether these patterns differ across policy domains, comparing something like health policy against education policy.
\n\n\n\nIf you have thoughts on this work, or you’re doing something similar, we’d love to hear from you. Get in touch at nathalie@overton.io or kat@overton.io.
\n\n\n\n![\"\"](\"https://thebibliomagician.wordpress.com/wp-content/uploads/2026/09/nathalie.png?w=704\")
Nathalie Corn\u00e9e helps the research and policy community around the world get the most out of Overton. She joined in 2023, bringing more than 20 years of experience across the pharma, research, and academic library sectors, always focused on highlighting the significance of research and supporting the researchers behind it, including through bibliometric analysis. She takes a practical, hands-on approach to helping users find the data they need and make sense of research impact.
\n![\"\"](\"https://thebibliomagician.wordpress.com/wp-content/uploads/2026/09/kat.png?w=200\")
Kat Hart is a Senior Analyst at Overton, where she curates policy data and uses bibliometrics to explore evidence-use patterns worldwide. Alongside this, she’s a PhD researcher at the University of Nottingham, studying UK water governance. She previously worked as a knowledge broker in UK higher education, focused on ‘what works’ in connecting academic research with public policy.
\n![\"\"](\"https://thebibliomagician.wordpress.com/wp-content/uploads/2019/07/c-cby-1.png\")
Unless it states otherwise, the content of the Bibliomagician is licensed under a Creative Commons Attribution 4.0 International License.
DOI: https://doi.org/10.59350/74r3r-67451
\n\n\n\n","doi":"https://doi.org/10.59350/a3dpx-knp02","guid":"https://thebibliomagician.wordpress.com/?p=5646","image":"https://thebibliomagician.wordpress.com/wp-content/uploads/2026/09/image.png?w=770","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1788739200,"reference":[{"id":"https://blogs.lse.ac.uk/impactofsocialsciences/2026/03/23/we-should-focus-less-on-research-impact-and-more-on-research-value/","unstructured":"O'Connor, R., Malde, S., Russell, W., & Haviland, M. (2026, March 23). We should focus less on research impact and more on research value. LSE Impact Blog."}],"rid":"xx8x7-m3t11","summary":"Nathalie Corn\u00e9e & Kat Hart Citations aren't the only way research shows up in policy. Sometimes it's a named researcher, a nod to \"the evidence,\" or a reference tucked into the text with no formal citation at all.","tags":["Altmetrics","Policy"],"title":"Cited or Mentioned? What We Learned About How Research Actually Shows Up in Policy","updated_at":1792434907,"url":"https://thebibliomagician.wordpress.com/2026/09/07/cited-or-mentioned-what-we-learned-about-how-research-actually-shows-up-in-policy/","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/0524sp257","name":"University of Bristol"}],"contributor_roles":[],"family":"Taylor","given":"Mike","url":"https://orcid.org/0000-0002-1003-5675"}],"blog":{"authors":[{"name":"Mike Taylor","url":"https://orcid.org/0000-0002-1003-5675"},{"name":"Matt Wedel","url":"https://orcid.org/0000-0001-6082-3103"}],"community_id":"0e13541f-417e-46c0-a859-65927249df72","created":1675209600,"current_feed_url":null,"description":"SV-POW! ... All sauropod vertebrae, except when we're talking about Open Access. ISSN 3033-3695","doi":"https://doi.org/10.59350/svpow","favicon":"https://rogue-scholar.org/api/communities/0e13541f-417e-46c0-a859-65927249df72/logo","feed_format":"application/atom+xml","feed_url":"https://svpow.com/feed/atom/","filter":null,"generator":"WordPress.com","home_page_url":"https://svpow.com","issn":"3033-3695","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"svpow","status":"active","subfield":"1911","title":"Sauropod Vertebra Picture of the Week","updated":1792433290,"use_api":true},"blog_name":"Sauropod Vertebra Picture of the Week","blog_slug":"svpow","content_html":"Yesterday, an AI posted a comment on this blog \u2014 or tried to. It was superficially helpful, relevant to the top of the post it was on, and turned out to contain that characteristic blend of things we already knew and things that are (subtly or blatantly) wrong. Needless to say, we shoved it in the spam folder.
\nThis is a good moment to remind everyone of our AI policy: SV-POW! is an AI-free zone. (Follow the link for the detail.)
\n![\"\"](\"https://svpow.wordpress.com/wp-content/uploads/2025/12/no-to-ai.png\")
That applies to comments as well as to our own posts.
\nDo not post AI-generated content as a comment(*). If you do, we may warn you once, or we may just block you. As Matt put it in an email:
\n\nMake it clear that this is absolutely zero-tolerance. A single instance of posting AI slop can get you perma-banned. People can go publicly crap their pants on the rest of the internet.
We’re interested in what our readers have to say, and in engaging with humans. Slop is not welcome here.
\n\n
(*) The sole possible exception, and be very careful about this, is posting clearly labelled AI output for the purpose of going on to critique it. But given how grumpy we’re both feeling about aslopalypse, it’s probably best not to risk it.
\n","doi":"https://doi.org/10.59350/98nbq-v9p28","guid":"https://svpow.com/?p=26095","image":"https://svpow.wordpress.com/wp-content/uploads/2025/12/no-to-ai.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1792368000,"rid":"299wf-mvw22","summary":"Yesterday, an AI posted a comment on this blog \u2014 or tried to. It was superficially helpful, relevant to the top of the post it was on, and turned out to contain that characteristic blend of things we already knew and things that are (subtly or blatantly) wrong.","tags":["Artificial Intelligence","LLM","Navel Blogging"],"title":"No AI comments. None. No exceptions.","updated_at":1792433711,"url":"https://svpow.com/2026/10/19/no-ai-comments-none-no-exceptions/","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"
AI-heavy week, but the throughline is what happens when the platforms and infrastructure under our work start to rot, from GitHub to preprint servers to peer-reviewed literature.
The GitHub ship is sinking, the lifeboats leak
Vibe-coded into a corner
The Lancet sues the AI that cites The Lancet
Hallucinated citations, now with a denominator
arXiv takes the banhammer out for AI-written work
David Bushell wrote a vivid eulogy for GitHub (\"GitHub used to be cool and now it's a lame slop graveyard\"), and a few days later GitLab's new CEO Bill Staples published \"GitLab Act 2\", which to me sounds like a flailing pivot toward agentic everything.
Bushell's case against GitHub is part vibes but contains an uptime chart that does look bad after the Microsoft acquisition.
When I posted this chart on Bluesky earlier this week, I had a few folks pointing to the deluge of AI coding agents and automated pull requests as the reason. But if you look closely at the chart, this service degradation long predated AI coding agents and ChatGPT itself.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!6ew2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fed1b0b11-6f3f-4159-b566-3c60cdedf5c5_1273x916.png\" "\\"\\"")
But: Git is not GitHub. He recommends Codeberg (running Forgejo) as the safe alternative, with self-hosted Forgejo as the power-user option. He also recommended GitLab, then added an edit:
oh dear, they've contracted the rot
The Staples letter is what he means. GitLab is reorganizing into roughly 60 smaller R&D teams, flattening management, exiting up to 30% of its country footprint, and (oh my): \"rewiring internal processes with AI agents, automating the reviews, approvals, and handoffs to speed us up.\" The strategic thesis is that \"software will be built by machines, directed by people.\"
The blog post calls git \"designed for human-rate commits\" and pitches a \"generational rebuild of the underlying infrastructure to handle agent-rate work as the default.\"
I'm not an agent power user, and maybe I'll retract my thoughts on this in a few months. But, if your agents are opening so many merge requests that git itself is the bottleneck, the answer is probably fewer agents.
It'll be interesting to see if this translates into actual migrations. Pretty much all of computational biology / bioinformatics lives on GitHub: software packages, snakemake/nf-core workflows, Bioconductor packages, lab repos, course materials. Don't forget about other infrastructure as well: GitHub Pages (e.g. pkgdown pages for R packages), CI w/ GitHub Actions, GitHub container registry, etc. Moving the social graph (issues, PRs, stars, discoverability) is enormous work, and I don't know if Codeberg could absorb the load. I can't imagine what a realistic and practical exit plan would look like.
I'm not usually one to give air to the anti-AI hype. Most of what I read here is boring and irritating. However, Shubham's \"I'm going back to writing code by hand\" is good. He spent ~30 weekends and >200 commits building k10s, a GPU-aware Kubernetes TUI, entirely through Claude. Then he sat down and read model.go for the first time. It was 1690 lines. He's archiving it and starting over in Rust.
AI builds features, not architecture; every prompt landed cleanly, the cumulative effect was a god object. Vibe-coding \"made everything feel cheap\" so scope crept from a niche GPU tool to a generic k9s clone. My favorite line:
like \"em-dash\" is to ai writing, \"god-object\" is to ai coding
His proposed remedy is putting architectural invariants into CLAUDE.md or AGENTS.md so the model sees them on every invocation.
See also the HN thread. One comment stuck out: \"Can't you just ask AI to break up large files into smaller ones and also explain how the code works so you can understand it?\" If the AI got you into the god-object hole, asking the AI to refactor its way out is probably not the move.
I was at a week-long innovation lab around AI and drug discovery this weekend. A thought in the back of my head all week was something I keep coming back to in my thinking in computational biology: the bottleneck in agentic science is evaluation, not generation. You can prompt your way to a working pipeline in a weekend. You can't prompt your way to knowing it's right.
Elsevier has joined a class-action lawsuit against Meta alleging Meta reproduced copyrighted works to train Llama. As Nature notes, this is the first AI copyright suit from major publishing houses. Elsevier publishes Cell and The Lancet, so a fair chunk of the biomedical literature is now formally a plaintiff against a frontier lab.
The lawsuit alleges Meta used Common Crawl and also that Meta downloaded and torrented works from LibGen and Sci-Hub. Meta's defense is the usual one: training is transformative use. US judges in two 2025 rulings reportedly distinguished between training (often transformative) and acquisition (the act of downloading copyrighted material), which is where torrenting from Sci-Hub is going to be a problem for Meta regardless of how fair use resolves.
Irony is dead.
Elsevier sells access to research funded by taxpayers and donated by authors who weren't paid for the manuscript and often paid Elsevier APCs to make it open. Meta scraped some of that content, allegedly via Sci-Hub, which exists precisely because Elsevier's access model is widely considered indefensible. Now Elsevier wants damages on behalf of authors who didn't get paid the first time around either. The most coherent position is probably that Meta should have paid for licensed access and that Elsevier should not be the entity collecting the check, but that isn't on offer.
If this case establishes that scraping paywalled academic content is infringement, the practical effect on open-science-trained models could be larger than the effect on Llama. Llama already exists. A future model that wants to train on biomedical literature now has a clearer legal target on its back.
A new Lancet correspondence by Topaz et al. audited 2.5 million biomedical papers for fabricated references, and the numbers are bad. The accompanying Comment by Bauchner and Rivara argues that any published paper with a fabricated reference should be retracted, which is a reasonable position that approximately no one will implement.
Topaz and colleagues pulled >100 million references from >2 million papers in PubMed Central's Open Access subset between Jan 2023 and Feb 2026. They kept the 97 million (77%) with a PMID, verified those against PubMed, Crossref, OpenAlex, and Google Scholar, used Claude to filter reference errors (misformatted but real) from genuine fabrications, and validated precision at 91% with three independent reviewers. They found 4,046 fabricated references across 2,810 papers.
The fabrication rate increased more than 12 times, from approximately four per 10,000 papers in 2023, to 51.3 per 10,000 papers in the fourth quarter of 2025
![\"\"](\"https://substackcdn.com/image/fetch/$s_!2FMa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F022d121d-53f2-4aec-bbed-04737155bb3f_2121x1070.png\"){kind=spacer}
The inflection point is mid-2024, which is when LLM-assisted papers would start clearing typical 100-200 day submission lags. One paper on ureteroileal anastomotic techniques had 18 of 30 verified references fabricated, each tailored to the surgical topic and attributed to real urologists. The authors also flag a paper-mill pattern: the same two-author pair appearing across 11 papers in a single surgical journal in 2025.
Read this along with #3 above. If Elsevier wins the case against Meta, the legal incentives push toward licensed-only training data. The Lancet audit suggests the academic literature itself is already meaningfully polluted by models trained on it. I think this will get worse before it gets better.
Yesterday, Thomas G. Dietterich (arXiv moderator for cs.LG) announced that arXiv will implement 1-year ban for papers containing incontrovertible evidence of unchecked LLM-generated errors, such as hallucinated references or results. Here's the full text of the thread:
Attention arXiv authors: Our Code of Conduct states that by signing your name as an author of a paper, each author takes full responsibility for all its contents, irrespective of how the contents were generated.
If generative AI tools generate inappropriate language, plagiarized content, biased content, errors, mistakes, incorrect references, or misleading content, and that output is included in scientific works, it is the responsibility of the author(s).
We have recently clarified our penalties for this. If a submission contains incontrovertible evidence that the authors did not check the results of LLM generation, this means we can't trust anything in the paper.
The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue.
Examples of incontrovertible evidence: hallucinated references, meta-comments from the LLM (\"here is a 200 word summary; would you like me to make any changes?\"; \"the data in this table is illustrative, fill it in with the real numbers from your experiments\").
I'm about as split on this one as the comments on the OP are. On one hand, GOOD. Hallucinated citations and unchecked AI-authored content is polluting the literature everywhere. I think this borderlines on scientific misconduct, and it destroys trust in the entire scientific enterprise.
Oh the other hand there are lots of problems with detection and enforcement. AI detection software is a cat and mouse game that doesn't work reliably. And on the enforcement side \u2014 what happens when you're a co-author on a paper with a hallucinated citation? Say I tell my grad student to work on the methods section and they get an undergrad to write a small section that they worked on, and said undergrad carelessly inserts a fabricated citation. If this paper makes its way onto arXiv, does that infraction go all the way up the chain with the banhammer coming down on all co-authors? Have you ever submitted a paper written with a bunch of co-authors and meticulously looked up every reference or checked that every parameter setting in the detailed methods section is actually a valid parameter for whatever tool was being used? I have.
I think a middle ground solution with a big red banner or warning flag on the abstract page or search results noting that the paper likely contains unverified AI-generated content.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!71Lh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4b57310-795f-4211-80bb-6fe9ab994d50_1448x1086.png\")
Again, the detection piece is difficult, and authors should be given a chance to respond before getting the scarlet letter. I think the 1-year ban is severe, and if applied to all co-authors, could be extremely damaging to one's career just because a middle author responsible for page 98 of a large supplemental info section was careless and wasn't carefully scrutinized. I also worry that such a ban could be inconsistently applied since there's a lot of subjectivity and guesswork involved with this endeavor.
","doi":"https://doi.org/10.59350/fe03p-s2z80","guid":"197328359","image":"https://substackcdn.com/image/fetch/$s_!71Lh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4b57310-795f-4211-80bb-6fe9ab994d50_1448x1086.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1778803200,"rid":"ne4rm-36y89","summary":"GitHub problems, vibe coding regrets, Elsevier sues Meta, fabricated citations, the arXiv banhammer for AI-generated content","tags":["Papers","AI"],"title":"Five Things (May 15, 2026): AI is why we can't have nice things","updated_at":1792423601,"url":"https://blog.stephenturner.us/p/five-things-may-15-2026","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"Ever find yourself looking through a pkgdown page or a Quarto book, copying and pasting code chunks from your browser into your IDE? I do, and it's a minor annoyance.1
My friend and colleague VP Nagraj published a new R package called ctrlvee that makes this a lot easier.
It does one thing. Put your cursor anywhere in an R script in Positron or RStudio, call the add-in, provide a URL, and a few milliseconds later you'll have all the code from that page in your editor, separated by chunk boundaries (along with some metadata and a note to check the license!).
![\"\"](\"https://substackcdn.com/image/fetch/$s_!7WR6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdbfbd671-9318-421e-93b9-8ea4f5ef9e9a_1410x782.png\")
The package README provides a demonstration using the \"Data Validation and QA\" chapter of my Data Science Team Training book (dstt.stephenturner.us).
Install the package: install.packages(\"ctrlvee\")
Run the add-in. In Positron you'll open the command palette, search for Run RStudio Addin, then extract external R code and insert inline. You'll get a modal asking you for a URL.
Paste one in. E.g., https://dstt.stephenturner.us/validation.html
The R code from the website appears in your editor \ud83d\ude80
Here's a demo.
Here's what the extracted/inserted code looks like, from this source.
# -----------------------------------------------------------------\n# Chunks fetched by ctrlvee from: https://dstt.stephenturner.us/validation.html\n# Strategy: Rendered HTML page\n# Date: 2026-05-16 05:14:44\n# Chunks: 8\n# NOTE: Check the source license before reusing this code.\n# -----------------------------------------------------------------\n\nflu <- data.frame(\n week = c(1, 2, 3, 4, 4),\n county = c(\"Fairfax\", \"Arlington\", NA, \"Loudoun\", \"Loudoun\"),\n disease = c(\"Flu\", \"Flu\", \"Flu\", \"Flu\", \"Flu\"),\n cases = c(23, 41, 18, -5, 12),\n rate = c(2.1, 3.8, 1.6, NA, 1.1)\n)\n\nflu\n\n# ---- chunk boundary ----\n\nif (any(flu$cases < 0, na.rm = TRUE)) {\n stop(\"Negative case counts detected. Inspect raw data before proceeding.\")\n}\n\n# ---- chunk boundary ----\n\nstopifnot(\n \"Negative case counts\" = all(flu$cases >= 0, na.rm = TRUE),\n \"Missing county values\" = !anyNA(flu$county),\n \"Duplicate records\" = !anyDuplicated(flu[, c(\"week\", \"county\")])\n)\n\n# ---- chunk boundary ----\n\ninstall.packages(\"pointblank\")\n\n# ---- chunk boundary ----\n\nlibrary(pointblank)\n\nagent <- create_agent(tbl = flu, label = \"Weekly flu surveillance\") |>\n col_vals_gte(\n columns = cases,\n value = 0,\n label = \"Case counts must be non-negative\"\n ) |>\n col_vals_not_null(\n columns = c(week, county),\n label = \"Week and county cannot be missing\"\n ) |>\n rows_distinct(\n columns = c(week, county),\n label = \"No duplicate week/county records\"\n ) |>\n interrogate()\n\nagent\n\n# ---- chunk boundary ----\n\ncreate_agent(tbl = flu, label = \"Weekly flu surveillance \u2014 extended\") |>\n col_is_numeric(\n columns = c(cases, rate),\n label = \"Case count and rate must be numeric\"\n ) |>\n col_vals_in_set(\n columns = disease,\n set = c(\"Flu\", \"COVID-19\", \"RSV\"),\n label = \"Disease must be from the approved list\"\n ) |>\n col_vals_between(\n columns = week,\n left = 1,\n right = 52,\n label = \"Week must be between 1 and 52\"\n ) |>\n col_vals_gte(\n columns = rate,\n value = 0,\n na_pass = TRUE,\n label = \"Rate must be non-negative (NAs allowed)\"\n ) |>\n interrogate()\n\n# ---- chunk boundary ----\n\nif (!all_passed(agent)) {\n stop(\"Data validation failed. Review the agent report before proceeding.\")\n}\n\n# ---- chunk boundary ----\n\nlibrary(readr)\nlibrary(pointblank)\n\nflu <- read_csv(\"data/flu-2024.csv\")\n\n# Validate immediately after reading\nagent <- create_agent(tbl = flu, label = \"flu-2024 validation\") |>\n col_vals_gte(columns = cases, value = 0, label = \"No negative counts\") |>\n col_vals_not_null(columns = c(week, county), label = \"No missing keys\") |>\n rows_distinct(columns = c(week, county), label = \"No duplicate records\") |>\n interrogate()\n\nif (!all_passed(agent)) {\n stop(\"Validation failed \u2014 see agent report above.\")\n}Sure, if the pkgdown page or Quarto book or whatever has source code on GitHub, you could find the source and open that up directly. However, you won't find the source for everything, and the source will be crowded by markdown narrative you might not want if you're just looking for the code.
Big week in AI in life sciences (AIxBio). The Nature drop this week included three papers on AI scientists alongside an editorial and a comment piece pushing back on the whole project. Add 's three-part series on AI for biology, a new RAND/Helena workshop report on AIxBio mitigations, 's take on AI in genomics, and a new study on AI in peer review and I'm landing on a theme this week: how fast should we let AI into the production of biological knowledge, and what gets lost if we don't slow down to ask?
's case for shaping AI-for-biology before it shapes us
RAND and Helena on AIxBio mitigations
on the state of AI in genomics
Nature's AI scientists week, and the editorial pushback
45 expert scientists review the reviewers
(Johns Hopkins, Center for Health Security) published a three-part series last week on shaping AI progress for biology and biosecurity.
Part 1 sets up the series. AI will compress decades of biological research into years, but cures won't arrive by default, and the same systems that enable them can lower the barrier to weaponizing pathogens. We need proactive policy on both sides.
Part 2 is where it gets interesting. Pannu lays out what she calls autonomous biological discovery: AI systems that automate every step of the research cycle, including managing the cycle itself, with orgs like Isomorphic Labs, FutureHouse, Ginkgo Bioworks and others entering the fray.
AI-enabled feedback loops will be able to extend beyond this, exploring parts of biological space that nature has not.
Evolution selects for reproductive success and gets stuck in fitness valleys. AI-driven design doesn't have that constraint. Whether that's a feature or a terrifying bug depends on what you're designing.
Part 3 centers on smallpox eradication: 171 years between Jenner's cowpox demonstration in 1796 and Henderson's 1967 campaign, and only 10 of those years were spent actually eradicating. Her conclusion is that the bottleneck wasn't tech, it was coordination and political will, so even if AI drives the marginal cost of biology research to zero, we shouldn't expect cures to deploy themselves.
RAND and Helena released the conference proceedings (full PDF here) from a January 2026 workshop on AI-enabled biological threats. 22 participants from frontier labs, biotech, biosecurity, and academia, working over two days in DC under Chatham House rules, with three threat scenarios: a millenarian nonstate group releasing a novel influenza A, an agroterrorism scenario targeting US wheat with an engineered fungal pathogen, and a state-sponsored insider attack on a semiconductor plant using a biofilm-forming bacterium. Scenarios were deliberately compressed and the document withholds specifics for infohazard reasons.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!9tUv!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb0f9b385-73bc-47f3-96db-79701af448b6_1012x1192.png\")
The pandemic group prioritized pathogen-agnostic physical defenses (high-quality PPE, indoor air quality with filtration and UV), a voluntary credentialing system called \"BioTrust\" modeled on ORCID, and AI \"guardian models\" for intent monitoring. The agroterrorism group went after holistic biosurveillance, information-sharing modeled on the Kansas Intelligence Fusion Center, and synthetic DNA screening for agricultural pathogens (which gets less attention than human-pathogen screening, and the participants thought that was the most well-scoped problem of the bunch). The critical infrastructure group went hardest on LLM-side interventions: investing in safeguards to better infer intent from prompt patterns, information-sharing between LLM companies via something like the Frontier Model Forum, federated cross-platform behavior analysis, and (this one is interesting) training LLMs to de-escalate malicious intent by adapting techniques from suicide prevention hotlines.
The participants (which included Twist Bioscience, Anthropic, Microsoft, SecureBio, Los Alamos, and others) kept running into the same wall, which the report names explicitly:
A central theme was that technical feasibility and political backing together determine a mitigation's success.
I.e., most of these will fail without sustained funding and political will, and almost none of them have either. The other recurring caveat was attribution. AI-enabled biological incidents may remain unattributed indefinitely, which weakens deterrence and complicates response authority.
ran the University of Chicago's annual genetics, genomics, and systems biology symposium last week, and turned the speaker lineup into seven points on the current state of AI in genomics.
The whole post is worth reading (<10 minutes). I'm just going to highlight a few.
First, the scaling laws may not hold for DNA. Evo 2 is 40 billion parameters trained on 9 trillion nucleotides spanning every domain of life. But Vishniakov et al. (2025) compared seven genomic foundation models against randomly initialized baselines of matched architecture across 52 tasks. The random baselines often matched or beat the pretrained models. Tang et al. (2025) found that raw one-hot encoded sequences are competitive with learned DNA-LM representations on regulatory genomics tasks. As Alex Lu put it at the symposium, DNA isn't natural language: low signal-to-noise, vast repetitive tracts, no obvious word or sentence analogs, and sparse functional elements that interact combinatorially across long distances.
Second, Arjun Krishnan's rule of thumb on benchmarks:
The best model is usually the one that is consistently number 2 in benchmarks across the literature.
Whoever publishes a model also designs the benchmark, and the benchmark almost always flatters the model. A model that's consistently competitive but rarely first-place is more likely to be genuinely strong than one that wins on the benchmark its own authors built. I'm stealing this.
Third, toward the end, is a succinct rule for trainees using AI tools:
If you can validate what the AI produces, namely, if you can do the task yourself comfortably and check whether the AI did it correctly, then you can probably use AI to do the task. Otherwise, you should probably do it yourself, even if it feels hard.
This is the cleanest articulation of the trainee-and-AI problem I've seen. I've written about this before, highlighting work from a new colleague and co-author, Arjun Krishnan:
The friction of doing it the slow way is often the friction of actually learning, the \"productive struggle\" I've written about here before. An AI tool that produces output you can't evaluate is just a black box you're forced to trust. I'd extend that beyond trainees, frankly.
On Tuesday, Nature published three full-length papers on AI scientists, an editorial that hedges, and a comment piece that pushes back. All on the same day.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!OMAo!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83677cd0-18cf-4891-87d4-066d4be0443b_3594x1887.jpeg\")
Ghareeb, A. E. et al. A multi-agent system for automating scientific discovery. Nature 1\u20133 (2026) doi:10.1038/s41586-026-10652-y.
Ayg\u00fcn, E. et al. An AI system to help scientists write expert-level empirical software. Nature 1\u20133 (2026) doi:10.1038/s41586-026-10658-6.
Gottweis, J. et al. Accelerating scientific discovery with Co-Scientist. Nature 1\u20133 (2026) doi:10.1038/s41586-026-10644-y.
Messeri, L. & Crockett, M. J. The uncritical adoption of AI in science is alarming \u2014 we urgently need guard rails. Nature 653, 675\u2013676 (2026).
Why AI cannot do good science without humans. Nature 653, 650\u2013650 (2026).
The three papers: Google DeepMind's ERA, an LLM-plus-tree-search system that discovered 40 novel single-cell analysis methods that outperformed the top human methods on a public leaderboard, and 14 COVID hospitalization forecasting models that beat the CDC ensemble. Google's Co-Scientist, a multi-agent system built on Gemini that helped identify in vitro\u2013validated drug repurposing candidates for acute myeloid leukemia and (in a now-famous demo) recovered an antibiotic-resistance hypothesis that a Imperial College team had spent a decade developing but hadn't yet published, in days. And FutureHouse's Robin, which autonomously proposed enhancing RPE phagocytosis as a strategy for dry AMD, identified ripasudil (a clinically used ROCK inhibitor never previously proposed for AMD) as a candidate, validated it in vitro, then proposed an RNA-seq follow-up that fingered ABCA1 as a possible novel target. All hypotheses, experimental directions, data analyses, and main-text figures in the Robin paper were produced by Robin.
These are real results. With that throat-clearing out of the way\u2014
Then there's the editorial, \"Why AI cannot do good science without humans\", which is mostly anodyne until the closing paragraph:
Scientists should not allow a negative view of AI to drive them away from exploring the possibilities that AI co-scientists might hold for research. Equally, however, they must rise above the din of AI hype and advocate for their own importance, to remind the wider public, funders and fellow researchers that science still needs humanity, and that not every grant proposal need include an AI project.
Again: not every grant proposal need include an AI project.
Once more: not every grant proposal need include an AI project.
I read this as the editorial board, deliberately, on the day they published three papers about AI scientists, telling reviewers and program officers not to use \"no AI angle\" as a reason to triage a proposal.
The comment piece is more pointed. Lisa Messeri (Yale anthropology) and M. J. Crockett (Princeton psychology) wrote \"Uncritical use of AI in science needs reality check\". Some empirical claims: Hao et al. (2026) analyzed 41.3 million papers across biology, medicine, chemistry, physics, materials science, and geology and concluded that AI adoption seems to \"induce authors to converge on the same solutions to known problems rather than create new ones.\" I wrote about this earlier this year:
Kusumegi et al. (2025) looked at 264,125 papers and found that in LLM-assisted papers, good writing stopped being a useful heuristic for scientific quality. Organization Science audited 6,957 submissions from 2021 to 2026 and found LLM-assisted papers had poorer scientific quality by acceptance rate. The closing argument is about deskilling: cleaning raw data, reading and summarizing the literature, the entry-level grunt work that AI is now offered as a solution for, is also how scientists develop the tacit knowledge needed to supervise AI-assisted workflows. If trainees don't develop those skills, who oversees the AI?
Three papers showcasing autonomous discovery, an editorial gently telling reviewers not to fetishize AI angles in proposals, and a comment piece arguing that the productivity gains may be hollowing out the next generation. Read them together. Or, if you don't have time, listen to Nature's podcast.
One more, and this connects to a paper I co-authored. A preprint went up at arXiv:2605.20668 titled \"On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists.\" Big study. 45 domain scientists annotated reviews of Nature-family papers, comparing official human reviewers against three frontier LLM agents. Headline results:
On aggregate review-item quality, all three AI reviewers exceed the lowest-rated human, and GPT-5.2 exceeds the top-rated human.
AI reviewers raise more significant items but with lower correctness.
Replacing one human reviewer with one AI reviewer minimally erodes panel diversity, because human reviewers themselves surface largely disjoint sets of criticisms.
AI reviewers can augment but not replace a human panel.
Current frontier AI reviewers in an agentic framework provide genuine value on the rigor- and code-heavy aspects of peer review, while systematically failing on the field-context aspects.
On that last point: it's the same argument that Agnieszka Swiatecka-Urban, Arjun Krishnan, and I argued for in our preprint earlier this year.
Our claim was that AI is best deployed as a rubric enforcer for the systematic, criterion-checkable parts of review (consistency between scores and comments, statistical reporting, completeness of evaluation, internal consistency of reviewer reasoning) while humans retain authority on the parts that depend on argumentative-world knowledge (novelty, feasibility, recognizing creative leaps, judging whether an ambitious proposal might fail spectacularly or succeed brilliantly). The arXiv paper, working with a completely different methodology and 45 domain scientists doing item-level annotation of real Nature-family reviews, lands in the same place. AI is strong on rigor and code, weak on field context.
","doi":"https://doi.org/10.59350/8103y-x2w56","guid":"198822549","image":"https://substackcdn.com/image/fetch/$s_!OMAo!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83677cd0-18cf-4891-87d4-066d4be0443b_3594x1887.jpeg","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1779408000,"rid":"hkxas-9dy42","summary":"Jassi Pannu on AI and biosecurity, RAND/Helena AIxBio biosecurity mitigations, Blekhman on genomics AI, Nature's AI scientists week, AI in peer review.","tags":["Papers","Biosecurity","AI"],"title":"Five Things (May 23, 2026): AI in life sciences","updated_at":1792423596,"url":"https://blog.stephenturner.us/p/five-things-may-23-2026-aixbio","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"I originally wrote this for the AI Exchange @ UVA Substack newsletter on May 5, 2026. Even if you're not at UVA I highly recommend subscribing. Ryan Wright and Varun Korisapati are publishing some really interesting stuff over there.
A recent Nature paper from UW and Allen AI introduces OpenScholar, a retrieval-augmented language model that outperforms GPT-4o1 on scientific literature tasks despite being a fraction of its size.
Asai, et al. (2026). Synthesizing scientific literature with retrieval-augmented language models. Nature. https://doi.org/10.1038/s41586-025-10072-4
OpenScholar addresses a problem with using LLMs for research: citation hallucination.2 When asked to cite recent literature across fields like computer science and biomedicine, GPT-4o3 fabricates citations 78-90% of the time. The references look plausible, but the papers simply don't exist.
OpenScholar tries to ground responses in a data store of 45 million open-access papers with 236 million passage embeddings. Instead of generating citations from parametric memory, it retrieves relevant passages, synthesizes them, and iteratively refines its output through a self-feedback loop. The model drafts a response, critiques it, retrieves additional context if gaps are identified, and revises. This cycle continues until the output meets quality thresholds for factuality and citation accuracy.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!xQMY!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F92da1854-8ac8-422f-ba85-e46ee3c37613_1268x450.png\")
On ScholarQABench (a new benchmark the team developed with nearly 3,000 expert-written queries), OpenScholar-8B outperformed GPT-4o by 6.1% in correctness and beat PaperQA2 by 5.5%.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!gJFG!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F92b710c8-b1e0-4a9f-89a2-e0285adf87f7_1263x298.png)
When the team ran human evaluations with 16 PhD-level experts across computer science, physics, and biomedicine, experts preferred OpenScholar's responses over human-written answers 51% of the time (70% when using GPT-4o as the backbone).
![\"\"](\"https://substackcdn.com/image/fetch/$s_!qOMd!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd2e8b8e4-345b-4ae4-8072-e66294a5a780_1266x314.png)
Everything is released: models, training data, data store, retrieval indices, benchmark, and a public demo. The demo has already attracted 30,000+ users generating 90,000 queries.
Training data: https://huggingface.co/datasets/OpenSciLM/OS_Train_Data
Benchmarking: https://github.com/AkariAsai/ScholarQABench/tree/main/data
Queries: https://huggingface.co/datasets/allenai/openscilm_queries
OpenScholar code: https://github.com/AkariAsai/OpenScholar
Code to run the evals: https://github.com/AkariAsai/ScholarQABench
Public demo: https://openscholar.allen.ai
A notable limitation: the benchmarking predates current frontier models, and only compares OpenScholar to now-deprecated ancient models. recently remarked on a different paper to this effect:
![\"\"](\"https://substackcdn.com/image/fetch/$s_!3_Dx!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F822619c7-e134-4e72-b817-94f204781e83_637x323.png\" "\\"\\"")
That is: Benchmarks age fast. If a weaker model is close, a better model will often clear the bar later, and we learn little from the gap.
GPT-5.5 with extended thinking, Claude Opus 4.7, and commercial tools with deep research capabilities (like Gemini Deep Research or Perplexity Pro's updated pipelines) remain untested. Whether domain-specialized RAG maintains its edge against these newer systems is an open question.
See also my previous post on Consensus, and wiring up your own Zotero library to it, to be able to ask and answer questions about your own literature collection, find gaps, and fill those gaps in.
On June 17 2026 at noon I'll be teaching a workshop for UVA's AI Research Initiative AI upskilling series: Smarter Literature Reviews with AI-Powered Tools. You can register for the workshop here. Grab your spot now!
Yes, GPT-4o. A deprecated model that no one uses anymore.
I don't know if someone else has used the word previously, but I'm calling these \"hallucitations.\"
Personal anecdata: the frontier models at the actual frontier (GPT-5.4, Opus 4.7, as of this writing), especially with extended thinking enabled, have far fewer hallucitations than GPT-4o or other last-generated deprecated models. See the end of the post.
A short week for me here. Just ahead of the Opus 4.8 launch, two of these (writing/creativity and the despair of professors) hit the same theme, so they're stacked at the top. Next, the ESMFold2 release I haven't fully digested, a message from the pope on AI, and a piece of NIH news for researchers with foreign co-authors.
Two pieces from different authors converging on a similar point, and one I'll be writing about next week as well.
Ethan Mollick's essay Choosing to Stay Human isn't necessarily arguing against using AI or even AI for writing, but focuses on what to hand over. He's fine offloading phone numbers, arithmetic, cursive. He's not fine reflexively offloading writing, because writing is one of those tasks where the doing is the point. He cites his own BCG consultant study and a pair of Wharton-affiliated education experiments where small differences in classroom AI deployment produced opposite outcomes. In Turkey, students with plain ChatGPT did worse on tests than students without it. In Taipei, students with an AI tutor that gave personalized problem sequences scored 0.15 SD higher, roughly six to nine months of additional schooling.
Related is companion piece is Rebecca Winthrop's NYT op-ed on a Georgetown research program led by Adam Green that has been tracking college application essays before and after ChatGPT. In a study of 370,000+ personal statements, post-ChatGPT essays got rated as more creative by human judges while offering fewer novel ideas. A separate study found human-written essays contained up to 8x more new ideas than AI-assisted ones. The homogenizing effect was largest on students furthest from the mean, including neurodivergent students and racial and linguistic minorities. I.e., the kids whose voices are most distinctive are the ones most flattened by the assist.
Stay tuned next week. I'm writing a very short piece on the topic.
The natural sequel is Jay Caspian Kang's New Yorker piece, \"The Despair of the Professor in the Age of A.I.\" collecting testimonials from faculty members about what AI has done to their teaching. Hard to read. But nothing in it is surprising (I'd guess most academics have heard versions of these stories from colleagues).
The piece opens with Jane Sloan Peters, a religious studies professor at Mount Saint Vincent, describing a course she'd taught for years called \"Letters from Prison.\" Before AI, students struggled to land on themes and arrived somewhere personal through that struggle. Last year, every one of her students turned in something polished and empty.
From Beth Ritter-Conn at Belmont, on her honors students:
The tipping point was last year when I had Honors students\u2014Honors students!\u2014using A.I. to write reflection journals. Literally the only task there is \"tell me what you are thinking inside your own head.\" There is no right or wrong answer. It's just, Give me your thoughts on this thing. And I had students who outsourced that task to the robots.
From Susanna Boxall, a philosophy lecturer at Chico State, on online classes:
Now, online classes are a simulacrum of education: the students pretend to learn, and I have to pretend that I am teaching them something.
From Neal Hebert at Grambling State, who teaches theater and now assigns plays too obscure for ChatGPT to know about:
I've stopped being a collaborator in these intro courses and started being a plagiarism cop, and I do resent that a bit.
And from Jeremiah Croster, who teaches English at Houston City College:
The get-a-degree approach was already winning even before A.I., but now that it's here, the education part is starting to feel like something someone will write about in a history book. Or maybe A.I. will do it.
Not every voice in the piece is despairing. Auyon Siddiq at UCLA Anderson made his stats exam fully AI-permitted and reports the average is still 75%, because students who don't understand the material aren't saved by the tool. Daniel Silver at Toronto Scarborough redesigned his sociology assignments as multi-agent simulations and says the best final projects showed more creativity than what he used to get. So it's not uniformly grim, but the dominant note is loss.
Biohub (formerly known as EvolutionaryScale, now merged with the CZ Biohub network) announced ESMFold2 and a new ESM Cambrian (ESMC) family of protein language models this week, alongside an atlas of 6.8 billion protein sequences with 1.1 billion predicted structures. The paper is long and I haven't read it, so I'll be careful here.
What Alex Rives says in his thread, in his words: ESMFold2 is state of the art on protein interactions, especially antibody-antigen complexes. They designed and experimentally validated miniprotein binders and single-chain antibodies against five therapeutic targets with good hit rates including a PD-L1 minibinder that restores T-cell signaling at functional IC50 comparable to atezolizumab. They describe this as evidence that the model has materialized \"a world model of protein biology.\"
What is a \"world model\" anyway? Seems like \"world model\" is doing a lot of marketing work in AI right now, with no agreed-on technical meaning. Whether calling it a \"world model\" adds anything beyond \"the representations are structured in biologically meaningful ways\" is a fair question, right?
Either way, a quick look at their press release benchmarks and binder design results all look exciting. I'm looking forward to seeing what holds up under peer review.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!IPZi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F07b6ef47-ff05-4350-9de4-6b6b78cecaa1_1260x571.png\")
Jill Lepore's New Yorker piece on Pope Leo XIV's new encyclical Magnifica Humanitas provides a nice overview for those of us without time to read the 40k+ word papal document on AI. Tech leaders have spent 20 years borrowing the rhetoric of mission, salvation, and disruption from religion, and the first American Pope, who took his name in reference to Leo XIII's 1891 Rerum Novarum on industrial labor, has decided to reclaim the territory. From Lepore:
Little in the encyclical is surprising; its force lies in its being said all at once.
I haven't read the whole encyclical yet so I'm not going to summarize it. It's interesting that (and this is partly Lepore's read but also visible in a quick skim of the text itself) part of the encyclical is about language and writing. There's a whole subsection on \"an ecology of communication.\" The Pope and Ethan Mollick and Rebecca Winthrop in #1 above are, in different vocabularies, worried about the same thing, that we are losing the conditions under which humans develop their own thoughts.
Our first task is neither to demonize nor idolize technological tools, but to utilize them on the basis of a fundamental principle, namely that truth is a common good and not the property of those with power or influence. We must therefore promote an ecology of communication.
Just pausing to say here that I find it amusing and puzzling that the company I keep who'd normally be first in line to criticize the Church, its anti-scientific doctrine, and its history of abuses, are now lining up behind the Pope as soon as he says something critical of AI. Cat's post on the topic stuck with me.
Jeffrey Brainard at Science reports that NIH grant managers are privately telling investigators they need pre-approval for any co-authorship with a scholar affiliated with a foreign institution, even when all the work was done in the United States. NASA is reportedly doing something similar for collaborations with researchers in China. Neither agency has issued public guidance.
The mechanism is that NIH has long required approval for grants with a \"foreign component,\" historically defined as significant scientific work performed outside the U.S. Now NIH is suggesting that mere coauthorship with someone at a foreign institution now counts as a foreign component, regardless of where the work happened. NIH is asking grantees to remove papers with foreign coauthors from annual progress reports if no foreign component was originally approved. An NIGMS email seen by Science asks institutions to promise that flagged U.S. authors won't collaborate with the relevant foreign coauthors in the future.
This creates pressure to remove foreign-affiliated coauthors from papers before submission, which violates standard publication ethics. Foreign-affiliated coauthors include visiting students and postdocs temporarily in the U.S., collaborators who donated reagents but didn't run experiments, and U.S.-trained scientists who moved abroad after the work was done. The rule is seems to be administered through private emails rather than formal guidance which means it can shift without anyone being able to point to the rule that changed.
","doi":"https://doi.org/10.59350/dandr-t9984","guid":"199512579","image":"https://substackcdn.com/image/fetch/$s_!IPZi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F07b6ef47-ff05-4350-9de4-6b6b78cecaa1_1260x571.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1780012800,"rid":"2mask-hbg06","summary":"Writing and thinking, the despair of the professor, ESMFold2 and a world model of proteins, Pope Leo on AI, NIH's foreign co-author crackdown","tags":["Papers","AI"],"title":"Five Things (May 29, 2026): AI & writing, ESM","updated_at":1792423590,"url":"https://blog.stephenturner.us/p/five-things-may-29-2026-ai-writing-esm","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"Anthropic provides a free, self-paced course catalog on covering Claude, Claude Code, the Claude API, MCP, and several AI fluency tracks tailored to educators, students, and nonprofits. Available at anthropic.skilljar.com.
I've taken a couple of these. They're short. If it matters to you, when you complete the course you can get a certificate that you can wire up to your LinkedIn profile, which link out to the verified cert (example).
![\"\"](\"https://substackcdn.com/image/fetch/$s_!-ap9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Faa77796a-7aef-4c63-aac7-1ee7c5028a7f_432x611.png\")
Full list as of this week below.
Claude 101: Learn how to use Claude for everyday work tasks, understand core features, and explore resources for more advanced learning on other topics.
Claude Code 101: Learn how to use Claude Code effectively in your daily development workflow.
Introduction to Claude Cowork: Learn to work alongside Claude on your real files and projects. This hands-on course covers the Cowork task loop, plugins and skills, file and research workflows, and how to steer multi-step work responsibly, so you're productive in your first week.
Claude Code in Action: Integrate Claude Code into your development workflow.
AI Fluency: Framework & Foundations: Learn to collaborate with AI systems effectively, efficiently, ethically, and safely.
Building with the Claude API: This comprehensive course covers the full spectrum of working with Anthropic models using the Claude API.
Introduction to Model Context Protocol: Learn to build Model Context Protocol servers and clients from scratch using Python. Master MCP's three core primitives (tools, resources, and prompts) to connect Claude with external services.
AI Fluency for educators: This course empowers faculty, instructional designers, and educational leaders to apply AI Fluency into their own teaching practice and institutional strategy.
AI Fluency for students: This course empowers students to develop AI Fluency skills that enhance learning, career planning, and academic success through responsible AI collaboration.
Model Context Protocol: Advanced Topics: Discover advanced Model Context Protocol implementation patterns including sampling, notifications, file system access, and transport mechanisms for production MCP server development.
Claude with Amazon Bedrock: As part of an accreditation program created for AWS, Anthropic launched a first-of-its-kind training for AWS employees. Here's the full course so you can follow along.
Claude with Google Cloud's Vertex AI: This comprehensive course covers the full spectrum of working with Anthropic models through Google Cloud's Vertex AI.
Teaching AI Fluency: This course empowers academic faculty, instructional designers, and others to teach and assess AI Fluency in instructor-led settings.
AI Fluency for nonprofits: This course empowers nonprofit professionals to develop AI fluency in order to increase organizational impact and efficiency while staying true to their mission and values.
Introduction to agent skills: Learn how to build, configure, and share Skills in Claude Code, reusable markdown instructions that Claude automatically applies to the right tasks at the right time. This course takes you from creating your first Skill to distributing them across teams and troubleshooting common issues.
Introduction to subagents: Learn how to use and create sub-agents in Claude Code to manage context, delegate tasks, and build specialized workflows that keep your main conversation clean and focused.
AI Capabilities and Limitations: An introductory course about how AI works.
","doi":"https://doi.org/10.59350/wshf7-c5h65","guid":"196772830","image":"https://substackcdn.com/image/fetch/$s_!-ap9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Faa77796a-7aef-4c63-aac7-1ee7c5028a7f_432x611.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1780185600,"rid":"h404v-zmt61","summary":"17 free, self-paced courses from Anthropic covering Claude, Claude Code, the API, MCP, agent skills, and subagents, plus AI fluency tracks for educators, students, and nonprofits","tags":["TIL","AI"],"title":"Free Claude Courses from Anthropic","updated_at":1792423588,"url":"https://blog.stephenturner.us/p/free-claude-courses-from-anthropic","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"For the last few years I've done a Sober October1 where I abstain from any alcohol for the entire month. I don't drink much these days anyway, but a solid month of teetotaling abstinence is a nice reset button that turns something habitual into something I mindfully enjoy with friends and family.
I've developed a similar habitual relationship with AI. I reach for it thoughtlessly when I see an essay that's a little longer than I want to read,2 or when I don't feel like writing that cover letter for a manuscript,3 or when I want to avoid the mental strain of thinking hard about the architecture of whatever code I'm working on.4
AI deskilling is real. A couple weeks ago I was writing specific aims for a proposal with a few brilliant colleagues here in SDS, and I felt the constant urge to open up a Claude window. Just to write aims! In another setting, while in a coaching session with one of my CSTE Data Science Team Training teams last week, I opened Positron to demonstrate some fairly basic tidyr rectangling I've done hundreds of times, and couldn't easily remember how to use the pivot_*() functions. It felt like coming back to a foreign language after a few years without using it.5 AI deskilling is real, and if you use these tools with any regularity you know it deep down.
Next month I'm doing an AI Dry July.6 I invite you to join me.
I'm going to resist using generative AI as much as possible for the whole month. With alcohol in Sober October the rule is easy \u20147 if it contains ethanol, I'm not having it. It's a little more difficult with AI. You can't do a web search or read a product without AI overviews. But generally, I'm not going to reach for ChatGPT or Claude for reading, writing, or for most coding problems.8 I'm teaching a new class this fall (Genomics Foundations for Data Scientists). I'm going to ask my students to struggle through some of the readings and assignments,9 so it's only fair if I struggle through creating those assignments and course materials. I'm exhausted from reading AI slop in everything from academic literature to the New York Times to many of the blogs and newsletters I personally recommend, so it's about time to stop compulsively reaching for AI to polish my own writing.10
I don't really care that my mental arithmetic is terrible because of calculators, or that my navigation skills aren't what they were back in the pre-GPS days of the 90s (the best decade). I want to avoid the habitual slide into offloading some part of nearly every cognitive task to the chatbot. At least for a month, to re-center myself and take stock of where the deskilling is creeping in.
I know I've written very positively about AI in this newsletter, in everything from writing code to peer review. And I'll continue to do so. I'm neither an AI evangelical nor an AI doomer, and I'll highlight problems where I see them. Even in 2026 it's possible (gasp!) to have nuanced views. And I'm starting to think habitual and compulsive AI use is probably not that great for me over the long run.
We'll see how this works out. I'll write something at the end of the next month on how it went and what I took away from it.
TIL this is known as Ocsober in Australia, where the also have a Dry July. Others here in the U.S. do a Dry January, but where I live in Virginia we usually get at least one heavy snow in January that shuts down schools for a few days, where a Lord of the Rings marathon with a good bourbon is required.
Shamefully, I find myself doing this with real essays from real writers I know, clacking away on real keyboards with their real human fingers.
I went so far as to write a Claude skill that covers my tracks for writing I don't want to do.
Last year I spent some time in Spain, Mexico, and Costa Rica, and could comfortably carry a 30 minute conversation with a local in Spanish about my family, our interests, why we're going on this snake-infested hike, or where to get a good paella. Recently when trying to converse with a Salvadoran guy I hired to do some handiwork around my house, just a year later without much practice I struggled with getting basic concepts across. It's not like riding a bike, and I'm afraid that neither are all the things we do with AI, unassisted.
Why July? Why not start now, June 1, as I'm writing this? It's not just the rhyme. I'm teaching several workshops on AI-enhanced literature review and synthesis this month.
The em dash here and elsewhere in this newsletter reflects my stylistic preference and should not be interpreted as evidence of AI-assisted text generation.
I say most coding problems here because the place I'd have historically gone to ask questions, Stack Overflow, is dying a well-deserved death. I don't write as much code these days as I used to, but I'm leaving myself a little wiggle room here.
Wish me luck. I'm looking forward to having undergraduates in the classroom with our masters students, and I plan to assign seminal primary research papers and \"journal club\"-like oral discussions for assessments. I have no idea how this will go with the 2026 cohort of students who haven't experienced college (or part of high school) without ChatGPT.
I'm not the only one. Wharton School professor recently wrote about Choosing to Stay Human, and , director of the Center for Universal Education at the Brookings Institution, recently wrote something similar in an op-ed at the New York Times.
Before the usual five, one thing that isn't optional reading. Pay attention.
I'm not an activist, and I don't use this space for advocacy. As a rule, I keep my opinions about research politics and policy off the site (and anywhere else online in general). I'm breaking that rule today, because this reaches every part of the scientific research enterprise and whatever role the United States still hopes to play in it.
The Office of Management and Budget has proposed rewriting 2 CFR Part 200, the rule that governs how every federal grant gets spent. A Science editorial lays out what that would mean for research: every funding decision routed through political review, multiyear grants terminable with no due process, and case-by-case approval for any award that spends a dollar outside the US, which would effectively end most international collaboration. Steve Usdin of BioCentury, quoted in the editorial, called the drug industry's silence \"complicity in the destruction of US science.\" The Science editorial ends with a rallying call:
\ud83d\udea8 The red light is now flashing. All hands, report to stations \ud83d\udea8
, who spent >20 years as an NIH program officer, writes in an essay making the point that this is much larger than science. 2 CFR Part 200 is the universal framework for federal grants, so the same political override and discretionary termination provisions land on Medicaid, Title I schools, highway funds, and tribal health programs.
The comment period closes July 13. Ginexi's how-to guide is a great place to start, and mechanics are important - since a thousand identical form letters count as one comment, so write your own, in plain language, and cite the specific provisions that would hurt your work. If you hold, have ever held, or in the future want to hold a federal grant, this is ten minutes well spent.
Now, onto the five things that interested me this week. Most of this week circles one question, whether we can measure what AI models actually do and trust the measurement. OpenAI decides the answer is yes and is shipping a life-sciences model to vetted partners.
Open models stay (only!) about four months behind the frontier.
Where evals actually sit in the biorisk evidence stack.
DARPA and NSF want universities for AI Forge (bring an IP-sharing agreement).
Two papers demonstrate AIxBio refusals tell you almost nothing.
OpenAI hands GPT-Rosalind to biodefense partners.
Epoch AI put a number on this open/frontier model gap: since January, the best open-weight models have trailed the closed frontier by about 4 months, or 8 points on Epoch's Capabilities Index, roughly the distance from GPT-5 to GPT-5.5.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!7t7G!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fff1f7107-dbe1-4ca3-b96e-475d3fea48eb_2400x1820.png\")
The \"four months\" figure uses a generous definition of catching up: an open model counts if it beats the old frontier in at least 5% of bootstrap samples. Epoch also notes that closed labs sit on their most capable models for safety or commercial reasons, so four months probably understates it.
As frontier models move to token/usage-based pricing, the open option is real and usable, they're not terribly behind on some benchmarks. Whether those benchmark points translate into the work you actually do is a separate question, and section 4 suggests the answer is messier than a single index implies.
at answers a recurring jab at AI biorisk benchmarks (they don't measure what we care about, so why bother) by placing evals in a four-tier hierarchy of evidence.
Tier 1 is arguing from first principles, which got the field started but convinces no skeptic. Tier 2 is evals, where we are now. Tier 3 is real-world uplift studies, the gold standard and the expensive one. Tier 4 is an actual incident, which nobody wants as a data point.
evals provide the best cheap, comparable and repeatable evidence we can get
I really liked the section G\u00f6tting calls the rocky terrain between tiers 2 and 3, so much that I'm quoting the whole segment in full below. A well-powered uplift study costs millions and goes stale the moment the next model ships. Jasper proposes a bridge: if you collect rich data on how participants used the model during an uplift study, you can map their failures and successes back onto benchmark tasks, building what he calls correlates of uplift. That gives you a cheap proxy you can re-run on every new model instead of re-running the whole study. A small number of expensive studies can underpin a much larger cheap monitoring effort.
Assume you ran your large uplift study, found some effect, and two months later, the new generation of frontier models is released. Can you make any sensible claim about the impact on real-world risk of these models (besides \"probably as good as the last gen\") without re-running everything? Maybe! You carefully approach the edge of the cost cliff you valiantly scaled and peer down to the vast plain of evals again.
If rich, qualitative data on LLM usage were collected during the uplift study, you should in principle be able to map the failures and AI-assisted successes of your study participants onto benchmarks measuring a corresponding capability. If model A failed to guide participants around a common methodological pitfall, and model B didn't, is the same pattern detectable in a benchmark task about this method? Is the delta in overall uplift-effectiveness between models or model generations the same as the delta in the relevant benchmark results? Did you find interesting model differences and behaviors that don't yet map onto a benchmark, but such a benchmark could be made? Did you encounter missing refusals to questions that should be refused in hindsight? All these can let you draw connections between evals and real-world uplift studies.
Finding these connections and correlations between uplift studies and in silico evals (\"correlates of uplift\") is extremely important as it provides you with an OOM cheaper estimator for the underlying model capability that you can tweak and run over and over again for every newly released model; and, crucially, also for pre-release testing and risk assessment.
Jasper is blunt about the failure modes (specification gaming, underelicitation, sandbagging, weak construct validity), but the point stands: in biorisk, a false negative costs vastly more than a false positive, so waiting for tier 3 certainty has a price that critics rarely put on the books.
See also:
Hong, S. Z. et al. Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology. Preprint at https://doi.org/10.48550/arXiv.2602.16703 (2026).
Zhang, C. B. C. et al. LLM Novice Uplift on Dual-Use, In Silico Biology Tasks. Preprint at https://doi.org/10.48550/arXiv.2602.23329 (2026).
Paskov, P. et al. RCTs & Human Uplift Studies: Methodological Challenges and Practical Solutions for Frontier AI Evaluation. Preprint at https://doi.org/10.48550/arXiv.2603.11001 (2026).
![\"\"](\"https://substackcdn.com/image/fetch/$s_!dCii!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fec2adbb5-fb04-4db2-96ed-bbdc2cf9b7b1_1152x605.jpeg\" "\\"\\"")
DARPA and the NSF, with CAISI at NIST, announced AI Forge, a program to fund university research on 3 problems: AI interpretability, AI control, and adversarial robustness. The structure is a forum of universities, frontier AI companies, and government, running what DARPA calls Project Ventures, fast one-year university efforts from $750K to $3M or more, several per year. The Critical AI Challenges report alignts in part with this week's other items, heavy on whether you can evaluate, interpret, and trust model behavior. One of its challenges is evaluating AI systems for scientific discovery once they outrun human expertise.
The RFI (DARPA-SN-26-80) asking universities to describe their capabilities is due June 22 at 5 PM ET. Two requirements are worth noting before spending too much time on a response (which must come from your VPR/Provost, not your individual lab). Key personnel are limited to US citizens and permanent residents, and the IP generated is meant to be shared across the forum, preferably under the MIT License.
This RFI is targeting U.S. universities. Further, AI Forge Project Venture designated key personnel (e.g., Principal Investigators [PIs], Co-PIs, and key research personnel) are anticipated to be limited to U.S. citizens or permanent residents.
Please note that Intellectual Property (IP) generated from Project Ventures is intended to be shared among AI Forge forum participants, preferably under liberal open-source licensing (e.g., the MIT License). Universities unable or unwilling to agree to a shared IP framework should not respond to this RFI.
Two preprints landed this week on we evaluate model safety in biology: we count how often a model refuses, and almost never check whether the refusal means anything. Both are preprints, and one is a hackathon project rather than peer-reviewed work, so weight them accordingly.
BioRefusalAudit, by Caleb DeLeeuw, built over a single weekend on a consumer GPU, asks whether a refusal is structurally sound or melts away under small changes. The results across five small models are not reassuring. Gemma 2 2B never refused across 75 prompts; it just hedged. Gemma 4 refused 65 of 75 prompts with chat-template formatting and 0 of 75 without it, and both Gemma models dropped to zero refusals once you capped output at 80 tokens, the kind of cap production systems use for cost and latency. Qwen and Phi-3 swung the other way, flagging 83 to 87% of benign biology as hazardous. The psilocybin probe was interesting: some models refused to discuss cultivating a legal-to-possess, biologically non-toxic substance more often than genuinely hazardous biology, which suggests refusal is tracking legal status and cultural taboo rather than actual CBRN risk.
RefusalBench, from Applied Scientific Intelligence, is the heavier study: 141 prompts in matched triples that hold the task constant and vary only biological risk tier, run across 19 frontier models. On identical prompts, strict refusal rates ran from 0.1% to 94.6%. Jurisdiction didn't predict refusal, but provider did, with Anthropic's API stack predicting refusal at an odds ratio around 21. The authors are careful that this is an access-path effect, not a statement about model weights: 99.8% of Anthropic's refusals carried the same canned policy reason code, consistent with template-based filtering rather than case-by-case reasoning.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!L_Mb!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8256e2b0-f4e6-4c28-bad2-bfbfe2a58e52_1339x602.png\")
The TLDR I took away from this is that refusal rate misranks safety.
Grok 4.20 (lol, seriously?) had the best tier discrimination in the panel while ranking only seventh by refusal rate, and Claude Opus 4.7's discrimination dropped 65% from the prior version, driven entirely by refusing more legitimate research prompts (benign-tier false positives jumped from 33% to 77%) with no gain in catching dual-use ones. 9 of 18 frontier models showed a hedge-but-help pattern at the dual-use tier.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!kKFj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb920abd5-13de-416e-9af7-650082b2da67_1375x922.png\")
Put together, a 4B model on a laptop and the frontier lineup all tell the same story in that refusal rate is close to noise as a safety signal. Thinking of wiring an LLM into a protein design pipeline? A more conservative model isn't a safer one if it terminates your legitimate workflow at step 2, and the RefusalBench authors found the most-refusing models were often the worst at telling benign from dangerous. That a vendor selling agentic biology tooling ran this study is worth noting, but the matched-triple design and the should-refuse control set make it hard to write off.
All the data and prompts are available at github.com/AppliedScientific/refusalbench.
OpenAI launched Rosalind Biodefense, sponsoring access to GPT-Rosalind, its life-sciences reasoning model, for vetted developers building biodefense tools, and extending trusted access to select US government and allied partners. The launch cohort includes Fourth Eon and SecureBio on the screening and detection side, with LLNL, Johns Hopkins APL, and CEPI on the government side. The pitch is \"defensive acceleration,\" putting frontier capability in defenders' hands first.
A vetted-access model is a sensible way to deploy a tool whose offensive and defensive uses are the same capability pointed in opposite directions, which is exactly why OpenAI gates it similar to what Anthropic did with Mythos. It is also a way to expand deployment of a model the company itself treats as High Capability in biology (the threshold it first crossed with ChatGPT agent last July) while keeping the optics clean (take \"meaningfully advantage defenders\" as a goal not a demonstrated result, yet!).
This dropped the same week as the OpenAI Foundation's resilience announcement (more below) and a new cyber-focused executive order, so the bio, cyber, and safety messaging is moving together. And SecureBio appears here as a detection partner, in section 2 as the source of the evals argument, and again in the community roundup below. For a field this consequential, it runs on a small but mighty number of organizations doing important and exciting work!
(Section drafted/organized with Claude Opus 4.8)
Frontiers' open special issue, Artificial Intelligence and the Future of Biosecurity (edited by Bin Hu of Los Alamos, Dov Greenbaum of Yale, and Michelle Holko at Berkeley), collects perspectives and frameworks on the dual-use problem of AI-enabled biology. Worth a scan rather than a read-through:
The June GCBR organization update rounds up Active Site, the Asia Centre for Health Security, IBBIS, and SecureBio.
at always has a great weekly recap, so much so that I model the recap here off of his weekly \"Five Things\" series. His Five Things: May 31, 2026 post covers Illinois SB 315, ESMFold2, papal encyclical, Opus 4.8, OpenAI's Rosalind Biodefense.
A new executive order on AI innovation and security sets up a \"covered frontier model\" designation, a Treasury-run vulnerability clearinghouse, and a voluntary window of up to 30 days of early government access to qualifying models before wider release.
The OpenAI Foundation detailed an AI Resilience program across four areas (bio, cyber, model safety, and effects on young people), saying it is finalizing more than $130M in grants and intends to commit over $1B in the next year.
openRxiv (Richard Sever and Tracy Teal) sketched where bioRxiv and medRxiv are headed on trust signals as AI makes fake-but-plausible research cheaper: verified data deposits, funder metadata (now on roughly 80% of papers), and the hard problem of identity verification, since ORCID disambiguates authors but doesn't confirm they are who they claim.
NeurIPS desk-rejected 178 position-paper submissions (18.4%) for being substantially AI-written, using the Pangram detector, with another 123 asked to prove human authorship.
Nathalie Corn\u00e9e & Kat Hart
\n\n\n\nCitations aren’t the only way research shows up in policy. Sometimes it’s a named researcher, a nod to “the evidence,” or a reference tucked into the text with no formal citation at all. Kat Hart and Nathalie Corn\u00e9e dug into Overton’s data to find out how often that happens, and what it means for how we understand research’s reach into policy.
\n\n\n\nWhen we want to know if research has influenced policy, we usually turn to citations. They’re easy to spot and easy to count, so it makes sense that they’ve become the go-to signal for research influence.
\n\n\n\nBut citations are only part of the story. Policymakers don’t always follow academic conventions. Sometimes they name a researcher directly, reference a report rather than the paper behind it, or they simply gesture at “the evidence” without pointing to anything specific at all.
\n\n\n\nSo, we set out to ask a simple question: if we look past formal citations, what else might be going on? Using Overton’s data, we compared the scale of formal citations against these more informal mentions and tried to get a sense of what the gap between them might look like. This is an exploratory pilot rather than a definitive study, as the sample is far too small to draw firm conclusions. We hope its value lies more in how it reframes the question of what counts as 'research engaging with policy' (and vice versa) in the first place.
\n\n\n\nDefinitions of research impact are shifting. A recent Australian study published on LSE's impact blog made a useful point on this: institutional definitions of impact often don’t match how researchers actually work (O'Connor et al., 2026). Instead of asking “did this have impact?”, the authors suggest asking “what value does this create, and for whom?” It’s a small shift in framing, but it moves the conversation from proving impact to understanding it.
\n\n\n\n\n\n\n\n\nIf citation counts alone don’t capture the full picture, what other signals should we be paying attention to?
\n
We used the Overton Index as our starting point. At the time of the study, Overton indexed over 27 million policy documents from more than 190 countries and 3,100 sources, spanning everything from intergovernmental organisations like the UN and WHO to NGOs, think tanks, national governments, as well as regional and local bodies like Canadian provinces and UK devolved nations.
\n\n\n\nFor the pilot itself, we kept things deliberately focused. We took a random sample of 50 UK government policy documents from 2025 and used Overton’s AI classification to pull out document type, length, and authorship. Then came the manual part: we read and coded every single document by hand.
\n\n\n\nThis was a conscious choice. Automated tools are good at catching formal citations, but they miss the less obvious ways research shows up in policy text. We wanted to capture all of it, so we built a coding framework covering two broad categories:
\n\n\n\nFigure 1. Workflow for identifying and classifying attention to actors and objects in UK government policy documents.
\n\n\n\nFor each one, we recorded whether it was a citation (a formal, traceable reference) or a mention (an informal acknowledgement with no formal citation attached). Where a document referenced several things together, we logged each one individually rather than lumping them into a single entry. Across just 50 documents, that added up to over 2,200 individual references, which gave us plenty to work with despite the small sample size.
\n\n\n\nAcross our sample, we identified 2,244 objects and actors in total. Policy actors (think tanks, government bodies, IGOs) and policy evidence (past reports, transcripts, consultation outputs) were by far the most common categories in our data, together with in-house research accounting for around 75% of everything referenced.
\n\n\n\nPeer-reviewed publications, by comparison, made up just under 5% of all references. Generic, non-specific nods to “experts” or “previous research” outnumbered them, at around 9%. In other words, plenty of documents acknowledge that evidence exists somewhere, without pointing to anything specific.
\n\n\n\nPeer-reviewed articles were the type of evidence most likely to receive a full, formal citation. Commissioned research told a different story: it often got absorbed into the narrative (“the data suggests”, “interviews found”) with no formal attribution at all. Policy outputs sat somewhere in between, appearing as both citations and mentions in meaningful numbers.
\n\n\n\nAcross the whole sample, mentions were consistently more common than citations.
\n\n\n\nReferencing patterns varied a lot depending on who authored the document. Our five most prolific authors in the sample were the Department for Education, the Law Commission, UK Government more broadly, the Ministry of Justice, and the Department for Environment, Food and Rural Affairs (DEFRA) together with the Animal and Plant Health Agency (APHA). Each had a distinct pattern of what they referenced and how often.
\n\n\n\nA working paper from the Foreign and Commonwealth Office, written by commissioned academics, leaned heavily on peer-reviewed citations, as you’d expect. A similar paper from Technopolis, a consultancy commissioned by the Department for Science, Innovation and Technology (DSIT), leaned instead on funder acknowledgements and commissioned research. Same document type, same broad government context, very different citation behaviour.
\n\n\n\nWe also tested the intuitive assumption that longer documents would contain more citations. They don’t, at least not cleanly. Some of the highest citation counts did appear in longer documents, but plenty of short documents cited heavily too, and plenty of long ones cited very little. There’s no straightforward relationship between page count and citation volume in this sample.
\n\n\n\nMost documents in our sample didn’t have a formal bibliography at all, relying on footnotes instead. And even where a reference qualified as a formal citation, it only had roughly a 50/50 chance of actually appearing in a bibliography rather than just the body text. Citation habits clearly vary a lot from one policymaker to another.
\n\n\n\nThe research being cited mostly comes from a small number of places
\n\n\n\nOf the peer-reviewed research we identified, half came from UK universities alone, 43% came from international institutions, and the rest reflected UK and international collaborations.
\n\n\n\nA few things stood out consistently across the sample:
\n\n\n\nTaken together, these early findings suggest that citation counts alone may only capture a part of how research reaches policy.
\n\n\n\nWhere we think this work is genuinely useful is in how it reframes the question. Right now, most tools only ask whether a policy document cited a piece of research or not. Splitting that into citations and mentions lets us ask a more useful question: how is research actually showing up, even when there’s no formal reference at all? That feels like a real gap worth exploring.
\n\n\n\nThe coding framework itself, sorting references into different types of evidence and different types of actors, and tagging each one as a citation or a mention, is a first step towards making that visible. It gives us a way to describe the kinds of engagement that citation counts alone may miss, so it’s a starting point for more work.
\n\n\n\nThis also fits with a broader pattern often noted in research impact literature: that scholarly expertise tends to reach policy indirectly, through reviews, consultations, and expert commentary, rather than through direct citation. That raises a genuinely open question: how much of what we’re seeing is a passing reference, and how much reflects real engagement with the research itself?
\n\n\n\n\n\n\n\n\n\n\nBeing able to tell the difference matters if we want to understand not just whether research is being used, but how and why.
\n
This was a small, exploratory pilot, and we’re upfront about its limits. Fifty documents from one country over one year is a start, not a conclusion.
\n\n\n\nNext, we’d like to scale up the sample across a wider, more representative set of countries using an LLM to help with the coding. We’d also like to track whether the balance between citations and mentions is shifting over time and compare document types more systematically to understand whether citation behaviour is driven more by document function or by the authoring body. Beyond that, we want to develop ways to assess the depth of engagement behind a reference, not just whether it exists, and look at whether these patterns differ across policy domains, comparing something like health policy against education policy.
\n\n\n\nIf you have thoughts on this work, or you’re doing something similar, we’d love to hear from you. Get in touch at nathalie@overton.io or kat@overton.io.
\n\n\n\nNathalie Corn\u00e9e helps the research and policy community around the world get the most out of Overton. She joined in 2023, bringing more than 20 years of experience across the pharma, research, and academic library sectors, always focused on highlighting the significance of research and supporting the researchers behind it, including through bibliometric analysis. She takes a practical, hands-on approach to helping users find the data they need and make sense of research impact.
\nKat Hart is a Senior Analyst at Overton, where she curates policy data and uses bibliometrics to explore evidence-use patterns worldwide. Alongside this, she’s a PhD researcher at the University of Nottingham, studying UK water governance. She previously worked as a knowledge broker in UK higher education, focused on ‘what works’ in connecting academic research with public policy.
\n Unless it states otherwise, the content of the Bibliomagician is licensed under a Creative Commons Attribution 4.0 International License.
DOI: https://doi.org/10.59350/74r3r-67451
\n\n\n\n","doi":"https://doi.org/10.59350/a3dpx-knp02","guid":"https://thebibliomagician.wordpress.com/?p=5646","image":"https://thebibliomagician.wordpress.com/wp-content/uploads/2026/09/image.png?w=770","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1788739200,"reference":[{"id":"https://blogs.lse.ac.uk/impactofsocialsciences/2026/03/23/we-should-focus-less-on-research-impact-and-more-on-research-value/","unstructured":"O'Connor, R., Malde, S., Russell, W., & Haviland, M. (2026, March 23). We should focus less on research impact and more on research value. LSE Impact Blog."}],"rid":"xx8x7-m3t11","summary":"Nathalie Corn\u00e9e & Kat Hart Citations aren't the only way research shows up in policy. Sometimes it's a named researcher, a nod to \"the evidence,\" or a reference tucked into the text with no formal citation at all.","tags":["Altmetrics","Policy"],"title":"Cited or Mentioned? What We Learned About How Research Actually Shows Up in Policy","updated_at":1792434907,"url":"https://thebibliomagician.wordpress.com/2026/09/07/cited-or-mentioned-what-we-learned-about-how-research-actually-shows-up-in-policy/","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/0524sp257","name":"University of Bristol"}],"contributor_roles":[],"family":"Taylor","given":"Mike","url":"https://orcid.org/0000-0002-1003-5675"}],"blog":{"authors":[{"name":"Mike Taylor","url":"https://orcid.org/0000-0002-1003-5675"},{"name":"Matt Wedel","url":"https://orcid.org/0000-0001-6082-3103"}],"community_id":"0e13541f-417e-46c0-a859-65927249df72","created":1675209600,"current_feed_url":null,"description":"SV-POW! ... All sauropod vertebrae, except when we're talking about Open Access. ISSN 3033-3695","doi":"https://doi.org/10.59350/svpow","favicon":"https://rogue-scholar.org/api/communities/0e13541f-417e-46c0-a859-65927249df72/logo","feed_format":"application/atom+xml","feed_url":"https://svpow.com/feed/atom/","filter":null,"generator":"WordPress.com","home_page_url":"https://svpow.com","issn":"3033-3695","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"svpow","status":"active","subfield":"1911","title":"Sauropod Vertebra Picture of the Week","updated":1792433290,"use_api":true},"blog_name":"Sauropod Vertebra Picture of the Week","blog_slug":"svpow","content_html":"Yesterday, an AI posted a comment on this blog \u2014 or tried to. It was superficially helpful, relevant to the top of the post it was on, and turned out to contain that characteristic blend of things we already knew and things that are (subtly or blatantly) wrong. Needless to say, we shoved it in the spam folder.
\nThis is a good moment to remind everyone of our AI policy: SV-POW! is an AI-free zone. (Follow the link for the detail.)
\n
That applies to comments as well as to our own posts.
\nDo not post AI-generated content as a comment(*). If you do, we may warn you once, or we may just block you. As Matt put it in an email:
\n\nMake it clear that this is absolutely zero-tolerance. A single instance of posting AI slop can get you perma-banned. People can go publicly crap their pants on the rest of the internet.
We’re interested in what our readers have to say, and in engaging with humans. Slop is not welcome here.
\n\n
(*) The sole possible exception, and be very careful about this, is posting clearly labelled AI output for the purpose of going on to critique it. But given how grumpy we’re both feeling about aslopalypse, it’s probably best not to risk it.
\n","doi":"https://doi.org/10.59350/98nbq-v9p28","guid":"https://svpow.com/?p=26095","image":"https://svpow.wordpress.com/wp-content/uploads/2025/12/no-to-ai.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1792368000,"rid":"299wf-mvw22","summary":"Yesterday, an AI posted a comment on this blog \u2014 or tried to. It was superficially helpful, relevant to the top of the post it was on, and turned out to contain that characteristic blend of things we already knew and things that are (subtly or blatantly) wrong.","tags":["Artificial Intelligence","LLM","Navel Blogging"],"title":"No AI comments. None. No exceptions.","updated_at":1792433711,"url":"https://svpow.com/2026/10/19/no-ai-comments-none-no-exceptions/","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"
AI-heavy week, but the throughline is what happens when the platforms and infrastructure under our work start to rot, from GitHub to preprint servers to peer-reviewed literature.
The GitHub ship is sinking, the lifeboats leak
Vibe-coded into a corner
The Lancet sues the AI that cites The Lancet
Hallucinated citations, now with a denominator
arXiv takes the banhammer out for AI-written work
David Bushell wrote a vivid eulogy for GitHub (\"GitHub used to be cool and now it's a lame slop graveyard\"), and a few days later GitLab's new CEO Bill Staples published \"GitLab Act 2\", which to me sounds like a flailing pivot toward agentic everything.
Bushell's case against GitHub is part vibes but contains an uptime chart that does look bad after the Microsoft acquisition.
When I posted this chart on Bluesky earlier this week, I had a few folks pointing to the deluge of AI coding agents and automated pull requests as the reason. But if you look closely at the chart, this service degradation long predated AI coding agents and ChatGPT itself.
But: Git is not GitHub. He recommends Codeberg (running Forgejo) as the safe alternative, with self-hosted Forgejo as the power-user option. He also recommended GitLab, then added an edit:
oh dear, they've contracted the rot
The Staples letter is what he means. GitLab is reorganizing into roughly 60 smaller R&D teams, flattening management, exiting up to 30% of its country footprint, and (oh my): \"rewiring internal processes with AI agents, automating the reviews, approvals, and handoffs to speed us up.\" The strategic thesis is that \"software will be built by machines, directed by people.\"
The blog post calls git \"designed for human-rate commits\" and pitches a \"generational rebuild of the underlying infrastructure to handle agent-rate work as the default.\"
I'm not an agent power user, and maybe I'll retract my thoughts on this in a few months. But, if your agents are opening so many merge requests that git itself is the bottleneck, the answer is probably fewer agents.
It'll be interesting to see if this translates into actual migrations. Pretty much all of computational biology / bioinformatics lives on GitHub: software packages, snakemake/nf-core workflows, Bioconductor packages, lab repos, course materials. Don't forget about other infrastructure as well: GitHub Pages (e.g. pkgdown pages for R packages), CI w/ GitHub Actions, GitHub container registry, etc. Moving the social graph (issues, PRs, stars, discoverability) is enormous work, and I don't know if Codeberg could absorb the load. I can't imagine what a realistic and practical exit plan would look like.
I'm not usually one to give air to the anti-AI hype. Most of what I read here is boring and irritating. However, Shubham's \"I'm going back to writing code by hand\" is good. He spent ~30 weekends and >200 commits building k10s, a GPU-aware Kubernetes TUI, entirely through Claude. Then he sat down and read model.go for the first time. It was 1690 lines. He's archiving it and starting over in Rust.
AI builds features, not architecture; every prompt landed cleanly, the cumulative effect was a god object. Vibe-coding \"made everything feel cheap\" so scope crept from a niche GPU tool to a generic k9s clone. My favorite line:
like \"em-dash\" is to ai writing, \"god-object\" is to ai coding
His proposed remedy is putting architectural invariants into CLAUDE.md or AGENTS.md so the model sees them on every invocation.
See also the HN thread. One comment stuck out: \"Can't you just ask AI to break up large files into smaller ones and also explain how the code works so you can understand it?\" If the AI got you into the god-object hole, asking the AI to refactor its way out is probably not the move.
I was at a week-long innovation lab around AI and drug discovery this weekend. A thought in the back of my head all week was something I keep coming back to in my thinking in computational biology: the bottleneck in agentic science is evaluation, not generation. You can prompt your way to a working pipeline in a weekend. You can't prompt your way to knowing it's right.
Elsevier has joined a class-action lawsuit against Meta alleging Meta reproduced copyrighted works to train Llama. As Nature notes, this is the first AI copyright suit from major publishing houses. Elsevier publishes Cell and The Lancet, so a fair chunk of the biomedical literature is now formally a plaintiff against a frontier lab.
The lawsuit alleges Meta used Common Crawl and also that Meta downloaded and torrented works from LibGen and Sci-Hub. Meta's defense is the usual one: training is transformative use. US judges in two 2025 rulings reportedly distinguished between training (often transformative) and acquisition (the act of downloading copyrighted material), which is where torrenting from Sci-Hub is going to be a problem for Meta regardless of how fair use resolves.
Irony is dead.
Elsevier sells access to research funded by taxpayers and donated by authors who weren't paid for the manuscript and often paid Elsevier APCs to make it open. Meta scraped some of that content, allegedly via Sci-Hub, which exists precisely because Elsevier's access model is widely considered indefensible. Now Elsevier wants damages on behalf of authors who didn't get paid the first time around either. The most coherent position is probably that Meta should have paid for licensed access and that Elsevier should not be the entity collecting the check, but that isn't on offer.
If this case establishes that scraping paywalled academic content is infringement, the practical effect on open-science-trained models could be larger than the effect on Llama. Llama already exists. A future model that wants to train on biomedical literature now has a clearer legal target on its back.
A new Lancet correspondence by Topaz et al. audited 2.5 million biomedical papers for fabricated references, and the numbers are bad. The accompanying Comment by Bauchner and Rivara argues that any published paper with a fabricated reference should be retracted, which is a reasonable position that approximately no one will implement.
Topaz and colleagues pulled >100 million references from >2 million papers in PubMed Central's Open Access subset between Jan 2023 and Feb 2026. They kept the 97 million (77%) with a PMID, verified those against PubMed, Crossref, OpenAlex, and Google Scholar, used Claude to filter reference errors (misformatted but real) from genuine fabrications, and validated precision at 91% with three independent reviewers. They found 4,046 fabricated references across 2,810 papers.
The fabrication rate increased more than 12 times, from approximately four per 10,000 papers in 2023, to 51.3 per 10,000 papers in the fourth quarter of 2025
The inflection point is mid-2024, which is when LLM-assisted papers would start clearing typical 100-200 day submission lags. One paper on ureteroileal anastomotic techniques had 18 of 30 verified references fabricated, each tailored to the surgical topic and attributed to real urologists. The authors also flag a paper-mill pattern: the same two-author pair appearing across 11 papers in a single surgical journal in 2025.
Read this along with #3 above. If Elsevier wins the case against Meta, the legal incentives push toward licensed-only training data. The Lancet audit suggests the academic literature itself is already meaningfully polluted by models trained on it. I think this will get worse before it gets better.
Yesterday, Thomas G. Dietterich (arXiv moderator for cs.LG) announced that arXiv will implement 1-year ban for papers containing incontrovertible evidence of unchecked LLM-generated errors, such as hallucinated references or results. Here's the full text of the thread:
Attention arXiv authors: Our Code of Conduct states that by signing your name as an author of a paper, each author takes full responsibility for all its contents, irrespective of how the contents were generated.
If generative AI tools generate inappropriate language, plagiarized content, biased content, errors, mistakes, incorrect references, or misleading content, and that output is included in scientific works, it is the responsibility of the author(s).
We have recently clarified our penalties for this. If a submission contains incontrovertible evidence that the authors did not check the results of LLM generation, this means we can't trust anything in the paper.
The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue.
Examples of incontrovertible evidence: hallucinated references, meta-comments from the LLM (\"here is a 200 word summary; would you like me to make any changes?\"; \"the data in this table is illustrative, fill it in with the real numbers from your experiments\").
I'm about as split on this one as the comments on the OP are. On one hand, GOOD. Hallucinated citations and unchecked AI-authored content is polluting the literature everywhere. I think this borderlines on scientific misconduct, and it destroys trust in the entire scientific enterprise.
Oh the other hand there are lots of problems with detection and enforcement. AI detection software is a cat and mouse game that doesn't work reliably. And on the enforcement side \u2014 what happens when you're a co-author on a paper with a hallucinated citation? Say I tell my grad student to work on the methods section and they get an undergrad to write a small section that they worked on, and said undergrad carelessly inserts a fabricated citation. If this paper makes its way onto arXiv, does that infraction go all the way up the chain with the banhammer coming down on all co-authors? Have you ever submitted a paper written with a bunch of co-authors and meticulously looked up every reference or checked that every parameter setting in the detailed methods section is actually a valid parameter for whatever tool was being used? I have.
I think a middle ground solution with a big red banner or warning flag on the abstract page or search results noting that the paper likely contains unverified AI-generated content.
Again, the detection piece is difficult, and authors should be given a chance to respond before getting the scarlet letter. I think the 1-year ban is severe, and if applied to all co-authors, could be extremely damaging to one's career just because a middle author responsible for page 98 of a large supplemental info section was careless and wasn't carefully scrutinized. I also worry that such a ban could be inconsistently applied since there's a lot of subjectivity and guesswork involved with this endeavor.
","doi":"https://doi.org/10.59350/fe03p-s2z80","guid":"197328359","image":"https://substackcdn.com/image/fetch/$s_!71Lh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4b57310-795f-4211-80bb-6fe9ab994d50_1448x1086.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1778803200,"rid":"ne4rm-36y89","summary":"GitHub problems, vibe coding regrets, Elsevier sues Meta, fabricated citations, the arXiv banhammer for AI-generated content","tags":["Papers","AI"],"title":"Five Things (May 15, 2026): AI is why we can't have nice things","updated_at":1792423601,"url":"https://blog.stephenturner.us/p/five-things-may-15-2026","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"Ever find yourself looking through a pkgdown page or a Quarto book, copying and pasting code chunks from your browser into your IDE? I do, and it's a minor annoyance.1
My friend and colleague VP Nagraj published a new R package called ctrlvee that makes this a lot easier.
It does one thing. Put your cursor anywhere in an R script in Positron or RStudio, call the add-in, provide a URL, and a few milliseconds later you'll have all the code from that page in your editor, separated by chunk boundaries (along with some metadata and a note to check the license!).
The package README provides a demonstration using the \"Data Validation and QA\" chapter of my Data Science Team Training book (dstt.stephenturner.us).
Install the package: install.packages(\"ctrlvee\")
Run the add-in. In Positron you'll open the command palette, search for Run RStudio Addin, then extract external R code and insert inline. You'll get a modal asking you for a URL.
Paste one in. E.g., https://dstt.stephenturner.us/validation.html
The R code from the website appears in your editor \ud83d\ude80
Here's a demo.
Here's what the extracted/inserted code looks like, from this source.
![](\"https://substackcdn.com/image/fetch/$s_!ytmH!,w_56,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffabebe05-d8d0-4141-821f-4fb29b38a346_3871x3871.jpeg\")
![](\"https://substackcdn.com/image/fetch/$s_!ciq4!,w_56,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe4252d92-80bd-4a83-81aa-9ec866a51fe7_608x608.png\")
![](\"https://substackcdn.com/image/fetch/$s_!XFpD!,w_56,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff2b7cb38-a2a5-40c5-a984-92d6f2a0e3a1_1024x1024.png\")
![](\"https://substackcdn.com/image/fetch/$s_!jhE9!,w_56,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6d31dbac-a74f-4c4a-9683-348b1f4dbee5_500x500.png\")
# -----------------------------------------------------------------\n# Chunks fetched by ctrlvee from: https://dstt.stephenturner.us/validation.html\n# Strategy: Rendered HTML page\n# Date: 2026-05-16 05:14:44\n# Chunks: 8\n# NOTE: Check the source license before reusing this code.\n# -----------------------------------------------------------------\n\nflu <- data.frame(\n week = c(1, 2, 3, 4, 4),\n county = c(\"Fairfax\", \"Arlington\", NA, \"Loudoun\", \"Loudoun\"),\n disease = c(\"Flu\", \"Flu\", \"Flu\", \"Flu\", \"Flu\"),\n cases = c(23, 41, 18, -5, 12),\n rate = c(2.1, 3.8, 1.6, NA, 1.1)\n)\n\nflu\n\n# ---- chunk boundary ----\n\nif (any(flu$cases < 0, na.rm = TRUE)) {\n stop(\"Negative case counts detected. Inspect raw data before proceeding.\")\n}\n\n# ---- chunk boundary ----\n\nstopifnot(\n \"Negative case counts\" = all(flu$cases >= 0, na.rm = TRUE),\n \"Missing county values\" = !anyNA(flu$county),\n \"Duplicate records\" = !anyDuplicated(flu[, c(\"week\", \"county\")])\n)\n\n# ---- chunk boundary ----\n\ninstall.packages(\"pointblank\")\n\n# ---- chunk boundary ----\n\nlibrary(pointblank)\n\nagent <- create_agent(tbl = flu, label = \"Weekly flu surveillance\") |>\n col_vals_gte(\n columns = cases,\n value = 0,\n label = \"Case counts must be non-negative\"\n ) |>\n col_vals_not_null(\n columns = c(week, county),\n label = \"Week and county cannot be missing\"\n ) |>\n rows_distinct(\n columns = c(week, county),\n label = \"No duplicate week/county records\"\n ) |>\n interrogate()\n\nagent\n\n# ---- chunk boundary ----\n\ncreate_agent(tbl = flu, label = \"Weekly flu surveillance \u2014 extended\") |>\n col_is_numeric(\n columns = c(cases, rate),\n label = \"Case count and rate must be numeric\"\n ) |>\n col_vals_in_set(\n columns = disease,\n set = c(\"Flu\", \"COVID-19\", \"RSV\"),\n label = \"Disease must be from the approved list\"\n ) |>\n col_vals_between(\n columns = week,\n left = 1,\n right = 52,\n label = \"Week must be between 1 and 52\"\n ) |>\n col_vals_gte(\n columns = rate,\n value = 0,\n na_pass = TRUE,\n label = \"Rate must be non-negative (NAs allowed)\"\n ) |>\n interrogate()\n\n# ---- chunk boundary ----\n\nif (!all_passed(agent)) {\n stop(\"Data validation failed. Review the agent report before proceeding.\")\n}\n\n# ---- chunk boundary ----\n\nlibrary(readr)\nlibrary(pointblank)\n\nflu <- read_csv(\"data/flu-2024.csv\")\n\n# Validate immediately after reading\nagent <- create_agent(tbl = flu, label = \"flu-2024 validation\") |>\n col_vals_gte(columns = cases, value = 0, label = \"No negative counts\") |>\n col_vals_not_null(columns = c(week, county), label = \"No missing keys\") |>\n rows_distinct(columns = c(week, county), label = \"No duplicate records\") |>\n interrogate()\n\nif (!all_passed(agent)) {\n stop(\"Validation failed \u2014 see agent report above.\")\n}Sure, if the pkgdown page or Quarto book or whatever has source code on GitHub, you could find the source and open that up directly. However, you won't find the source for everything, and the source will be crowded by markdown narrative you might not want if you're just looking for the code.
Big week in AI in life sciences (AIxBio). The Nature drop this week included three papers on AI scientists alongside an editorial and a comment piece pushing back on the whole project. Add 's three-part series on AI for biology, a new RAND/Helena workshop report on AIxBio mitigations, 's take on AI in genomics, and a new study on AI in peer review and I'm landing on a theme this week: how fast should we let AI into the production of biological knowledge, and what gets lost if we don't slow down to ask?
's case for shaping AI-for-biology before it shapes us
RAND and Helena on AIxBio mitigations
on the state of AI in genomics
Nature's AI scientists week, and the editorial pushback
45 expert scientists review the reviewers
(Johns Hopkins, Center for Health Security) published a three-part series last week on shaping AI progress for biology and biosecurity.
Part 1 sets up the series. AI will compress decades of biological research into years, but cures won't arrive by default, and the same systems that enable them can lower the barrier to weaponizing pathogens. We need proactive policy on both sides.
Part 2 is where it gets interesting. Pannu lays out what she calls autonomous biological discovery: AI systems that automate every step of the research cycle, including managing the cycle itself, with orgs like Isomorphic Labs, FutureHouse, Ginkgo Bioworks and others entering the fray.
AI-enabled feedback loops will be able to extend beyond this, exploring parts of biological space that nature has not.
Evolution selects for reproductive success and gets stuck in fitness valleys. AI-driven design doesn't have that constraint. Whether that's a feature or a terrifying bug depends on what you're designing.
Part 3 centers on smallpox eradication: 171 years between Jenner's cowpox demonstration in 1796 and Henderson's 1967 campaign, and only 10 of those years were spent actually eradicating. Her conclusion is that the bottleneck wasn't tech, it was coordination and political will, so even if AI drives the marginal cost of biology research to zero, we shouldn't expect cures to deploy themselves.
RAND and Helena released the conference proceedings (full PDF here) from a January 2026 workshop on AI-enabled biological threats. 22 participants from frontier labs, biotech, biosecurity, and academia, working over two days in DC under Chatham House rules, with three threat scenarios: a millenarian nonstate group releasing a novel influenza A, an agroterrorism scenario targeting US wheat with an engineered fungal pathogen, and a state-sponsored insider attack on a semiconductor plant using a biofilm-forming bacterium. Scenarios were deliberately compressed and the document withholds specifics for infohazard reasons.
The pandemic group prioritized pathogen-agnostic physical defenses (high-quality PPE, indoor air quality with filtration and UV), a voluntary credentialing system called \"BioTrust\" modeled on ORCID, and AI \"guardian models\" for intent monitoring. The agroterrorism group went after holistic biosurveillance, information-sharing modeled on the Kansas Intelligence Fusion Center, and synthetic DNA screening for agricultural pathogens (which gets less attention than human-pathogen screening, and the participants thought that was the most well-scoped problem of the bunch). The critical infrastructure group went hardest on LLM-side interventions: investing in safeguards to better infer intent from prompt patterns, information-sharing between LLM companies via something like the Frontier Model Forum, federated cross-platform behavior analysis, and (this one is interesting) training LLMs to de-escalate malicious intent by adapting techniques from suicide prevention hotlines.
The participants (which included Twist Bioscience, Anthropic, Microsoft, SecureBio, Los Alamos, and others) kept running into the same wall, which the report names explicitly:
A central theme was that technical feasibility and political backing together determine a mitigation's success.
I.e., most of these will fail without sustained funding and political will, and almost none of them have either. The other recurring caveat was attribution. AI-enabled biological incidents may remain unattributed indefinitely, which weakens deterrence and complicates response authority.
ran the University of Chicago's annual genetics, genomics, and systems biology symposium last week, and turned the speaker lineup into seven points on the current state of AI in genomics.
The whole post is worth reading (<10 minutes). I'm just going to highlight a few.
First, the scaling laws may not hold for DNA. Evo 2 is 40 billion parameters trained on 9 trillion nucleotides spanning every domain of life. But Vishniakov et al. (2025) compared seven genomic foundation models against randomly initialized baselines of matched architecture across 52 tasks. The random baselines often matched or beat the pretrained models. Tang et al. (2025) found that raw one-hot encoded sequences are competitive with learned DNA-LM representations on regulatory genomics tasks. As Alex Lu put it at the symposium, DNA isn't natural language: low signal-to-noise, vast repetitive tracts, no obvious word or sentence analogs, and sparse functional elements that interact combinatorially across long distances.
Second, Arjun Krishnan's rule of thumb on benchmarks:
The best model is usually the one that is consistently number 2 in benchmarks across the literature.
Whoever publishes a model also designs the benchmark, and the benchmark almost always flatters the model. A model that's consistently competitive but rarely first-place is more likely to be genuinely strong than one that wins on the benchmark its own authors built. I'm stealing this.
Third, toward the end, is a succinct rule for trainees using AI tools:
If you can validate what the AI produces, namely, if you can do the task yourself comfortably and check whether the AI did it correctly, then you can probably use AI to do the task. Otherwise, you should probably do it yourself, even if it feels hard.
This is the cleanest articulation of the trainee-and-AI problem I've seen. I've written about this before, highlighting work from a new colleague and co-author, Arjun Krishnan:
The friction of doing it the slow way is often the friction of actually learning, the \"productive struggle\" I've written about here before. An AI tool that produces output you can't evaluate is just a black box you're forced to trust. I'd extend that beyond trainees, frankly.
On Tuesday, Nature published three full-length papers on AI scientists, an editorial that hedges, and a comment piece that pushes back. All on the same day.
Ghareeb, A. E. et al. A multi-agent system for automating scientific discovery. Nature 1\u20133 (2026) doi:10.1038/s41586-026-10652-y.
Ayg\u00fcn, E. et al. An AI system to help scientists write expert-level empirical software. Nature 1\u20133 (2026) doi:10.1038/s41586-026-10658-6.
Gottweis, J. et al. Accelerating scientific discovery with Co-Scientist. Nature 1\u20133 (2026) doi:10.1038/s41586-026-10644-y.
Messeri, L. & Crockett, M. J. The uncritical adoption of AI in science is alarming \u2014 we urgently need guard rails. Nature 653, 675\u2013676 (2026).
Why AI cannot do good science without humans. Nature 653, 650\u2013650 (2026).
The three papers: Google DeepMind's ERA, an LLM-plus-tree-search system that discovered 40 novel single-cell analysis methods that outperformed the top human methods on a public leaderboard, and 14 COVID hospitalization forecasting models that beat the CDC ensemble. Google's Co-Scientist, a multi-agent system built on Gemini that helped identify in vitro\u2013validated drug repurposing candidates for acute myeloid leukemia and (in a now-famous demo) recovered an antibiotic-resistance hypothesis that a Imperial College team had spent a decade developing but hadn't yet published, in days. And FutureHouse's Robin, which autonomously proposed enhancing RPE phagocytosis as a strategy for dry AMD, identified ripasudil (a clinically used ROCK inhibitor never previously proposed for AMD) as a candidate, validated it in vitro, then proposed an RNA-seq follow-up that fingered ABCA1 as a possible novel target. All hypotheses, experimental directions, data analyses, and main-text figures in the Robin paper were produced by Robin.
These are real results. With that throat-clearing out of the way\u2014
Then there's the editorial, \"Why AI cannot do good science without humans\", which is mostly anodyne until the closing paragraph:
Scientists should not allow a negative view of AI to drive them away from exploring the possibilities that AI co-scientists might hold for research. Equally, however, they must rise above the din of AI hype and advocate for their own importance, to remind the wider public, funders and fellow researchers that science still needs humanity, and that not every grant proposal need include an AI project.
Again: not every grant proposal need include an AI project.
Once more: not every grant proposal need include an AI project.
I read this as the editorial board, deliberately, on the day they published three papers about AI scientists, telling reviewers and program officers not to use \"no AI angle\" as a reason to triage a proposal.
The comment piece is more pointed. Lisa Messeri (Yale anthropology) and M. J. Crockett (Princeton psychology) wrote \"Uncritical use of AI in science needs reality check\". Some empirical claims: Hao et al. (2026) analyzed 41.3 million papers across biology, medicine, chemistry, physics, materials science, and geology and concluded that AI adoption seems to \"induce authors to converge on the same solutions to known problems rather than create new ones.\" I wrote about this earlier this year:
Kusumegi et al. (2025) looked at 264,125 papers and found that in LLM-assisted papers, good writing stopped being a useful heuristic for scientific quality. Organization Science audited 6,957 submissions from 2021 to 2026 and found LLM-assisted papers had poorer scientific quality by acceptance rate. The closing argument is about deskilling: cleaning raw data, reading and summarizing the literature, the entry-level grunt work that AI is now offered as a solution for, is also how scientists develop the tacit knowledge needed to supervise AI-assisted workflows. If trainees don't develop those skills, who oversees the AI?
Three papers showcasing autonomous discovery, an editorial gently telling reviewers not to fetishize AI angles in proposals, and a comment piece arguing that the productivity gains may be hollowing out the next generation. Read them together. Or, if you don't have time, listen to Nature's podcast.
One more, and this connects to a paper I co-authored. A preprint went up at arXiv:2605.20668 titled \"On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists.\" Big study. 45 domain scientists annotated reviews of Nature-family papers, comparing official human reviewers against three frontier LLM agents. Headline results:
On aggregate review-item quality, all three AI reviewers exceed the lowest-rated human, and GPT-5.2 exceeds the top-rated human.
AI reviewers raise more significant items but with lower correctness.
Replacing one human reviewer with one AI reviewer minimally erodes panel diversity, because human reviewers themselves surface largely disjoint sets of criticisms.
AI reviewers can augment but not replace a human panel.
Current frontier AI reviewers in an agentic framework provide genuine value on the rigor- and code-heavy aspects of peer review, while systematically failing on the field-context aspects.
On that last point: it's the same argument that Agnieszka Swiatecka-Urban, Arjun Krishnan, and I argued for in our preprint earlier this year.
Our claim was that AI is best deployed as a rubric enforcer for the systematic, criterion-checkable parts of review (consistency between scores and comments, statistical reporting, completeness of evaluation, internal consistency of reviewer reasoning) while humans retain authority on the parts that depend on argumentative-world knowledge (novelty, feasibility, recognizing creative leaps, judging whether an ambitious proposal might fail spectacularly or succeed brilliantly). The arXiv paper, working with a completely different methodology and 45 domain scientists doing item-level annotation of real Nature-family reviews, lands in the same place. AI is strong on rigor and code, weak on field context.
","doi":"https://doi.org/10.59350/8103y-x2w56","guid":"198822549","image":"https://substackcdn.com/image/fetch/$s_!OMAo!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83677cd0-18cf-4891-87d4-066d4be0443b_3594x1887.jpeg","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1779408000,"rid":"hkxas-9dy42","summary":"Jassi Pannu on AI and biosecurity, RAND/Helena AIxBio biosecurity mitigations, Blekhman on genomics AI, Nature's AI scientists week, AI in peer review.","tags":["Papers","Biosecurity","AI"],"title":"Five Things (May 23, 2026): AI in life sciences","updated_at":1792423596,"url":"https://blog.stephenturner.us/p/five-things-may-23-2026-aixbio","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"I originally wrote this for the AI Exchange @ UVA Substack newsletter on May 5, 2026. Even if you're not at UVA I highly recommend subscribing. Ryan Wright and Varun Korisapati are publishing some really interesting stuff over there.
A recent Nature paper from UW and Allen AI introduces OpenScholar, a retrieval-augmented language model that outperforms GPT-4o1 on scientific literature tasks despite being a fraction of its size.
Asai, et al. (2026). Synthesizing scientific literature with retrieval-augmented language models. Nature. https://doi.org/10.1038/s41586-025-10072-4
OpenScholar addresses a problem with using LLMs for research: citation hallucination.2 When asked to cite recent literature across fields like computer science and biomedicine, GPT-4o3 fabricates citations 78-90% of the time. The references look plausible, but the papers simply don't exist.
OpenScholar tries to ground responses in a data store of 45 million open-access papers with 236 million passage embeddings. Instead of generating citations from parametric memory, it retrieves relevant passages, synthesizes them, and iteratively refines its output through a self-feedback loop. The model drafts a response, critiques it, retrieves additional context if gaps are identified, and revises. This cycle continues until the output meets quality thresholds for factuality and citation accuracy.
On ScholarQABench (a new benchmark the team developed with nearly 3,000 expert-written queries), OpenScholar-8B outperformed GPT-4o by 6.1% in correctness and beat PaperQA2 by 5.5%.
When the team ran human evaluations with 16 PhD-level experts across computer science, physics, and biomedicine, experts preferred OpenScholar's responses over human-written answers 51% of the time (70% when using GPT-4o as the backbone).
Everything is released: models, training data, data store, retrieval indices, benchmark, and a public demo. The demo has already attracted 30,000+ users generating 90,000 queries.
Training data: https://huggingface.co/datasets/OpenSciLM/OS_Train_Data
Benchmarking: https://github.com/AkariAsai/ScholarQABench/tree/main/data
Queries: https://huggingface.co/datasets/allenai/openscilm_queries
OpenScholar code: https://github.com/AkariAsai/OpenScholar
Code to run the evals: https://github.com/AkariAsai/ScholarQABench
Public demo: https://openscholar.allen.ai
A notable limitation: the benchmarking predates current frontier models, and only compares OpenScholar to now-deprecated ancient models. recently remarked on a different paper to this effect:
That is: Benchmarks age fast. If a weaker model is close, a better model will often clear the bar later, and we learn little from the gap.
GPT-5.5 with extended thinking, Claude Opus 4.7, and commercial tools with deep research capabilities (like Gemini Deep Research or Perplexity Pro's updated pipelines) remain untested. Whether domain-specialized RAG maintains its edge against these newer systems is an open question.
See also my previous post on Consensus, and wiring up your own Zotero library to it, to be able to ask and answer questions about your own literature collection, find gaps, and fill those gaps in.
On June 17 2026 at noon I'll be teaching a workshop for UVA's AI Research Initiative AI upskilling series: Smarter Literature Reviews with AI-Powered Tools. You can register for the workshop here. Grab your spot now!
Yes, GPT-4o. A deprecated model that no one uses anymore.
I don't know if someone else has used the word previously, but I'm calling these \"hallucitations.\"
Personal anecdata: the frontier models at the actual frontier (GPT-5.4, Opus 4.7, as of this writing), especially with extended thinking enabled, have far fewer hallucitations than GPT-4o or other last-generated deprecated models. See the end of the post.
A short week for me here. Just ahead of the Opus 4.8 launch, two of these (writing/creativity and the despair of professors) hit the same theme, so they're stacked at the top. Next, the ESMFold2 release I haven't fully digested, a message from the pope on AI, and a piece of NIH news for researchers with foreign co-authors.
Two pieces from different authors converging on a similar point, and one I'll be writing about next week as well.
Ethan Mollick's essay Choosing to Stay Human isn't necessarily arguing against using AI or even AI for writing, but focuses on what to hand over. He's fine offloading phone numbers, arithmetic, cursive. He's not fine reflexively offloading writing, because writing is one of those tasks where the doing is the point. He cites his own BCG consultant study and a pair of Wharton-affiliated education experiments where small differences in classroom AI deployment produced opposite outcomes. In Turkey, students with plain ChatGPT did worse on tests than students without it. In Taipei, students with an AI tutor that gave personalized problem sequences scored 0.15 SD higher, roughly six to nine months of additional schooling.
Related is companion piece is Rebecca Winthrop's NYT op-ed on a Georgetown research program led by Adam Green that has been tracking college application essays before and after ChatGPT. In a study of 370,000+ personal statements, post-ChatGPT essays got rated as more creative by human judges while offering fewer novel ideas. A separate study found human-written essays contained up to 8x more new ideas than AI-assisted ones. The homogenizing effect was largest on students furthest from the mean, including neurodivergent students and racial and linguistic minorities. I.e., the kids whose voices are most distinctive are the ones most flattened by the assist.
Stay tuned next week. I'm writing a very short piece on the topic.
The natural sequel is Jay Caspian Kang's New Yorker piece, \"The Despair of the Professor in the Age of A.I.\" collecting testimonials from faculty members about what AI has done to their teaching. Hard to read. But nothing in it is surprising (I'd guess most academics have heard versions of these stories from colleagues).
The piece opens with Jane Sloan Peters, a religious studies professor at Mount Saint Vincent, describing a course she'd taught for years called \"Letters from Prison.\" Before AI, students struggled to land on themes and arrived somewhere personal through that struggle. Last year, every one of her students turned in something polished and empty.
From Beth Ritter-Conn at Belmont, on her honors students:
The tipping point was last year when I had Honors students\u2014Honors students!\u2014using A.I. to write reflection journals. Literally the only task there is \"tell me what you are thinking inside your own head.\" There is no right or wrong answer. It's just, Give me your thoughts on this thing. And I had students who outsourced that task to the robots.
From Susanna Boxall, a philosophy lecturer at Chico State, on online classes:
Now, online classes are a simulacrum of education: the students pretend to learn, and I have to pretend that I am teaching them something.
From Neal Hebert at Grambling State, who teaches theater and now assigns plays too obscure for ChatGPT to know about:
I've stopped being a collaborator in these intro courses and started being a plagiarism cop, and I do resent that a bit.
And from Jeremiah Croster, who teaches English at Houston City College:
The get-a-degree approach was already winning even before A.I., but now that it's here, the education part is starting to feel like something someone will write about in a history book. Or maybe A.I. will do it.
Not every voice in the piece is despairing. Auyon Siddiq at UCLA Anderson made his stats exam fully AI-permitted and reports the average is still 75%, because students who don't understand the material aren't saved by the tool. Daniel Silver at Toronto Scarborough redesigned his sociology assignments as multi-agent simulations and says the best final projects showed more creativity than what he used to get. So it's not uniformly grim, but the dominant note is loss.
Biohub (formerly known as EvolutionaryScale, now merged with the CZ Biohub network) announced ESMFold2 and a new ESM Cambrian (ESMC) family of protein language models this week, alongside an atlas of 6.8 billion protein sequences with 1.1 billion predicted structures. The paper is long and I haven't read it, so I'll be careful here.
What Alex Rives says in his thread, in his words: ESMFold2 is state of the art on protein interactions, especially antibody-antigen complexes. They designed and experimentally validated miniprotein binders and single-chain antibodies against five therapeutic targets with good hit rates including a PD-L1 minibinder that restores T-cell signaling at functional IC50 comparable to atezolizumab. They describe this as evidence that the model has materialized \"a world model of protein biology.\"
What is a \"world model\" anyway? Seems like \"world model\" is doing a lot of marketing work in AI right now, with no agreed-on technical meaning. Whether calling it a \"world model\" adds anything beyond \"the representations are structured in biologically meaningful ways\" is a fair question, right?
Either way, a quick look at their press release benchmarks and binder design results all look exciting. I'm looking forward to seeing what holds up under peer review.
Jill Lepore's New Yorker piece on Pope Leo XIV's new encyclical Magnifica Humanitas provides a nice overview for those of us without time to read the 40k+ word papal document on AI. Tech leaders have spent 20 years borrowing the rhetoric of mission, salvation, and disruption from religion, and the first American Pope, who took his name in reference to Leo XIII's 1891 Rerum Novarum on industrial labor, has decided to reclaim the territory. From Lepore:
Little in the encyclical is surprising; its force lies in its being said all at once.
I haven't read the whole encyclical yet so I'm not going to summarize it. It's interesting that (and this is partly Lepore's read but also visible in a quick skim of the text itself) part of the encyclical is about language and writing. There's a whole subsection on \"an ecology of communication.\" The Pope and Ethan Mollick and Rebecca Winthrop in #1 above are, in different vocabularies, worried about the same thing, that we are losing the conditions under which humans develop their own thoughts.
Our first task is neither to demonize nor idolize technological tools, but to utilize them on the basis of a fundamental principle, namely that truth is a common good and not the property of those with power or influence. We must therefore promote an ecology of communication.
Just pausing to say here that I find it amusing and puzzling that the company I keep who'd normally be first in line to criticize the Church, its anti-scientific doctrine, and its history of abuses, are now lining up behind the Pope as soon as he says something critical of AI. Cat's post on the topic stuck with me.
Jeffrey Brainard at Science reports that NIH grant managers are privately telling investigators they need pre-approval for any co-authorship with a scholar affiliated with a foreign institution, even when all the work was done in the United States. NASA is reportedly doing something similar for collaborations with researchers in China. Neither agency has issued public guidance.
The mechanism is that NIH has long required approval for grants with a \"foreign component,\" historically defined as significant scientific work performed outside the U.S. Now NIH is suggesting that mere coauthorship with someone at a foreign institution now counts as a foreign component, regardless of where the work happened. NIH is asking grantees to remove papers with foreign coauthors from annual progress reports if no foreign component was originally approved. An NIGMS email seen by Science asks institutions to promise that flagged U.S. authors won't collaborate with the relevant foreign coauthors in the future.
This creates pressure to remove foreign-affiliated coauthors from papers before submission, which violates standard publication ethics. Foreign-affiliated coauthors include visiting students and postdocs temporarily in the U.S., collaborators who donated reagents but didn't run experiments, and U.S.-trained scientists who moved abroad after the work was done. The rule is seems to be administered through private emails rather than formal guidance which means it can shift without anyone being able to point to the rule that changed.
","doi":"https://doi.org/10.59350/dandr-t9984","guid":"199512579","image":"https://substackcdn.com/image/fetch/$s_!IPZi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F07b6ef47-ff05-4350-9de4-6b6b78cecaa1_1260x571.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1780012800,"rid":"2mask-hbg06","summary":"Writing and thinking, the despair of the professor, ESMFold2 and a world model of proteins, Pope Leo on AI, NIH's foreign co-author crackdown","tags":["Papers","AI"],"title":"Five Things (May 29, 2026): AI & writing, ESM","updated_at":1792423590,"url":"https://blog.stephenturner.us/p/five-things-may-29-2026-ai-writing-esm","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"Anthropic provides a free, self-paced course catalog on covering Claude, Claude Code, the Claude API, MCP, and several AI fluency tracks tailored to educators, students, and nonprofits. Available at anthropic.skilljar.com.
I've taken a couple of these. They're short. If it matters to you, when you complete the course you can get a certificate that you can wire up to your LinkedIn profile, which link out to the verified cert (example).
Full list as of this week below.
Claude 101: Learn how to use Claude for everyday work tasks, understand core features, and explore resources for more advanced learning on other topics.
Claude Code 101: Learn how to use Claude Code effectively in your daily development workflow.
Introduction to Claude Cowork: Learn to work alongside Claude on your real files and projects. This hands-on course covers the Cowork task loop, plugins and skills, file and research workflows, and how to steer multi-step work responsibly, so you're productive in your first week.
Claude Code in Action: Integrate Claude Code into your development workflow.
AI Fluency: Framework & Foundations: Learn to collaborate with AI systems effectively, efficiently, ethically, and safely.
Building with the Claude API: This comprehensive course covers the full spectrum of working with Anthropic models using the Claude API.
Introduction to Model Context Protocol: Learn to build Model Context Protocol servers and clients from scratch using Python. Master MCP's three core primitives (tools, resources, and prompts) to connect Claude with external services.
AI Fluency for educators: This course empowers faculty, instructional designers, and educational leaders to apply AI Fluency into their own teaching practice and institutional strategy.
AI Fluency for students: This course empowers students to develop AI Fluency skills that enhance learning, career planning, and academic success through responsible AI collaboration.
Model Context Protocol: Advanced Topics: Discover advanced Model Context Protocol implementation patterns including sampling, notifications, file system access, and transport mechanisms for production MCP server development.
Claude with Amazon Bedrock: As part of an accreditation program created for AWS, Anthropic launched a first-of-its-kind training for AWS employees. Here's the full course so you can follow along.
Claude with Google Cloud's Vertex AI: This comprehensive course covers the full spectrum of working with Anthropic models through Google Cloud's Vertex AI.
Teaching AI Fluency: This course empowers academic faculty, instructional designers, and others to teach and assess AI Fluency in instructor-led settings.
AI Fluency for nonprofits: This course empowers nonprofit professionals to develop AI fluency in order to increase organizational impact and efficiency while staying true to their mission and values.
Introduction to agent skills: Learn how to build, configure, and share Skills in Claude Code, reusable markdown instructions that Claude automatically applies to the right tasks at the right time. This course takes you from creating your first Skill to distributing them across teams and troubleshooting common issues.
Introduction to subagents: Learn how to use and create sub-agents in Claude Code to manage context, delegate tasks, and build specialized workflows that keep your main conversation clean and focused.
AI Capabilities and Limitations: An introductory course about how AI works.
","doi":"https://doi.org/10.59350/wshf7-c5h65","guid":"196772830","image":"https://substackcdn.com/image/fetch/$s_!-ap9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Faa77796a-7aef-4c63-aac7-1ee7c5028a7f_432x611.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1780185600,"rid":"h404v-zmt61","summary":"17 free, self-paced courses from Anthropic covering Claude, Claude Code, the API, MCP, agent skills, and subagents, plus AI fluency tracks for educators, students, and nonprofits","tags":["TIL","AI"],"title":"Free Claude Courses from Anthropic","updated_at":1792423588,"url":"https://blog.stephenturner.us/p/free-claude-courses-from-anthropic","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/0153tk833","name":"University of Virginia"}],"contributor_roles":[],"family":"Turner","given":"Stephen","url":"https://orcid.org/0000-0001-9140-9028"}],"blog":{"authors":[{"name":"Stephen D. Turner","url":"https://orcid.org/0000-0001-9140-9028"}],"community_id":"382941a7-2ffa-41df-8bbb-5f772188517f","created":1780876800,"current_feed_url":null,"description":"A practicing data scientist's take on AI, genomics, biosecurity, and the ways AI is reshaping how science gets done. Weekly updates from the field. Occasional notes on programming.","doi":"https://doi.org/10.59350/stephenturner","favicon":"https://rogue-scholar.org/api/communities/382941a7-2ffa-41df-8bbb-5f772188517f/logo","feed_format":"application/rss+xml","feed_url":"https://blog.stephenturner.us/feed","filter":null,"generator":"Substack","home_page_url":"https://blog.stephenturner.us","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"stephenturner","status":"active","subfield":"1311","title":"Paired Ends","updated":1792412123,"use_api":true},"blog_name":"Paired Ends","blog_slug":"stephenturner","content_html":"For the last few years I've done a Sober October1 where I abstain from any alcohol for the entire month. I don't drink much these days anyway, but a solid month of teetotaling abstinence is a nice reset button that turns something habitual into something I mindfully enjoy with friends and family.
I've developed a similar habitual relationship with AI. I reach for it thoughtlessly when I see an essay that's a little longer than I want to read,2 or when I don't feel like writing that cover letter for a manuscript,3 or when I want to avoid the mental strain of thinking hard about the architecture of whatever code I'm working on.4
AI deskilling is real. A couple weeks ago I was writing specific aims for a proposal with a few brilliant colleagues here in SDS, and I felt the constant urge to open up a Claude window. Just to write aims! In another setting, while in a coaching session with one of my CSTE Data Science Team Training teams last week, I opened Positron to demonstrate some fairly basic tidyr rectangling I've done hundreds of times, and couldn't easily remember how to use the pivot_*() functions. It felt like coming back to a foreign language after a few years without using it.5 AI deskilling is real, and if you use these tools with any regularity you know it deep down.
Next month I'm doing an AI Dry July.6 I invite you to join me.
I'm going to resist using generative AI as much as possible for the whole month. With alcohol in Sober October the rule is easy \u20147 if it contains ethanol, I'm not having it. It's a little more difficult with AI. You can't do a web search or read a product without AI overviews. But generally, I'm not going to reach for ChatGPT or Claude for reading, writing, or for most coding problems.8 I'm teaching a new class this fall (Genomics Foundations for Data Scientists). I'm going to ask my students to struggle through some of the readings and assignments,9 so it's only fair if I struggle through creating those assignments and course materials. I'm exhausted from reading AI slop in everything from academic literature to the New York Times to many of the blogs and newsletters I personally recommend, so it's about time to stop compulsively reaching for AI to polish my own writing.10
I don't really care that my mental arithmetic is terrible because of calculators, or that my navigation skills aren't what they were back in the pre-GPS days of the 90s (the best decade). I want to avoid the habitual slide into offloading some part of nearly every cognitive task to the chatbot. At least for a month, to re-center myself and take stock of where the deskilling is creeping in.
I know I've written very positively about AI in this newsletter, in everything from writing code to peer review. And I'll continue to do so. I'm neither an AI evangelical nor an AI doomer, and I'll highlight problems where I see them. Even in 2026 it's possible (gasp!) to have nuanced views. And I'm starting to think habitual and compulsive AI use is probably not that great for me over the long run.
We'll see how this works out. I'll write something at the end of the next month on how it went and what I took away from it.
TIL this is known as Ocsober in Australia, where the also have a Dry July. Others here in the U.S. do a Dry January, but where I live in Virginia we usually get at least one heavy snow in January that shuts down schools for a few days, where a Lord of the Rings marathon with a good bourbon is required.
Shamefully, I find myself doing this with real essays from real writers I know, clacking away on real keyboards with their real human fingers.
I went so far as to write a Claude skill that covers my tracks for writing I don't want to do.
Last year I spent some time in Spain, Mexico, and Costa Rica, and could comfortably carry a 30 minute conversation with a local in Spanish about my family, our interests, why we're going on this snake-infested hike, or where to get a good paella. Recently when trying to converse with a Salvadoran guy I hired to do some handiwork around my house, just a year later without much practice I struggled with getting basic concepts across. It's not like riding a bike, and I'm afraid that neither are all the things we do with AI, unassisted.
Why July? Why not start now, June 1, as I'm writing this? It's not just the rhyme. I'm teaching several workshops on AI-enhanced literature review and synthesis this month.
The em dash here and elsewhere in this newsletter reflects my stylistic preference and should not be interpreted as evidence of AI-assisted text generation.
I say most coding problems here because the place I'd have historically gone to ask questions, Stack Overflow, is dying a well-deserved death. I don't write as much code these days as I used to, but I'm leaving myself a little wiggle room here.
Wish me luck. I'm looking forward to having undergraduates in the classroom with our masters students, and I plan to assign seminal primary research papers and \"journal club\"-like oral discussions for assessments. I have no idea how this will go with the 2026 cohort of students who haven't experienced college (or part of high school) without ChatGPT.
I'm not the only one. Wharton School professor recently wrote about Choosing to Stay Human, and , director of the Center for Universal Education at the Brookings Institution, recently wrote something similar in an op-ed at the New York Times.
Before the usual five, one thing that isn't optional reading. Pay attention.
I'm not an activist, and I don't use this space for advocacy. As a rule, I keep my opinions about research politics and policy off the site (and anywhere else online in general). I'm breaking that rule today, because this reaches every part of the scientific research enterprise and whatever role the United States still hopes to play in it.
The Office of Management and Budget has proposed rewriting 2 CFR Part 200, the rule that governs how every federal grant gets spent. A Science editorial lays out what that would mean for research: every funding decision routed through political review, multiyear grants terminable with no due process, and case-by-case approval for any award that spends a dollar outside the US, which would effectively end most international collaboration. Steve Usdin of BioCentury, quoted in the editorial, called the drug industry's silence \"complicity in the destruction of US science.\" The Science editorial ends with a rallying call:
\ud83d\udea8 The red light is now flashing. All hands, report to stations \ud83d\udea8
, who spent >20 years as an NIH program officer, writes in an essay making the point that this is much larger than science. 2 CFR Part 200 is the universal framework for federal grants, so the same political override and discretionary termination provisions land on Medicaid, Title I schools, highway funds, and tribal health programs.
The comment period closes July 13. Ginexi's how-to guide is a great place to start, and mechanics are important - since a thousand identical form letters count as one comment, so write your own, in plain language, and cite the specific provisions that would hurt your work. If you hold, have ever held, or in the future want to hold a federal grant, this is ten minutes well spent.
Now, onto the five things that interested me this week. Most of this week circles one question, whether we can measure what AI models actually do and trust the measurement. OpenAI decides the answer is yes and is shipping a life-sciences model to vetted partners.
Open models stay (only!) about four months behind the frontier.
Where evals actually sit in the biorisk evidence stack.
DARPA and NSF want universities for AI Forge (bring an IP-sharing agreement).
Two papers demonstrate AIxBio refusals tell you almost nothing.
OpenAI hands GPT-Rosalind to biodefense partners.
Epoch AI put a number on this open/frontier model gap: since January, the best open-weight models have trailed the closed frontier by about 4 months, or 8 points on Epoch's Capabilities Index, roughly the distance from GPT-5 to GPT-5.5.
The \"four months\" figure uses a generous definition of catching up: an open model counts if it beats the old frontier in at least 5% of bootstrap samples. Epoch also notes that closed labs sit on their most capable models for safety or commercial reasons, so four months probably understates it.
As frontier models move to token/usage-based pricing, the open option is real and usable, they're not terribly behind on some benchmarks. Whether those benchmark points translate into the work you actually do is a separate question, and section 4 suggests the answer is messier than a single index implies.
at answers a recurring jab at AI biorisk benchmarks (they don't measure what we care about, so why bother) by placing evals in a four-tier hierarchy of evidence.
Tier 1 is arguing from first principles, which got the field started but convinces no skeptic. Tier 2 is evals, where we are now. Tier 3 is real-world uplift studies, the gold standard and the expensive one. Tier 4 is an actual incident, which nobody wants as a data point.
evals provide the best cheap, comparable and repeatable evidence we can get
I really liked the section G\u00f6tting calls the rocky terrain between tiers 2 and 3, so much that I'm quoting the whole segment in full below. A well-powered uplift study costs millions and goes stale the moment the next model ships. Jasper proposes a bridge: if you collect rich data on how participants used the model during an uplift study, you can map their failures and successes back onto benchmark tasks, building what he calls correlates of uplift. That gives you a cheap proxy you can re-run on every new model instead of re-running the whole study. A small number of expensive studies can underpin a much larger cheap monitoring effort.
Assume you ran your large uplift study, found some effect, and two months later, the new generation of frontier models is released. Can you make any sensible claim about the impact on real-world risk of these models (besides \"probably as good as the last gen\") without re-running everything? Maybe! You carefully approach the edge of the cost cliff you valiantly scaled and peer down to the vast plain of evals again.
If rich, qualitative data on LLM usage were collected during the uplift study, you should in principle be able to map the failures and AI-assisted successes of your study participants onto benchmarks measuring a corresponding capability. If model A failed to guide participants around a common methodological pitfall, and model B didn't, is the same pattern detectable in a benchmark task about this method? Is the delta in overall uplift-effectiveness between models or model generations the same as the delta in the relevant benchmark results? Did you find interesting model differences and behaviors that don't yet map onto a benchmark, but such a benchmark could be made? Did you encounter missing refusals to questions that should be refused in hindsight? All these can let you draw connections between evals and real-world uplift studies.
Finding these connections and correlations between uplift studies and in silico evals (\"correlates of uplift\") is extremely important as it provides you with an OOM cheaper estimator for the underlying model capability that you can tweak and run over and over again for every newly released model; and, crucially, also for pre-release testing and risk assessment.
Jasper is blunt about the failure modes (specification gaming, underelicitation, sandbagging, weak construct validity), but the point stands: in biorisk, a false negative costs vastly more than a false positive, so waiting for tier 3 certainty has a price that critics rarely put on the books.
See also:
Hong, S. Z. et al. Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology. Preprint at https://doi.org/10.48550/arXiv.2602.16703 (2026).
Zhang, C. B. C. et al. LLM Novice Uplift on Dual-Use, In Silico Biology Tasks. Preprint at https://doi.org/10.48550/arXiv.2602.23329 (2026).
Paskov, P. et al. RCTs & Human Uplift Studies: Methodological Challenges and Practical Solutions for Frontier AI Evaluation. Preprint at https://doi.org/10.48550/arXiv.2603.11001 (2026).
DARPA and the NSF, with CAISI at NIST, announced AI Forge, a program to fund university research on 3 problems: AI interpretability, AI control, and adversarial robustness. The structure is a forum of universities, frontier AI companies, and government, running what DARPA calls Project Ventures, fast one-year university efforts from $750K to $3M or more, several per year. The Critical AI Challenges report alignts in part with this week's other items, heavy on whether you can evaluate, interpret, and trust model behavior. One of its challenges is evaluating AI systems for scientific discovery once they outrun human expertise.
The RFI (DARPA-SN-26-80) asking universities to describe their capabilities is due June 22 at 5 PM ET. Two requirements are worth noting before spending too much time on a response (which must come from your VPR/Provost, not your individual lab). Key personnel are limited to US citizens and permanent residents, and the IP generated is meant to be shared across the forum, preferably under the MIT License.
This RFI is targeting U.S. universities. Further, AI Forge Project Venture designated key personnel (e.g., Principal Investigators [PIs], Co-PIs, and key research personnel) are anticipated to be limited to U.S. citizens or permanent residents.
Please note that Intellectual Property (IP) generated from Project Ventures is intended to be shared among AI Forge forum participants, preferably under liberal open-source licensing (e.g., the MIT License). Universities unable or unwilling to agree to a shared IP framework should not respond to this RFI.
Two preprints landed this week on we evaluate model safety in biology: we count how often a model refuses, and almost never check whether the refusal means anything. Both are preprints, and one is a hackathon project rather than peer-reviewed work, so weight them accordingly.
BioRefusalAudit, by Caleb DeLeeuw, built over a single weekend on a consumer GPU, asks whether a refusal is structurally sound or melts away under small changes. The results across five small models are not reassuring. Gemma 2 2B never refused across 75 prompts; it just hedged. Gemma 4 refused 65 of 75 prompts with chat-template formatting and 0 of 75 without it, and both Gemma models dropped to zero refusals once you capped output at 80 tokens, the kind of cap production systems use for cost and latency. Qwen and Phi-3 swung the other way, flagging 83 to 87% of benign biology as hazardous. The psilocybin probe was interesting: some models refused to discuss cultivating a legal-to-possess, biologically non-toxic substance more often than genuinely hazardous biology, which suggests refusal is tracking legal status and cultural taboo rather than actual CBRN risk.
RefusalBench, from Applied Scientific Intelligence, is the heavier study: 141 prompts in matched triples that hold the task constant and vary only biological risk tier, run across 19 frontier models. On identical prompts, strict refusal rates ran from 0.1% to 94.6%. Jurisdiction didn't predict refusal, but provider did, with Anthropic's API stack predicting refusal at an odds ratio around 21. The authors are careful that this is an access-path effect, not a statement about model weights: 99.8% of Anthropic's refusals carried the same canned policy reason code, consistent with template-based filtering rather than case-by-case reasoning.
The TLDR I took away from this is that refusal rate misranks safety.
Grok 4.20 (lol, seriously?) had the best tier discrimination in the panel while ranking only seventh by refusal rate, and Claude Opus 4.7's discrimination dropped 65% from the prior version, driven entirely by refusing more legitimate research prompts (benign-tier false positives jumped from 33% to 77%) with no gain in catching dual-use ones. 9 of 18 frontier models showed a hedge-but-help pattern at the dual-use tier.
Put together, a 4B model on a laptop and the frontier lineup all tell the same story in that refusal rate is close to noise as a safety signal. Thinking of wiring an LLM into a protein design pipeline? A more conservative model isn't a safer one if it terminates your legitimate workflow at step 2, and the RefusalBench authors found the most-refusing models were often the worst at telling benign from dangerous. That a vendor selling agentic biology tooling ran this study is worth noting, but the matched-triple design and the should-refuse control set make it hard to write off.
All the data and prompts are available at github.com/AppliedScientific/refusalbench.
OpenAI launched Rosalind Biodefense, sponsoring access to GPT-Rosalind, its life-sciences reasoning model, for vetted developers building biodefense tools, and extending trusted access to select US government and allied partners. The launch cohort includes Fourth Eon and SecureBio on the screening and detection side, with LLNL, Johns Hopkins APL, and CEPI on the government side. The pitch is \"defensive acceleration,\" putting frontier capability in defenders' hands first.
A vetted-access model is a sensible way to deploy a tool whose offensive and defensive uses are the same capability pointed in opposite directions, which is exactly why OpenAI gates it similar to what Anthropic did with Mythos. It is also a way to expand deployment of a model the company itself treats as High Capability in biology (the threshold it first crossed with ChatGPT agent last July) while keeping the optics clean (take \"meaningfully advantage defenders\" as a goal not a demonstrated result, yet!).
This dropped the same week as the OpenAI Foundation's resilience announcement (more below) and a new cyber-focused executive order, so the bio, cyber, and safety messaging is moving together. And SecureBio appears here as a detection partner, in section 2 as the source of the evals argument, and again in the community roundup below. For a field this consequential, it runs on a small but mighty number of organizations doing important and exciting work!
(Section drafted/organized with Claude Opus 4.8)
Frontiers' open special issue, Artificial Intelligence and the Future of Biosecurity (edited by Bin Hu of Los Alamos, Dov Greenbaum of Yale, and Michelle Holko at Berkeley), collects perspectives and frameworks on the dual-use problem of AI-enabled biology. Worth a scan rather than a read-through:
The June GCBR organization update rounds up Active Site, the Asia Centre for Health Security, IBBIS, and SecureBio.
at always has a great weekly recap, so much so that I model the recap here off of his weekly \"Five Things\" series. His Five Things: May 31, 2026 post covers Illinois SB 315, ESMFold2, papal encyclical, Opus 4.8, OpenAI's Rosalind Biodefense.
A new executive order on AI innovation and security sets up a \"covered frontier model\" designation, a Treasury-run vulnerability clearinghouse, and a voluntary window of up to 30 days of early government access to qualifying models before wider release.
The OpenAI Foundation detailed an AI Resilience program across four areas (bio, cyber, model safety, and effects on young people), saying it is finalizing more than $130M in grants and intends to commit over $1B in the next year.
openRxiv (Richard Sever and Tracy Teal) sketched where bioRxiv and medRxiv are headed on trust signals as AI makes fake-but-plausible research cheaper: verified data deposits, funder metadata (now on roughly 80% of papers), and the hard problem of identity verification, since ORCID disambiguates authors but doesn't confirm they are who they claim.
NeurIPS desk-rejected 178 position-paper submissions (18.4%) for being substantially AI-written, using the Pangram detector, with another 123 asked to prove human authorship.