![\"\"](\"https://substackcdn.com/image/fetch/$s_!CFBM!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2d1077fd-c0ac-42e3-a2d2-72f50e1c86fe_935x8.png "\\"\\"")
A new working paper about mass reproducibility in psychological science by Brodeur, Baki, & Fazio finds: \"84% of preregistered articles deviate from their pre-analysis plan, and nearly half of these deviations go unacknowledged.\"
![\"\"](\"https://substackcdn.com/image/fetch/$s_!EHQL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F57ae99a3-bbbb-48c6-90a8-f0bb7fcc0b80_730x950.png\")
As Brodeur et al. explained:
Among the 50 preregistered reported articles, 42 (84%) deviated from the pre-analysis plan in at least one respect, and 8 (16%) did not. Of the 42 articles with deviations, 25 (59.5%) disclosed some or all of them, while 17 (40.5%) disclosed none. The most common form of deviation was implementing a different procedure in the paper from the preregistration, occurring in 57.143% of preregistered articles with deviations. \u2026. Deviations involving design or reporting scope were most frequent, occurring in 25 articles, followed by estimation or modeling choices, occurring in 23 articles. Deviations related to sample size/data collection and data exclusions/filtering were each identified in 12 article, while deviations concerning variable construction were prevalent in 10.
The 84% deviation rate is close(ish) to my own estimate that \"around three-quarters of preregistered studies included at least one deviation\" (Rubin, 2025, p. 19, based on studies by Cheng, 2022; Claesen et al., 2021; Clayson et al., 2025; Heirene et al., 2024; Singh et al., 2021; TARG Meta-Research Group and Collaborators, 2023). In my article, I noted that this
evidence is unsurprising given that leading proponents of preregistration encourage researchers to treat preregistration as \"a plan, not a prison\" (Chambers, 2019, pp. 188, 189; DeHaven, 2017; Nosek et al., 2019, p. 817)\u2026 [But the problem is that] adjustable plans do not control the Type I error rate (see also Navarro, 2020, p. 8).
Brodeur, A., Baki, G. A., & Fazio, L. (2026). Mass reproducibility in psychological science. I4R Discussion Paper Series, No. 326, Institute for Replication (I4R). https://hdl.handle.net/10419/344024
and colleagues report the results of their study of prospective registration among observational studies of health care interventions. As part of their study, they surveyed 46 authors about the perceived benefits and barriers to registration. They found that 60% of responders expressed positive attitudes towards registration, and that barriers to registration included limited resources (62%, 26/42), lack of tradition (57%, 24/42), and insufficient knowledge of registries (50%, 21/42).
![\"\"](\"https://substackcdn.com/image/fetch/$s_!7TZM!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F776785f8-dd00-4e94-9187-9e633c50c3f3_1182x758.png\")
Our findings regarding author attitudes toward registration align with previous similar studies. Although authors typically acknowledge the benefits of registration for improved planning and methodological transparency of observational studies [26, 27], concerns persist regarding limited perceived added value (e.g., unnecessary bureaucratization, increased workload), unintended side-effects (increased costs, and discouragement of exploratory work), and reduced analytical flexibility [26\u201328].
Jespersen, C., Song, Z., Chan, A. W., & Hr\u00f3bjartsson, A. (2026). Registration of observational studies of interventions: Prevalence, author attitudes, and journal policies \u2013 a meta-research study. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-026-00258-3
![\"\"](\"https://substackcdn.com/image/fetch/$s_!ryOv!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fefb7690f-1076-48b0-a819-707482a5775d_935x8.png "\\"\\"")
Torsten Wilholt published an article in which he asks \"Is trust in science a good thing?\"
![\"This](\"https://substackcdn.com/image/fetch/$s_!42B4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1dc9d49-990a-4285-98bf-a6843f914edf_803x737.png\")
Part of Wilholt's answer addresses metascience and the open science movement. He notes that the response to the crisis of confidence has been to increase transparency and methodological standards in order to make science more controllable and verifiable. He contrasts this open science approach with Hardwig's (1985) thesis that scientists need to have a certain degree of trust between one another. He points out that:
If we can regulate all actions in the research process by standards and conventions and make them fully explicit, then there is no need to leave any discretionary powers to researchers, and trust can be replaced by control.
However, Wilholt ends up arguing against this position because standards designed to regulate scientific research cannot be applied to research that targets deep ignorance, in which we do not yet have any candidate answers, and radically deep ignorance, in which \"finding the correct answers would require a radical overhaul of our conceptual resources, our styles of reasoning, or our background beliefs.\"
Many of the standards advocated by the reform movement are embedded in frameworks of statistical analysis. This applies to the call to move away from simply calculating p-values and to pay closer attention to other statistical measures such as power, effect sizes, or confidence intervals, as well as, of course, to the more radical proposals to use Bayesian statistical analysis rather than frequentist methods. Statistical methods, whether Bayesian or frequentist, work only against the backdrop of a spectrum of previously identified competing hypotheses. The point also concerns other proposed methodological improvements, such as the pre-registration of studies, which virtually the entire open science movement expects to be highly effective. Questions for which we have no potential answers cannot be made the subject of research in a way that satisfies such frameworks. Yet science also needs research that aims to overcome deep and even radically deep ignorance. It is not at all clear how the substitution of trust with control is supposed to work in such research. Hardwig's thesis remains as valid as ever.
Wilholt's point remind me of points made by others about the difficulties of conducting preregistered confirmatory research with clearly defined alternative hypotheses (H1) and effect sizes. For example, Pek et al. (2024) explained that:
For most psychological research, if the value of H1 were known, there would be little reason to conduct the research. The unknown value of H1 is the very reason Fisher (1955) strongly opposed specifying an alternative H1 value. The high uncertainty in specifying H1 prevents sensible values of Type II error rate \u2026 and power \u2026 from being calculated
Similarly, although Scheel et al. (2021) see value in preregistered confirmatory research, they also cautioned that, in some cases, psychological scientists may not be ready to test hypotheses:
Psychological scientists have not developed methods for specifying the alternative hypothesis in sufficient detail to make it statistically falsifiable (Meehl, 1967; Morey & Lakens, 2016). This problem is not solved with mere practice\u2014forcing researchers to specify what would falsify their hypotheses when they have no theoretical basis for doing so can lead to testing against arbitrary values (Kruschke, 2018) and runs the risk of replacing one mindless ritual with another.
Wilholt concludes that:
to try and substitute control for trust would mean to fail to recognize that research cannot be fully standardized, as shown by the case of research aimed at overcoming deep ignorance. Hardwig's thesis about the irreplaceability of trust within science thus remains valid.
Wilholt, T. (2026). Is trust in science a good thing? European Journal for Philosophy of Science, 16, 74. https://doi.org/10.1007/s13194-026-00781-6
Tomorrow we\nwill have a discussion in our lab about how to use AI as a scientist. I have\nincreasingly started to use AI, having moved from Copilot Chat that our\nuniversity provides last year, to some free credit on GitHub Copilot under\ntheir educational program, to Claude Pro integrated in Visual Studio Code, to a\nClaude 20x max subscription (in part to also make these AI tools available to\nmy lab in case they want to use them).
\nI work with\ncollaborators whose opinions and values are important to me, and several of\nthem are more skeptical about the use of AI. I see their concerns, both in how\nmy students use AI, and how some peers use AI. I am as annoyed by people use write texts where AI has\nmade up references as anyone. I also see how tempting it is to use AI mindlessly. It does\nso much, so quickly, and validating that it has done things well is so\neffortful, that you get sucked into overtrusting AI. Some people don't want to\nuse AI at all, but for me, the question is not whether to use AI, but how to\nuse it well. Of course, I don't see any problems with how I use AI, and like in\nthe rest of my work, I need others to criticize me to point out my own biases.\nSo I am writing down my current thoughts, to make them as explicit as possible.
\nI use smart\nhome tools, and have been using If This Than That (IFTTT) for years. I like to\nuse technology to make my life more efficient. In the last year, I have been\nusing AI to create tools that improve my life. For example, I have asked an AI\nto create a pipeline where I can transform text (books and papers) to audio on\nmy laptop, sync it to my phone, where I have a custom made Android app that I\ncan use to listen to these audio files. When I swipe back, I can mark the\nsection I listen to, and the text (with the use of meta-data files that link\nthe time stamps in the audio to the sentences in the text) of this section gets\nautomatically synced to my Obsidian note-taking tool. If that feels like\noverdoing it to you, and that you might go as far as buying a tablet to read,\nbut that building custom software is a bridge too far, I understand. My\npersonality is such that I like to be efficient. I would predict that the more\nefficiency matters to you in your job, the more you use technology, including\nAI.
AI use also\naligns with work I do. I spend a lot of time behind my computer, interacting\nwith software, working on websites, and I program R packages and tools such as Metacheck. Of course a computer already automates a ton\nof our activities, so AI use is an extension of the tasks we automate. But if\nyou spend less time behind a computer, and especially if you do not code, there\nwill be less use-cases for AI.
\nI taught a\nHuman Factors course, and the book we used has a chapter on\u00a0human-automation interaction. It starts with\nthe question \"Why automate\" and offers 4 reasons: 1) it is impossible of\nhazardous for people to perform these tasks, 2) tasks are difficult or unpleasant,\n3) automation can extend human capability, or 4) we do it just because it is\npossible, even where it has no benefits. To the extent that we see AI as\nautomation, I have never used it to do impossible or hazardous tasks, but I\nthink I have experienced the other 3 categories. Sometimes the use of tools has\nbeen useful, sometimes less so (but learning when AI use is useful is for me\njust part of learning to use new technology).
\nLet's start\nwith unpleasant tasks. I often perform tasks that take a long time, and/or need\nto be monitored, but that require very little cognitive resources, and that an\nAI can do.
\nI teach\nIntroduction to Psychology and Technology in the bachelor this quarter, and\nMetascience in the master. Both courses have weekly assignments that should be\ngraded. The colleague who was supposed to assist with grading fell ill. We use\nthe Canvas learning management system. Many of the assignments are mainly\nintended to activate students \u2013 for example, they have to walk around campus\nand take pictures of things in the built environment to assist blind and/or\ndeaf people to find their way, or they perform an online experiment and have to\ncopy-paste their results. All I do for these assignments is check if they have\ncompleted them, and then I assign them the points. There are also open\nquestions that need actual grading by comparing answers to a rubric.
Canvas has\nan API, but figuring out how it works is too difficult and unpleasant so I\nnever used it in the past. But AI can figure out how the API works. I use AI to\nwrite R scripts that download all assignments, and store the questions and\nanswers in an spreadsheet. I use that spreadsheet to grade, which prevents me\nfrom having to click 20 times for all 160 students in the slow online Canvas\nsystem. When I am done with grading, I use an R script to upload the results to\nCanvas.
I also use\nClaude to check if assignments are completed. In the past, I would give all\nstudents a point for some questions without checking if they actually uploaded\nreal pictures, as the assignment instructed. I knew some students uploaded\nimages unrelated to the assignment in an attempt to make it look like they did\nthe assignment, and I disliked that I did not have the time to grade these\nassignments, but I found it an acceptable trade-off for more engaging class\nassignments. Now, I can use AI to check the assignments I would otherwise just\nassign points for, thereby keeping the more engaging assignments, while also\nchecking that students who try to get point without doing the assignment receive\nno points.
There are a\nnumber of similar use-cases for grading. I have compared the quality of grading\nof low-stakes assignments last year, compared to this year, and the use of AI\nhas improved the accuracy of the grades I assign. To be clear, I was aware that\nmy grading was too lenient, but I think weekly graded assignments are more\neducational than removing them due to a lack of time to grade them carefully.\nFor both courses, the assignments that are graded with AI are a small\npercentage of the final grade, unlikely to determine who passes a course, and\nwho not. The psychology course has an in person final exam and for the\nMetascience course I have switched to (rather labor intensive) oral exams to\nmake that master course AI proof.
The time\nsaved on grading with the use of AI is substantial (approximately 2 hours per\nweek, for 8 weeks), and the quality has increased. At the same time, I invest\nsome of this time back into education, making new and more engaging\nassignments, and more personal assignments and tests. I think it has made my\neducation better.
\nI perform\nmany tasks on a computer that take a long time. In the past, I ran simulations\nthat would last days, and currently, I run automated checks that Metacheck\nperforms on thousands of scientific articles. When a computer runs for days,\nthere are many things that can go wrong. AI helps me to create scripts that are\nmore reliable, that store intermediate results and can restart easily, informs\nme when there are unforeseen problems, and allows me to quickly resolve them.\nClaude allows you to chat with another computer through the browser, which\nmakes it very easy to leave a computer at work on over the weekend, while\nmonitoring its progress, and asking Claude to resolve problems that emerge. For\nexample, if during a run it becomes clear that there is a bug in a script,\nClaude can notice this before the run has continued for days, fix the code, and\nrestart. I remember how on the past I would often rerun the same thing for\nmultiple times across multiple days (often a real source of frustration in my\nwork).
\nI try to\nimprove my research practices, but there is only so much I am willing to learn.\nI have been using GitHub, as an amateur, but at least I managed to made code\nopen and have version control. But I never received training in Git, and I have\nused it suboptimally. Claude makes it much easier to adopt good coding\npractices, as well as cleaning up a repository after I made a mess of things.\nCreating new branches, using Large File Storage, creating public releases,\nmaking code reproducible, and documentation have improved a lot.
I am also\nincreasingly using Docker, which was another tool I knew I had to learn to use,\nbut found difficult. I was able to run containers in the past of there was a\nclear tutorial, but now I can also make docker containers and share them (for\nexample as part of Metacheck's reproducibility_check where we automatically\ncheck if code in repositories is computationally reproducible in a docker\nenvironment where most R or Python packages are pre-installed to speed up the\nchecks).
In an\nongoing project, where I am iteratively improving the code I am using, and\ntherefore the data I am generating is updated, I was losing track of which\nfiles where updated, and which not, with the risk that the manuscript described\nstale results. I knew targets\nwas the tool I should use to prevent this confusion, but I really did not have\nthe time to learn it. With the help of Claude, I reorganized my repo and\nimplemented it anyway.
\nThere are\nonly so many things I can learn, and my lack of knowledge limits the things I\ncan do. This is especially true when it comes to writing software and code.
AI helps me\nto complete ideas I have that I would otherwise never get around to complete.\nFor example, I turned static pictures into interactive visualizations in my html textbook. I want free\nopen textbooks to be better than paid textbooks by commercial publishers, and\ninteractive apps is one way to achieve this. But I don't have the time to learn\nhow to turn my code into an interactive app. AI is very good at this. This is\nalso true for the creation of Shiny apps. It took me a long time to learn Shiny\nat a mediocre level, but with AI, turning code into an R Shiny app is trivially\neasy. I have to load Shiny apps on my server,\nwhich runs on Linux, but I don't know Linux well, and AI is also of great help\nhere. Shiny apps are now more up to date, because maintaining them and updating\nthem takes less time.
One area\nwhere the use of AI becomes more problematic is when 'extending human\ncapabilities' means 'doing more in less time'. Where scientists used to have to\nbike to the university computer, hand in their programming cards, and return a\nday later to get the results, we can now perform statistics on our laptop in\nseconds. Where we used to browse through paper journals, we can now download\nany pdf version of an article we want. These innovations mean we can do things\nmore quickly. Where should we use AI to speed up our work, even if we can also\ndo this work manually? And, how do we know we are extending our capabilities,\ninstead of using AI to create flawed work?
\nWhen I let\nAI write code, I extensively validate the code against ground truth. This is\neasy when I know what results code should return, and not possible when I do\nnot know what code should return. When I program on Metacheck and I develop a\nnew module, for example one that automatically retrieves datafiles from\nrepositories, I use manually coded ground truth to validate the code I create\nwith AI. I know that my code should find repositories, and I can check if it\nactually finds them. Even in this validating step, AI can be very helpful.\nWhere I might manually check 20 repositories, I can use AI to check all\nrepositories.
Here, we\nget to some challenges I experience when using AI. Manual validation against\nground truth sounds like a good golden standard. But it is effortful, so using\nAI to validate AI is tempting. It is less work, and in all fairness, we are\nalso fallible human beings, so sometimes our manually coded ground truth is\nwrong, and AI points this out. I think I spent more time validating code as I\ndid before I used AI, but validation efforts are more extensive and more\nsystematic, and they often occur at a higher level. I can feel a greater\ndistance between the code and data and my validation work, towards an\nevaluation of the output of an entire piece of code.
Still,\nwhere needed, I can easily jump back in, and look at the dataframes my code\ngenerates, to see if the contents match my expectations. This requires coding\nskills, so AI can't replace learning to code (even if it might change how we\nlearn to code). For most of what I am building, AI seems to be doing very well,\nso this is increasingly less necessary \u2013 or maybe I am overtrusting my AI, and\nI think it is not necessary.
I spent a\nhuge amount of time validating code (as I spend less time coding, I spend more\ntime validating, and for many of the modules in Metacheck that I have created I\nspend weeks (often hundreds of hours) validating that they work, and\niteratively improving them. For this reason, I don't like the term\n'vibe-coding'. I feel this is a pejorative term that is used when people throw\ntogether code in a short amount of time without knowing what their code does,\nand whether it works. This is not what I do when I code with AI.
When I\nstarted coding, I did everything manually. Then I moved to copying code from\nstack overflow, then to copy-pasting code from ChatGPT, and now I ask Claude to\ncreate the code. This is an improvement over asking ChatGPT (and if you have\nnot used AI inside your programming software, you don't really known how coding\nwith AI works and can't judge it). Because Claude has access to the code, and\nthe results, it can check the input, code, and output. If you have a good\ndefault prompt (e.g., you tell it to always check everything, never make\nassumptions, etc.) the quality of the code is hundreds of times better (and\nreally not comparable) to asking ChatGPT 'how do I create a dataframe with data\norganized like I want'.
I also\nthink there is a lot to learn about how to set up a workflow where I am even\nmore likely to notice when AI created code that is not in line with what I\nintend to create. I am just learning how to use these tools \u2013 of course in a\ndecade I will be much better at it. When I learned to share my data openly, I\nlinked to a dropbox folder in my scientific articles. Of course these links are\ndead now, and this was really bad practice. I am sure I am making similar\nmistakes when I use AI. But the mistakes might already be smaller than when I\nfigured out new technology (like data sharing) myself.
When there\nis no ground truth, I would be much more careful in using AI. I also use AI to\nwrite reproducible manuscripts where I do not know what the results should be. In\nthese cases, I don't know if a number should be high, or low. I might still ask\nAI to create a first version of the calculation of the descriptive report. But\nthen I have to double check every number, and here, I often intended to compute\nother numbers than AI thinks I wanted to generate. It is still helpful, and the\ncorrections are less work than writing all reproducible analysis code from\nscratch, but the required oversight is much larger, and I really need to look\nat the raw data and code to make sure everything is accurate. Here, I will\nstill use AI to ask how to compute certain output, then I check the code, I\ncorrect it where needed, but then I would also ask AI to check my corrections.\nAfter all, I also make mistakes, and AI can help to catch these more quickly.
So, there\nis a lot of contextual variation in how much I use AI, and what role it plays\nin the work I am doing. I don't spend less time checking my work \u2013 if anything,\nI spend more time on it \u2013 but the nature of the checks have moved to a slightly\nhigher level, often more away from the data and individual code lines, to the\nresults that the code should produce. This is actually not that different from\nwhen I used to code simulations manually. I would also evaluate the results on\na higher level (the results of the simulation) and only jump into the\ndataframes and individual lines if I expected that there was something wrong.\nAt the same time, those simulations were built in smaller steps, and the\nindividual sections of code would receive at least a basic check. Now, these\nlower level checks are still present, but instead of manual checks, I often ask\nAI to write explicit tests (e.g., in this dataframe, column 1 should be a\nnumber between 0 and 1). This is on the one hand a better practice, but\nsometimes also comes closer to letting AI check AI. It's a space where I still\nneed to explore how to create a reliable and efficient workflow.
\nAs\nenthusiastic I am about using AI for code, as disappointing I am in using AI to\nwrite. Maybe it's because I have a lot of experience in writing, but I find AI\nwritten text mostly mediocre. This also holds for Peer Review. The only use I find\nin AI for peer review is (beyond maybe using Claude to automatically reproduce\nyour results) is to remove the Dutch Directness that can creep in some reviews.\n
There are\nsome exceptions. Sometimes AI can improve parts of text I write. I write too\nlong sentences, and sometimes as a non-native English speaker I create weirdly\nformatted sentence constructions, I don't know the right words, or the thoughts\njust appear on paper somewhat unclear. Asking AI to suggest an alternative\nsometimes (but not always) provides inspiration for how to improve the\nsentence.
Another\nexception is low-stakes writing. Sometimes as academics we need to create text\nthat will not be read (or not be read by many people), where we would otherwise\ninvest so little effort that our writing would not be better than what AI\nproduces. Of course, you could also say 'why not reduce bureaucracy in\ninstitutions'? I agree. With many things, AI use amplifies underlying problems.\nIf people use AI to write, maybe what they are writing does not need to exist.\nIf AI increases our reflection on this topic, that is a good thing.
Another\nexception is documentation. If I want to summarize the code in a repository, AI\nwill do so more diligently that I can. Similarly, the Metacheck Manual has large sections that are written\nby AI (although I also rewrote a lot of sections) because it summarizes what\ncode does, and AI can do this relatively well. Ideally I would write every line\nmyself, but I don't have the time, and the entire team is busy making software.\nSo, AI makes documenting things less work, and although it is certainly not as\ngood as what a human would produce, given our priorities and limited resources,\nit suffices. If you prefer manually written documentation, as I do, and you\nhave time, it would be a great way to help with Metacheck.
\nI\npersonally like the ability to automate boring tasks (e.g., asking AI to\nprepare a meeting, copy files from my computer to a harddrive, cleaning up\nmessy folders, look at which of 10 meetings dates is free in my calendar, etc.).\nI also like the voice interface of Claude. Instead of reading Wikipedia (which\nI often do), I can ask Claude questions while am walking around. The answers\nlargely come from public sources like Wikipedia, but I can more quickly switch\nand develop my thoughts about a topic I don't know a lot about. I don't like\nthe use of AI to generate pictures, videos, or presentations. I find them bland\nand uninteresting. But, this drifts away from the specific use of AI as a\nscientist, and this blog is long enough as it is, so I will not go into details\nabout other uses of AI too much.
\nI am regularly\nreflecting on the ethical aspects of using AI. I am not an expert on this, so I\ncan't say sensible things. I hope there will soon be a European Union version\nof Claude for scientists, and I will switch over to one if it exists. I prefer to\nspend my money on non-commercial and open access tools. I am hopeful this will\nsoon become possible. But in the meantime, I think Anthropic is not that\ndifferent from my bank. There are a lot of values I disagree with, but the\nbenefits of having a bank outweigh the problems I have with banks in general. I\nam supportive of the general policy in the EU around the use and development of\nAI, so I hope that in the future I will be happier with the organizations that\nprovide me access to AI than I am with banks. I do not have a drivers license,\nbut if you have, you should probably be more upset about the companies behind\nthe oil you buy than about the company that provides your AI inference, in my\nhumble opinion.
Then there\nis the climate. Data servers use a lot of energy. Here, the question is how\nmuch energy we want to use, and what we want to use it on. I think people using\nAI to generate or adjust pictures and videos should stop using AI. I don't see\nany societal benefit, and it uses quite some energy. I also think we should\nreduce our reliance on data servers where possible. I personally have stopped automatically\nsyncing everything to the cloud. For example, I maintain a local back-up of my\npictures, but I no longer store them in a datacenter. If we want to\u00a0reduce the use of datacenters, I would\npersonally prioritize a reduced use of video streaming. I feel services like\nInstagram reels waste everyone's time and are overly addictive, and I would not\nmind it if people who want to use them need to pay the environmental cost\n(compared to the current free use) of streaming, and I think the world is\nbetter off if these services do not exist. I also think we should stop\nstreaming video with services such as Netflix. As someone who likes vinyl, I\nthink we should own the media we consume. If we want to rely less on\ndatacenters, I would prefer a societal movement away from all streaming\nservices such as Netflix. As video streaming takes up a much larger percentage of the use\nof datacenters,\nI think for my own personal life, I prefer to use energy more on AI, and less\non things like video streaming.
\nThis blog\nis mainly an attempt to make my current behavior and beliefs more explicit, so\nthat they become easier to criticize. I am sure I am getting things wrong, I am\nmissing things I should weigh more, I have not discussed all important aspects,\nand in 10 years I will read this and know better. But I at least feel prepared\nto go into our lab meeting next Monday, and spend some time with the\nold-fashioned use of a keyboard to organize my thoughts. (Yes, this 100% human\nwriting but it would be funny if you thought otherwise).
There's a funny thing that happens to me from time to time. I randomly stumble across an essay or a paper that resonates with me, sometimes one that I can't stop thinking about for days. Then I come to find out the author is a colleague here at my own University of Virginia in the building next door who I never knew of.
is a professor of psychology here at UVA (I don't know him, but hope to meet you soon, Dan, and compare notes). I just came across an essay Dan published back in August. It's good, and you should take the <5 minutes to read it in full. Even if you're not an educator, you probably still have junior folks working for you or may be doing some on-the-job training, and Dan's points are still relevant, so substitute whatever you want for \"students\" in your case.
His argument is that students should only use AI for the things they already know how to do well. Not brainstorm-with-AI-but-draft-yourself vibes, and not summarize-with-AI-but-write-the-essay-or-whatever-yourself vibes, which are what a lot of AI \"policies\" look like in higher ed, and even K-12. E.g., here are the policies from our own Albemarle County Public Schools around us here in Charlottesville Virginia:
![\"\"](\"https://substackcdn.com/image/fetch/$s_!lH-e!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff254a632-4d6b-4a48-97f1-03af66430266_1343x1064.png\")
It wasn't Dan's conclusion that stuck with me the most. It was the way he set up the argument by contrasting the objectives of the workplace versus education.
In the workplace, the product is the thing. The report, the grant proposal, the manuscript, the R package, the Python notebook, the pitch deck, whatever. The product is the thing that matters, and here we all agree to this ideal that the human should be the lead in the driver's seat, with AI being the associate.1
In education, the product doesn't matter. The product isn't the point at all.
Students produce products, but the product is seldom the point. No one wants to read the papers they write, and the math problems they solve have no practical value. Teachers have them do these tasks because they provide practice in cognitive abilities that we think matter.
The the R/Bioconductor exercises or the journal club-like essays from landmark genomics papers I assign in my class have zero value to the outside world. They exist purely so that a student practices integrating evidence, thinking about how to tackle a problem with code (even if they'll write that code later with an AI like I do), how to build up a persuasive argument.2
Dan's argument isn't a total ban on AI in the classroom, but to only allow AI for things that the students already know well. He draws an analogy to calculators.
Once you've mastered arithmetic operations, there's no benefit to hand calculations when you're solving an algebra problem. But if you're still learning algebra, don't use the algebraic functions.
Dan goes on to steelman a few objections toward the end of the essay. I recommend reading it in full, and I'd wager after you do you'll agree with his conclusion:
Students should only use AI for things that they already know how to do well.
Further reading:
Although please, I beg you, be respectful to yourself and to your colleagues, and don't lower your intellectual value in your workplace to zero by simply being the meat proxy between me and an AI that I'm perfectly capable of interrogating myself. Your colleagues, bosses, direct reports, and students can all smell the Claudish in the parts-per-trillion from a mile away, and no one wants to spend an hour reading something you spent less than one minute \"writing.\" I'm not innocent here, but I'm trying to do better.
The vast majority of what a senior/principal scientist does in industry and academia is persuasion, sometimes in the form of persuading others to believe that our results are real and meaningful, but most often in the form of begging for money to support research and trainees.
I'll be radically honest here - I never really got the idea of \"Test Driven Development\". I'm much more of a \"telnet into production and damn the consequences\" kind of guy. Look, it's much more fun and my cardiologist recommends severe shocks every now and again.
\n\nThe essence of unit testing is pretty simple:
\n\nSend it good data and if it returns \"true\" the test passes. Send it bad data and make sure it returns false. Easy peasy lemon squeezy.
\n\nAs part of my NLnet funded work on ActivityBot, I'm writing some proper tests for my bot framework. I am hopeful that some of them will be useful to other projects doing similar things.
\n\nThere are four major classes of tests - here are the problems I've found while writing them.
\n\nA basic test might be how to validate that an ActivityPub username is valid. Typically it would be something like @edent@mastodon.social - what assumptions do we make with that? It starts with an @ and has another @ somewhere in it, dividing into two parts, the user and the server. The server has to be a valid domain name - but that can include non-ASCII Internationalised Domain Names and Emoji domains.
What about the user part? Is it just A-Z and 0-9? Can it have dots? What about apostrophes like an email address?
\n\nBasically, how easy is it find the specifications for all the various component parts of ActivityPub? Not very!
\n\nWhat does a WebFinger response look like? Which parts are mandatory and what values can they contain?
\n\nWith any message that your project generates, how can you be sure that other ActivityPub servers will understand it?
\n\nSimilarly, when we receive a follow message from an external server, what should it look like?
\n\nThere are some JSON Schemas which can be used to validate some type of messages.
\n\nBut, again, there are many scattered specifications.
\n\nAs I've written about before, HTTP Signatures are difficult. There are two main problems. The first is verifying signatures.
\n\nIn order to do that, you need to gather a bunch of signatures and then find a way to replay them into the test. So I'm in the process of saving signatures which I can then share with others.
\n\nThe second problem is generating signatures. Sure, you can run tests on the signatures - but there's only one real way to test whether they're acceptable\u2026
\n\nPassing tests means nothing if you can't communicate with others and they can't communicate with you.
\n\nSome tests are easier than others. Sending a signed request should be easy and repeatable. But receiving messages isn't easily automated. It requires being able to control an external server and asking that to send messages.
\n\nWith the Mastodon API it is possible to make requests which will interact with your server - but only if it is a public server.
\n\nI'm putting all my tests online. I'm also sharing a collection of messages and signatures which you can use in your tests.
\n\nI'd love to know if you find them useful.
![](\"https://shkspr.mobi/blog/wp-content/themes/edent-wordpress-theme/info/okgo.php?ID=75774&HTTP_REFERER=DOI\"){kind=small}
","doi":"https://doi.org/10.59350/nmxzr-19z59","guid":"https://shkspr.mobi/blog/?p=75774","image":"https://shkspr.mobi/blog/wp-content/uploads/2012/09/709ae69b.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1791590400,"rid":"s1wdt-2k111","summary":"I'll be radically honest here - I never really got the idea of \"Test Driven Development\". I'm much more of a \"telnet into production and damn the consequences\" kind of guy. Look, it's much more fun and my cardiologist recommends severe shocks every now and again.","tags":["/etc/","ActivityBot","ActivityPub","Php"],"title":"Some quick thoughts on Unit Testing ActivityPub","updated_at":1791703811,"url":"https://shkspr.mobi/blog/2026/10/some-thoughts-on-unit-testing-activitypub/","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Bekkers","given":"Rene"}],"blog":{"authors":null,"community_id":"c04223e6-4d3d-42ad-a718-878a8fc35d32","created":1739059200,"current_feed_url":null,"description":"Rene Bekkers","doi":"https://doi.org/10.59350/renebekkers","favicon":"https://rogue-scholar.org/api/communities/c04223e6-4d3d-42ad-a718-878a8fc35d32/logo","feed_format":"application/atom+xml","feed_url":"https://renebekkers.wordpress.com/feed/atom","filter":null,"generator":"WordPress.com","home_page_url":"https://renebekkers.wordpress.com","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"renebekkers","status":"active","subfield":"1404","title":"Rene Bekkers","updated":1791665072,"use_api":true},"blog_name":"Rene Bekkers","blog_slug":"renebekkers","content_html":"There seems to be s a lot of enthusiasm for AI powered tools for research these days, if I look at the feeds that the tech industry controls. No doubt that these tools greatly reduce the effort for a literature review that sounds plausible. It’s great to get a nice set of studies as a starting point for a new project in a new research area. But in these cases we usually don’t know what we don’t see. So here’s a cautionary tale from a rare case where we do know the errors of omission in AI powered tools.
\n\n\n\nThe challenge I gave the AI-powered tools is difficult: I asked them to identify replications of previous experiments in the multidisciplinary field of research on charitable giving. Such replications are needles in a field of haystacks. But we found them by hand, with a multidisciplinary team of six scholars in a study on replications of research on charitable giving. We conducted a comprehensive manual search on Google Scholar and identified 48 replications of experiments. How many replications from the set of 48 can AI-powered tools identify?
\n\n\n\nNot many, it turns out. The best tool, Google Scholar Labs, found one third of them. Most tools – even popular ones such as Perplexity, find fewer than 10. LeapSpace, the tool that Elsevier is trying to sell to university libraries in the Netherlands, performed abysmally. Collectively the twelve tools I assessed identified exactly half of all replications that we had found by hand. But it gets worse. Many studies that the AI tools identified as replications were not replications at all. Prompting more elaborately with explanations and examples did not make the results better.
\n\n\n\n![\"\"](\"https://renebekkers.wordpress.com/wp-content/uploads/2026/10/12aitools.png\")
The news is not all bad. The tools also identified a number of new studies that we had not found because we had limited our search to keywords that returned a feasible number of results. The newly found replications used other terms than we had looked for. Another upside: at least the searches came back quickly. The slowest tool, Claude Science, took 35 minutes. The quickest tool returned results within a few seconds. The result: it could not find any of the replications.
\n\n\n\nThe results underscore the necessity of human review of results from AI-powered tools. At face value, the reports produced by the tools seem entirely plausible. However, human review of the results showed a high number of false positives, up to 70% of all results for LeapSpace. Conclusions based on the searches of the AI-powered tools are incomplete and incorrect. Without an accurate and efficient method to identify false positives the speed with which the tools concluded their work gives users the false impression that they can outsource identifying relevant previous work to AI-powered tools.
\n\n\n\nRead the full paper including the prompts and supplementary materials with all details and a link to the data and code here.
\n\n\n\n","doi":"https://doi.org/10.59350/gvwch-yhj37","guid":"https://renebekkers.wordpress.com/?p=4290","image":"https://renebekkers.wordpress.com/wp-content/uploads/2026/10/12aitools.png?w=975","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1791590400,"rid":"wb9gx-88f29","summary":"There seems to be s a lot of enthusiasm for AI powered tools for research these days, if I look at the feeds that the tech industry controls. No doubt that these tools greatly reduce the effort for a literature review that sounds plausible.","tags":["AI","Data","Experiments","Household Giving","Meta Science"],"title":"Fast, but incomplete and inaccurate: twelve AI-powered tools for literature search","updated_at":1791703809,"url":"https://renebekkers.wordpress.com/2026/10/10/fast-but-incomplete-and-inaccurate-twelve-ai-powered-tools-for-literature-search/","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/050qmg959","name":"Singapore Management University"}],"contributor_roles":[],"family":"Tay","given":"Chee Hsien, Aaron","url":"https://orcid.org/0000-0003-0159-013X"}],"blog":{"authors":null,"community_id":"f34e2211-9904-4b58-97ab-0beeb79ef6f7","created":1697068800,"current_feed_url":null,"description":"Aaron Tay's thoughts about academic librarianship","doi":"https://doi.org/10.59350/musings","favicon":"https://rogue-scholar.org/api/communities/f34e2211-9904-4b58-97ab-0beeb79ef6f7/logo","feed_format":"application/rss+xml","feed_url":"https://aarontay.substack.com/feed","filter":null,"generator":"Substack","home_page_url":"https://aarontay.substack.com","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"musings","status":"active","subfield":"3309","title":"Aaron Tay's Musings about Librarianship","updated":1791665977,"use_api":true},"blog_name":"Aaron Tay's Musings about Librarianship","blog_slug":"musings","content_html":"![\"\"](\"https://substackcdn.com/image/fetch/$s_!qJ8a!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa3ae9690-4868-49a3-aea3-98f9d25f605f_925x643.png\")
I think I may have discovered the \"evil conspiracy\" behind why academic search vendors are so reluctant to show us what happens on the semantic side of AI search.
Recently, I have been wondering about a strange asymmetry. I noticed that when academic search vendors use AI to expand lexical searches, they are almost always happy to show you what they did.
For example, Web of Science Smart Search will happily show you the Boolean search it used. The same is true of products such as EBSCO AI-assisted Search, scite Assistant, Primo Natural Language Search and many others.
Then you get to the semantic-search part, and suddenly the curtain comes down.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!zkhL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F562e3e05-a286-4ab6-a7e2-d8a79f30012f_705x561.png\")
Take Web of Science Smart Search again. We are told it runs a hybrid search, meaning it runs both lexical and semantic searches in parallel, combines the results and then somehow ranks the combined list.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!TwCy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Faa7d058b-a740-4939-ac63-bfab6519a58a_1566x754.png\")
Yet we know almost nothing about what happens in the semantic-search portion, beyond the fact that it was done.
What exactly was searched? Did they expand your query before embedding it? Why all the secrecy?
Similarly, Scopus AI has a copilot that decides \"whether to run a vector search (aka Semantic Search) and/or a keyword search\". Yet again, all the interface displays is the expanded Boolean Search.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!9XSK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6c8875ff-317c-4f22-9396-ade5a1c4cf3f_720x405.png\")
Recently, while studying a new AI search system, I accidentally got a look backstage.
And I discovered a shocking \"truth\". The system was generating document-like passages for semantic retrieval rather than simply embedding my original query.
They were not just synonyms or alternative search terms. The generated passages could introduce assumptions that were never part of my question, or even contain outright false statements.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!L_vU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d9bfe88-1242-4bbd-b28f-816b985835fb_1094x633.png\")
Imagine entering the following query:
Can Google Scholar be used alone for systematic reviews?
And finding out that, behind your back, the AI generates the following for expansion:
Google Scholar is not sufficient as a standalone source for systematic reviews because studies have shown that while it indexes almost all relevant research, it is difficult to retrieve all the relevant papers within the top 1000 results it allows access to.
Hold on a moment. I never said Google Scholar was not sufficient! That's precisely what I'm trying to find out! Why is the system apparently answering my question before it has even searched?
Now the reason for the \"conspiracy\" seemed obvious.
Vendors don't want us to find out that semantic search is being fed hallucinated AI-generated query strings!
Okay, okay, I was just being facetious here. There is NO evil conspiracy.
This is not to say that systems never do something similar to what I have described. They may generate text resembling relevant documents to help with retrieval, but that text could introduce assumptions that do not reflect the user's information need or even contain outright falsehoods. Such additions could potentially introduce bias if used in a traditional search.
But what I have described is not necessarily problematic. In fact, it resembles established techniques such as Query2doc and HyDE for expanding semantic searches.
While it may seem strange, the fact is:
A generated passage that is not entirely accurate or true can still be useful, perhaps even more useful than the original query, when embedded for semantic retrieval.
Confused? Let me start from the beginning.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!1BmU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd66bc5ce-4c44-468d-aed5-e3e4e70c7ebd_1076x612.png\")
The system itself was fairly typical of the current generation of AI-powered academic search tools.
There was a query interpretation step, probably involving an LLM. The system identified things such as entities that could be turned into filters, then ran a hybrid search combining lexical and semantic retrieval.
The results were merged and deduplicated, and an LLM was then used to classify them (by prompting) into relevance categories such as \"Very relevant\" and \"Relevant\".
None of this is especially unusual, and you can see parts of this process in many academic search systems. For example, using LLMs to extract relevance criteria and then classify results is something we see in AI2's Asta Find paper. But let's focus on the even more common idea of hybrid search.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!MtRG!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3433d222-7ba9-4526-bfc8-e1254ebb930f_1519x802.png\")
Hybrid search is becoming increasingly popular in the search industry. Typically, it involves running both lexical or keyword searches and semantic searches.
The logic is straightforward. Lexical search is good at matching words and phrases. Semantic search can retrieve documents that express similar ideas using different language.
This potentially gives you the best of both worlds. You just need a way to combine both sets of candidate results and rank or rerank them somehow.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!-KRe!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F71669c7b-13ad-479f-9a6f-11104b1c7ad7_917x615.png\")
So far, so good. But let's look at the interface (which I have mocked up) that was shown to me.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!BvEW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3bfc19b-99a4-4f22-8c5d-6a479002f98f_1672x941.png\")
When it comes to lexical search, we are shown quite a lot. We can see that it runs the original query and two expanded, nested Boolean search strategies (probably generated by an LLM).
But on the semantic side, we are told almost nothing, except that two searches were conducted.
Why this distinction?
When I was interviewed by the vendor, I naturally asked for more details about what was happening on the semantic search side.
Intriguingly, the vendor told me that an earlier version of the product had exposed more of the internal search process. However, users found it confusing, so some of the details were removed.
My reaction at the time was predictable. I did not understand this view at all. Surely more transparency would be better?
Perhaps they could even add an advanced mode for those who wanted more information.
But then I discovered some of the details that might have been hidden, and I began to empathise with why the vendor had chosen not to show everything.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!haIR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9d07efad-32d8-423c-b4ae-35670f55083c_1094x614.png\")
While testing the reproducibility of the search, I ran into a completely different problem. No matter what I did, the system seemed determined to cache the results.
While troubleshooting this, on the advice of ChatGPT, I started recording the browser's network traffic using Chrome DevTools. We will revisit this in a future post.
This produces a HAR, or HTTP Archive, file: essentially a log of the requests and responses exchanged between your browser and the servers behind the application.
As a side effect, this exposed considerably more of the search pipeline than the interface itself did.
The logs confirmed that the system was running:
My original lexical query;
Two additional nested Boolean searches;
Two semantic searches.
There were also hints on many details. For example, about how many of the top results from each search were retained and how they were combined. But for now, let's focus on semantic search.
While there was very little information about the actual embedding model used, there was something more interesting.
I could see the actual text being used for semantic retrieval, and it was definitely not my original query.
Roughly speaking, the system appeared to be doing the following:
Original query + an LLM-generated short description of what a relevant document might look like
In short, it appeared to be taking my query and appending a chunk of LLM-generated text before creating the query embedding.
Those with some training or knowledge of information retrieval will recognise this as resembling a family of techniques such as Query2doc-style expansion or HyDE (Hypothetical Document Embeddings).
I spend quite a bit of time testing AI-powered academic search tools and trying to work out what is really happening behind their interfaces. If you find this kind of independent investigation useful, please consider buying me a coffee on Ko-fi. It helps support more investigations like this, which I share freely on this blog.
If you are already familiar with how embeddings and dense retrieval work, feel free to skip this section.
As I have noted many times on my blog, \"semantic search\" is not a single technique. It is better understood as an objective: retrieving or ranking items according to their estimated meaning.
It is typically contrasted with lexical or keyword search, where retrieval and ranking depend primarily on matches between terms in the query and terms in the documents.
The techniques used to achieve semantic search have changed over time. In recent years, semantic search has often, though not always, referred to dense retrieval or dense vector search.
![\"placeholder](\"https://substackcdn.com/image/fetch/$s_!0Bw7!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc74706b6-ec48-4ba6-9766-8ab2d78a023d_880x585.png\" "\\"placeholder")
In a typical dense retrieval system, the query is passed through an encoder model, often based on the Transformer architecture, to produce an embedding: a vector representation (a long series of numbers) intended to capture useful aspects of the query's meaning1.
Documents, passages or text chunks are encoded in a similar way, usually in advance during indexing. The query embedding can then be compared with these precomputed document embeddings.
Both query and document embeddings are represented as vectors: long sequences of numbers. The system calculates a similarity score between them, commonly using measures such as cosine similarity or dot product. Documents whose vectors are closer to the query vector are treated as better candidates for relevance.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!LWWA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd5386548-ab18-4efb-84e0-a5afa947f88b_1448x1086.png\")
Important to note: the query and document do not necessarily need to contain the same terms to be considered similar. Rather, the encoder is able to convert texts that are similar in \"meaning\" into representations that end up close together in vector space.
Conceptually, for dense retrieval, what we are doing is this:
Query \u2192 embedding \u2192 find nearby document embeddings
Semantic search via embeddings is inherently something of a black box
While a Boolean search like
(\"artificial intelligence\" OR \"machine learning\") AND libraries
allows you to inspect the search strategy to understand why some documents are retrieved and others are not. Embedding retrieval, however, is much less transparent.
Firstly, it is extremely difficult to interpret the long series of numbers representing each query or document. Secondly, even if I gave you the calculated similarity score between a query embedding and a document embedding, that would still not provide a satisfying, human-readable explanation of why they are close.
I have long expected this to be unavoidable.2 But surely vendors can at least tell us what text they embedded?
So why don't we just use the query as entered? While this can work in practice, one approach that can improve retrieval is to do the following:
Use an LLM to generate a document, or at least part of one, that might be relevant.
Use that generated text to help construct the query representation for retrieval.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!fiVe!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1c2a5b30-e3f2-45ac-bef1-6753c2a7a4b9_1672x941.png\" "\\"\\"")
The generated pseudo-document or hypothetical document is then used to help with retrieval.Query2doc appends the generated pseudo-document to the original query before passing the combined text through an encoder to create the query embedding3. HyDE directly generates the embeddings of the generated hypothetical documents (without the query) and in some variants will average that embedding with that of the original query embedding.
Another related technique is RAG-Fusion, used by Elsevier's Scopus AI. Rather than generating hypothetical documents as HyDE and Query2doc do, it generates multiple variations of the original query, runs separate vector searches and combines the results using Reciprocal Rank Fusion. Like HyDE and Query2doc, it can potentially introduce assumptions not present in the original query.
Empirically, this can work better than using the original query alone. Why?
One way of thinking about it is that the query itself is typically quite different, in terms of expression, length and so on, from a relevant document containing the answer. The logic, then, is that by generating a document that might be relevant, its embedding may end up closer to the embeddings of the desired documents than the original query embedding would.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!MwrY!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F814d7f41-1966-490e-8dc1-d22384c48ab9_1448x1086.png\")
Imagine your query is:
Can you use Google Scholar alone for systematic reviews?
And you get a fairly neutral hypothetical passage explaining what systematic reviews are, why comprehensive searching matters, and so on.
I think most of us would be okay with that.
But what if it generates something like this?
Google Scholar should not be used alone for systematic reviews because it only displays approximately 1,000 search results at most and lacks the advanced search functionality needed to retrieve all relevant studies.
This is, to my knowledge, true. But would using this passage for retrieval introduce bias into the results?
Or worse, what if it uses this?
Google Scholar can be used alone for systematic reviews because Tay(2024) has shown that more than 99% of papers eventually included in a SR can be found in Google Scholar's index.
For those unfamiliar with the literature, there is no such Tay (2024) paper, although real studies have reported very high coverage of studies included in systematic reviews within Google Scholar.
That said, the problem is that being indexed is not the same as being retrievable through a search. Google Scholar restricts access to roughly the first 1,000 results, so even if most relevant papers are in its index, there is no guarantee that a search will surface them.
Surely that last hypothetical passage would produce horrible results?
Not necessarily.
Remember, the system is not necessarily treating the generated text passage as a factual answer. It is using the text to construct a representation for retrieval.
Even though the passage contains a fabricated citation and a potentially misleading conclusion, it also contains concepts highly relevant to the original question: Google Scholar, systematic reviews, database coverage, included studies and comprehensiveness.
It is entirely possible that embedding this passage would retrieve useful papers discussing Google Scholar's suitability for systematic reviews, including papers that argue against using it alone.
But what about bias? Could generating a hypothetical document arguing that Google Scholar cannot be used alone for systematic reviews favour papers reaching that conclusion over those arguing the opposite?
![\"\"](\"https://substackcdn.com/image/fetch/$s_!xwOP!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0cba71f0-2374-444a-8e78-cc05ccdaefa1_1448x1086.png\")
Again, not necessarily. Some research has found that neural retrievers, particularly bi-encoders, can be surprisingly poor at distinguishing between statements that differ only in negation. This suggests that simply reversing a claim might have less influence on retrieval than we would intuitively expect.
Still, this does not mean there is no risk of bias. Our hypothetical passages differ in more than just their conclusions. They also emphasise different concepts, which might influence which documents are retrieved and how they are ranked.
Whether that actually happens is something we would need to test empirically. We cannot tell simply by inspecting the generated text.
In fact, it is conceivable that a hypothetical passage containing entirely correct statements could produce worse retrieval results than one containing factual errors. After all, factual accuracy and usefulness for retrieval are not the same thing4. What matters is whether the generated passage helps the system retrieve relevant documents, although factual errors could certainly make that harder.
There is an old search tip that sounds superficially similar to what these systems are doing.
The advice goes something like this: when searching Google, think about what words or phrases a webpage answering your question might contain, then search using those terms.
Suppose I want to know whether Google Scholar can be used alone for systematic reviews.
I might reason that relevant papers would contain terms such as coverage, recall, comprehensiveness, included studies and search limitations, then search using those terms.
This makes sense and has a clear resemblance to what HyDE is doing.
In both cases, we are trying to anticipate what relevant documents might look like and search accordingly5?
![\"\"](\"https://substackcdn.com/image/fetch/$s_!idM2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F47b1ccf7-85b5-4b28-be67-584e85b79e36_1448x1086.png\")
There is, however, an important distinction here.
With the Google trick, I am predicting words or phrases that I expect to literally occur in relevant documents. If I guess incorrectly, I may damage the search, which is one reason why using LLMs to generate nested Boolean searches does not always produce the best results6.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!E1o8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7f4cd92f-3899-475f-99ea-de058a43c919_1672x941.png\")
With HyDE, even if the generated document is factually wrong, its embedding can still be useful if it ends up closer to the embeddings of relevant documents than the original query embedding would.
The hope is that this richer, document-like representation lands somewhere near real documents discussing those concepts.
The fact that a generated passage reaches the wrong conclusion about Google Scholar does not necessarily prevent that from happening.
Of course, I am not saying that a hallucinated passage cannot push the query embedding further away from relevant documents.
In fact, some literature suggests that such expansion techniques tend to help weaker retrievers but may hurt stronger ones. They may also work less well in areas where the LLM's knowledge is limited. As always in information retrieval, whether a technique improves performance depends on factors such as the retriever, the types of queries and the dataset being searched. Ultimately, these are empirical questions that should be settled through testing (which hopefully the vendor has done) rather than decided a priori.
The point is simply that spotting a hallucination does not, by itself, tell us how serious the problem is.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!J4mU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ab2b2a3-99f6-4db2-8fe7-ec94f8137cc1_1448x1086.png\")
This brings us back to my supposed conspiracy.
Imagine the interface shows you:
\"Google Scholar\" AND (\"systematic review\" OR \"evidence synthesis\") AND (coverage OR recall OR limitations)
A librarian might question why certain terms were included, but they would still recognise what they were looking at and feel comfortable with it.
Now imagine the interface instead shows:
Google Scholar can be used alone for systematic reviews because Tay (2024) demonstrated that more than 99% of studies eventually included in systematic reviews can be found in Google Scholar's index.
I suspect the reaction would be much stronger.
Why is the system citing a paper that doesn't exist? Why has it apparently decided that Google Scholar is sufficient? Is it now biased towards papers supporting that conclusion?
These are very reasonable questions if you interpret the passage as an answer or a literal statement of the search criteria.
But in the world of semantic embeddings, it is not necessarily either. It may simply be an intermediate representation used to create an embedding.
Perhaps part of the problem is that what happens inside modern semantic search no longer maps neatly onto what librarians traditionally think of as a query.
Modern AI search really does expand your query.
The surprising part is that making things up isn't necessarily the problem.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!JYE_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97e4bb7c-1660-4c53-aa40-942acf73f813_1448x1086.png\")
And this, I think, explains at least part of the strange asymmetry I keep seeing in AI search interfaces. Even if vendors wanted to show such details, many users, including librarians, might find them confusing7.
The more details vendors show about the search process, the greater the chance that they will open a can of worms by raising questions and concerns among users and librarians. From the vendor's perspective, the safer option may be not to show them.
The other part of the issue is that modern information retrieval is largely driven by empirical experimentation. You can propose a perfectly logical change that should improve retrieval, but if the experiments show that it doesn't work, you shouldn't use it. The reverse is also true. Some techniques may seem counterintuitive to librarians but work well empirically8.
While I can understand how vendors might feel, I still don't think they should hide all the details. We have a legitimate reason to want to know what is happening under the hood.
For librarians, the practical implication is this: resist the instinct to judge the semantic side of AI search by the same standards as Boolean. Spotting a hallucination in a query expansion is not evidence that a system is broken. The more useful question to ask is not \"does the generated query look right?\" but \"does the system consistently surface relevant results?\" That requires empirical testing against known relevant papers, not query inspection.
For librarians supporting systematic reviews the stakes are higher of course given the higher standards of transparency and reproducibility required.
What do you think?
Some might argue that showing the expanded query used to generate the query embedding is not particularly important. But the question applies more broadly to all aspects of the search pipeline.
Would you like vendors to show every detail, even if you don't fully understand it or think some of the techniques are strange?
If you found this essay on what AI-powered academic search tools are really doing behind the scenes educational, please consider supporting my work.
How is the encoder model able to generate representations that capture something resembling \"meaning\"? Encoder models learn useful language representations during pretraining on large text corpora, then are typically fine-tuned for retrieval so that relevant query-document pairs receive more similar vector representations than irrelevant ones.
You can get more transparency using multi-vector or late-interaction methods such as ColBERT, or learnt sparse representation methods such as SPLADE, but these are currently not commonly used.
Both techniques use very similar prompts to generate documents that Query2doc calls pseudo-documents and HyDE calls hypothetical documents. In practice, there is little difference between the generated texts themselves. The main distinction is that Query2doc was designed as a general query-expansion method. The query and expanded text can be used for lexical matching as well as retrieval via embeddings. HyDE was designed specifically for embedding-based retrieval.
Factual accuracy and retrieval effectiveness are different properties, but that does not mean they are statistically independent or even negatively correlated! For example, Yoon et al. (2025) found that hypothetical-document expansion improved retrieval mainly when generated passages contained information supported by the actual evidence, raising concerns that some reported gains might reflect knowledge leakage rather than better retrieval.
Notice this technique also risks bias!
I have long argued that using LLMs to generate Boolean is not very productive mostly because they tend to be poor at generating Boolean search strategies and even if they did do it decently it does not help average searchers who already know how to do so. A recent paper by San Diego State University documented how difficult it was to get the LLM to generate reasonable boolean just with prompt engineering.
I can imagine librarians or researchers unfamiliar with information retrieval becoming upset when they see what is happening and assuming that the search is broken.
Of course, it is important that a technique works generally and not just on limited test sets.
I have written a lot of notes that I never saw again. Not because I deleted them. They were still somewhere on my computer, carefully saved inside a folder with a sensible name. The problem was remembering that they existed when I actually needed them. That is the strange thing about most note-taking systems. We spend a lot of time thinking about how to store information and surprisingly little time thinking about how we will find our way back to it.
\nYou read something interesting. You write a note. You choose a folder. Maybe you add a tag. Then you move on. Months later, you are working on something related and vaguely remember, I know I wrote something about this before. So you search, open a handful of notes that are not quite right. Eventually, you either find it after far too much searching or give up and recreate the idea from scratch. The note is still there. The problem is that it just never found its way back into your thinking.
![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig0.png\")
Obsidian approaches note-taking differently. Instead of treating notes as isolated documents that need to be carefully filed away, Obsidian encourages you to think about how ideas relate to one another. Suppose I am reading about confounding and create a note about it. Somewhere in that note, I mention g-computation. I can simply wrap g-computation in double brackets: [[g-computation]]. Those double brackets create a link. If I already have a note called g-computation, clicking the link takes me there. If I do not, Obsidian lets me create one. More importantly, the relationship works in the other direction too. When I eventually open my g-computation note, I can see that the confounding note links back to it. As I keep reading and writing, these connections accumulate and gradually becomes a network of ideas.
There is another reason I became comfortable with Obsidian. Your notes are simply Markdown files stored in an ordinary folder on your computer. They are not locked inside a proprietary database that only one application understands. Open your Obsidian vault in Finder or File Explorer and your notes are right there as .md files. You can open them with another text editor, move them, back them up, or manage them however you want. If you stopped using Obsidian tomorrow, your notes would still be yours and would still be readable.
Obsidian is also free to use. The features that make the application powerful, including plugins, themes, graph view, linking, and multiple vaults, are available without paying for a subscription. There are paid services such as Obsidian Sync for syncing your vault across devices and Obsidian Publish for publishing notes to the web, but they are optional conveniences. Check obsidian.md/pricing for current numbers, since they change.
\nAnd that brings me to this guide. I use Obsidian mainly for academic and research work, so that is the perspective I will use throughout this guide. You will see examples involving papers, methods, concepts, research projects, and ideas I want to revisit later. But there is nothing inherently academic about the system. The same approach can work for work projects, meeting notes, books, personal writing, recipes, or almost anything else you want to keep and connect.
\nI also do not want this to become one of those Obsidian tutorials where we install twenty-seven plugins, create fourteen folders, design an elaborate tagging taxonomy, and spend three hours building a system for taking notes before we have actually taken a note. We are going to start small. We will install Obsidian, create a vault, understand the few pieces of Markdown you actually need, create our first linked notes, and develop a simple structure for organizing them. From there, we can add templates, daily notes, plugins, backups, and some of the more powerful features, but only when there is a reason for them.
\nDownload Obsidian from obsidian.md for macOS, Windows, Linux, iOS, or Android. There is no account to create: the app opens straight into a prompt asking what you want to do. Choose Create new vault, give it a name, and pick where it lives on disk.
![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig2.png\")
A vault is a folder. Inside it are .md text files and a hidden .obsidian/ subfolder holding your settings. That is the entire architecture. Open the folder in Finder or Explorer right now and you will see your notes sitting there as ordinary files.
Open your vault and you will see a mostly blank window. Here is what you are looking at.
\n![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig1.png\")
On the left, the file explorer lists every note in your vault; folders here are real folders on disk. In the middle, the editor is where you write. On the right, the graph view shows your notes as dots and their links as lines.
A handful of shortcuts cover most of what you will do:
\n| Shortcut | \nWhat it does | \n
|---|---|
Ctrl/Cmd + N | \nNew note | \n
Ctrl/Cmd + O | \nQuick switcher \u2014 jump to any note by name | \n
Ctrl/Cmd + P | \nCommand palette \u2014 every command, searchable | \n
Ctrl/Cmd + E | \nToggle editing / reading view | \n
Ctrl/Cmd + Shift + F | \nSearch across the whole vault | \n
Ctrl/Cmd + click | \nOpen a link in a new pane | \n
If you learn only one, make it Ctrl/Cmd + O.
![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig7.png\")
Type three or four letters and the note you want is usually the top hit. Once you pass a few hundred notes this becomes how you navigate almost exclusively, and the file explorer turns into decoration.
\nObsidian notes are Markdown, but you rarely have to think about syntax. You type a few characters and the formatting appears live.
\n![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig6.png\")
The syntax you will actually use day to day is small:
\n![](\"https://substackcdn.com/image/fetch/$s_!Ijfn!,w_56,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4837ef94-97ad-4e8b-90f5-908cc363b846_1365x1365.jpeg\")
# Heading 1\n## Heading 2\n\n**bold** *italic* `inline code`\n\n- bullet\n- [ ] unchecked task\n- [x] completed task\n\n> blockquote\n\n[[Link to another note]]\n[[Link to another note|shown as this text]]\n![[Embed another note entirely]]\n\n#tag #nested/tagTwo Obsidian-specific ones are worth calling out. The pipe in [[Note|display text]] lets the link read naturally in a sentence while still pointing at the right file. And ![[Note]] with a leading exclamation mark embeds the whole note inline rather than linking to it (useful for pulling a definition into a summary without duplicating it).
This is the part that matters. Create a note called Confounding and write a couple of sentences. Somewhere in the text, type two opening square brackets: [[.
A dropdown appears. Start typing G-computation. Even though that note does not exist yet, press Enter \u2014 Obsidian creates it the moment you click through.
Now open G-computation and look at the Linked mentions panel.
![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig3.png\")
You typed one link. You got two. That is the whole trick, and it compounds: every time you link a new note to an old one, the old note gets richer without you touching it.
\nNew users spend their first week building an elaborate folder hierarchy and their second week discovering it does not fit. Obsidian gives you three organising tools, and they answer different questions.
\n![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig4.png\")
Folders answer where does this live? A note sits in exactly one. Use them for broad, stable categories. Three to six top-level folders is plenty. Tags answer what kind of note is this? Type #method or #toread anywhere in a note. A note can carry many, which makes tags good for status and type \u2014 things that cut across folders. Nested tags like #status/draft give you a little hierarchy without much cost. Links answer what is this related to? These do the real work. Folders and tags are coarse; links are specific.
The failure mode is over-investing in the first and under-using the third. A flat vault with good links is far more useful than a beautiful eleven-level folder tree with none.
\nHere is a starter structure:
\nmy-vault/\n\u251c\u2500\u2500 Daily/ # one note per day, capture inbox\n\u251c\u2500\u2500 Notes/ # the permanent, linked notes\n\u251c\u2500\u2500 Projects/ # one note per active project\n\u251c\u2500\u2500 Reading/ # books, papers, articles\n\u2514\u2500\u2500 Templates/ # reusable note skeletonsLet Notes/ grow flat and link heavily inside it.
Go to Settings \u2192 Core plugins and enable Daily notes. In its options, set the folder to Daily/ and the date format to YYYY-MM-DD so the files sort correctly.
![](\"https://solomoneshun.com/assets/imgs/fig8.png\"/)
The daily note is a capture inbox, not an archive. Anything that occurs to you during the day lands here with no organising: meeting notes, half-ideas, a paper someone recommended, a problem you hit in your analysis. The bar for writing something down should be almost zero.
\nLater, you promote the few things that earned it into real notes in Notes/, with claim-style titles and links. Most of what you capture will not earn it, and that is the point. The inbox absorbs the noise so your permanent notes stay signal.
Turning on the Calendar community plugin gives you the month view shown above, which makes it easy to jump back to \"that Tuesday when the model broke.\"
Enable the Templates core plugin and point it at Templates/. Then create a file there like this:
# {{title}}\n\n**Source:**\n**Date:** {{date}}\n\n## Summary\n\n## Key points\n-\n\n## Links\n-Now Ctrl/Cmd + P \u2192 \"Insert template\" drops that skeleton into any new note.
![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig9.png\")
The payoff is not the typing you save. It is that every literature note has the same shape, so six months later you can skim thirty of them quickly, and a Dataview query can pull fields out of them reliably. Two or three templates is usually the right number: one for literature, one for meetings, one for projects.
\nCommunity plugins live in Settings \u2192 Community plugins \u2192 Browse. You will need to turn off Restricted Mode first.
\nResist installing twenty of them. Each is a thing that can break, conflict, or need migrating, and the most common way to end up with a broken vault and no notes in it is to spend week one configuring instead of writing. Add plugins to solve problems you have actually hit.
\nThe ones that earn their place for most people:
\n| Plugin | \nWhat it solves | \n
|---|---|
| Dataview | \nTurns notes into a queryable database | \n
| Calendar | \nMonth view for daily notes | \n
| Templater | \nTemplates with logic \u2014 dates, prompts, scripting | \n
| Excalidraw | \nHand-drawn diagrams stored inside the vault | \n
| Style Settings | \nTweak theme colours and spacing without CSS | \n
Dataview is the one that changes how the vault feels. Instead of maintaining an index note by hand, you write a query and it stays current.
\n![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig10.png\")
That query sits in a normal note. Every time you open it, Obsidian scans the Reading/ folder and rebuilds the table, so a paper you add next month appears without you editing anything.
Here is the honest truth about the graph view: it is beautiful, it is motivating, and it is useless for the first month.
\n![](\"https://solomoneshun.com/posts/obsidian//../../assets/imgs/fig5.png\")
A graph of twelve notes tells you nothing you did not already know. A graph of four hundred shows you clusters you never planned, orphan notes you should link or delete, and hub notes that turn out to be the real organising centres of your thinking.
\nBuilding structure before content. You cannot design the right hierarchy for notes you have not written. Start flat and let structure emerge.
Installing too many plugins. Add them to solve problems you have actually hit, not problems a video told you about.
Treating it as an archive. A vault full of clipped articles you never linked or reread is a graveyard, not a knowledge base. One note you wrote in your own words beats ten you pasted.
Splitting into many vaults. Links do not cross vault boundaries. One vault, folders inside it.
Chasing the perfect system. Every month someone publishes a new methodology with an acronym. The people getting real value from Obsidian are mostly running something boring and consistent. Boring and consistent wins.
Over-tagging. If every note carries eight tags, tags stop narrowing anything. A handful you actually filter on beats a taxonomy you admire.
If you want something concrete to follow:
\nDays 1\u20132. Install, create one vault, make the five folders. Write three notes about things you already know well and link them to each other.Days 3\u20137. Turn on daily notes. Capture into them every day, without organising. Do not touch settings.Day 7. First review. Promote two or three captures into real notes. Notice what you keep reaching for.Week 2. Add a literature template. Install Dataview only if you feel the need for an index. Write a note that links to five existing ones.End of week 2. Open the graph for the first time. It will be small, and that is fine.Once the daily loop is automatic, the next steps are richer Dataview queries, Templater for dynamic templates, and if you want to publish, either Obsidian Publish or a static-site generator like Quartz, which renders a vault into a website for free.
\nBut none of that matters until the habit exists. After installation, everything else is elaboration. I hope this helps!
\nOpen Science und geistiges Eigentum werden oft als Gegens\u00e4tze verstanden: hier der freie Zugang zu Wissen, dort Patente, Urheberrechte und andere Schutzrechte. F\u00fcr den Transfer von Forschung in die Praxis greift diese Gegen\u00fcberstellung jedoch zu kurz. Die entscheidende Frage ist vielmehr: Was sollte offen sein, was muss gesch\u00fctzt werden \u2013 und wann?
\nDieser Frage wurde auf der Innovations-Roadshow des Deutschen Patent- und Markenamtes (DPMA) am 8. September 2026 in Hannover an der TIB nachgegangen.
\n![\"Teilnehmende](\"https://blog.tib.eu/wp-content/uploads/2026/10/2026-innovations-roadshow-1-1024x768.jpg\")
Forschungsergebnisse sind mehr als wissenschaftliche Publikationen. Sie entstehen als Daten, Software, Modelle, Videos, Prototypen oder technische Erfindungen. Damit stellen sich zugleich Fragen nach Zug\u00e4nglichkeit, Nachnutzung, Urheberrecht und gewerblichem Rechtsschutz.
\nWer Forschung offen zug\u00e4nglich machen und Innovation erm\u00f6glichen will, muss deshalb beides zusammendenken: Offenheit und Schutz.
\nEine wichtige Schnittstelle daf\u00fcr ist das Patentinformationszentrum (PIZ) Hannover an der TIB. Seit 1878 unterst\u00fctzt es dabei, technisches Wissen und Informationen \u00fcber Schutzrechte zug\u00e4nglich und nutzbar zu machen. Heute ist das PIZ das einzige Patentinformationszentrum in Niedersachsen und Kooperationspartner des Deutschen Patent- und Markenamts.
\nZu den kostenfreien Basisdiensten geh\u00f6ren Informationen und Fachliteratur zu Patenten und gewerblichen Schutzrechten, Unterst\u00fctzung bei Recherchen in Patent- und Schutzrechtsdatenbanken sowie Informationen zu nationalen und internationalen Schutzrechtsverfahren. Veranstaltungen und Sensibilisierungsangebote vermitteln zudem Grundlagen des gewerblichen Rechtsschutzes.
\nDabei versteht sich das PIZ Hannover als neutrale Lotsenstelle: Die kostenfreie Erfindererstberatung kann bei Bedarf zu IHK oder Patentanwaltschaft weiterf\u00fchren. F\u00fcr spezialisierte Recherchen, Analysen oder Patentstatistiken werden geeignete Anbieter und Einrichtungen vermittelt. Auch bei Fragen zu Verwertung, Schutzrechtsmanagement oder F\u00f6rderm\u00f6glichkeiten verweist das PIZ auf die jeweils zust\u00e4ndigen Stellen.
\nSo bietet das PIZ einen kostenfreien, neutralen und niedrigschwelligen Zugang zu Informationen und Orientierung im Netzwerk der Schutzrechts- und F\u00f6rderangebote.
\nF\u00fcr Forschende beginnt der Mehrwert oft schon vor einer Patentanmeldung: Welche L\u00f6sungen gibt es bereits? Was wurde schon ver\u00f6ffentlicht? Welche Entwicklungen sind neu \u2013 und wo gibt es Ankn\u00fcpfungspunkte f\u00fcr die eigene Forschung?
\nEine Patentrecherche kann dabei auch helfen, Unternehmen, Forschungseinrichtungen oder andere Akteure zu identifizieren, die an \u00e4hnlichen Technologien arbeiten und als m\u00f6gliche Kooperationspartner infrage kommen.
\nPatentinformationen k\u00f6nnen damit eine Br\u00fccke zwischen wissenschaftlicher Erkenntnis und technologischer Entwicklung bilden.
\nEin Beispiel f\u00fcr diese Verbindung ist die Patentdatenbank Orbit Intelligence. Die Plattform kann aus wissenschaftlichen Texten technische Merkmale extrahieren und dazu passende Patentdokumente identifizieren. Lizenziert f\u00fcr Nutzende der TIB. DBIS \u2013 Orbit Intelligence
\nSo l\u00e4sst sich etwa pr\u00fcfen, in welchen Patenten bestimmte Komponenten, Verfahren oder technische Zusammenh\u00e4nge beschrieben sind. Orbit unterst\u00fctzt damit die Recherche und Analyse \u2013 ersetzt aber keine rechtliche Pr\u00fcfung von Neuheit oder erfinderischer T\u00e4tigkeit.
\nF\u00fcr Forschende entsteht so eine praktische Br\u00fccke zwischen wissenschaftlicher Publikation und Patentinformation.
\n![\"Screenshot](\"https://blog.tib.eu/wp-content/uploads/2026/10/2026-orbit_piz-1024x448.png\")
Wie aus Patentinformation und Forschungsergebnissen konkreter Technologietransfer entstehen kann, zeigt das Beispiel von Bardia Akaberi der Firma Goldhoop.
\nAusgangspunkt war die Recherche nach dem Stand der Technik, auf die dann der Schutz der Erfindung als Gebrauchsmuster DE 20 2025 003 427 folgte. Doch der Transfer endete nicht mit dem Schutzrecht. Ein funktionierender Prototyp wurde umgesetzt und in einem Video dokumentiert. Denn ein Film sagt mehr als 1.000 Bilder: Die technische Idee wird anschaulich, verst\u00e4ndlich und f\u00fcr potenzielle Kooperations- und Transferpartner unmittelbar erlebbar.
\n\u00dcber das TIB AV-Portal (https://doi.org/10.5446/73021) bleibt die Dokumentation dauerhaft auffindbar und zitierf\u00e4hig. Im TIB-Portal \u00a0k\u00f6nnen Patent, Video und wissenschaftliche Informationen miteinander verkn\u00fcpft werden.
\n![\"\"](\"https://blog.tib.eu/wp-content/uploads/2026/10/2026-dpma-1024x582.png\")
Ein Schutzrecht allein macht eine Erfindung noch nicht sichtbar. Erst wenn eine technische Entwicklung verst\u00e4ndlich, auffindbar und dauerhaft dokumentiert ist, kann daraus Interesse entstehen.
\nSichtbarkeit \u2192 Verst\u00e4ndnis \u2192 Interesse \u2192 Kontakt \u2192 Technologietransfer
\nSo wird aus gesch\u00fctztem Wissen ein anschlussf\u00e4higes Forschungsergebnis \u2013 und aus Forschung kann Innovation entstehen.
\nDie TIB macht mit Ihren Services Digitale Dienste und Dienstleistungen der TIB \u00a0Wissen dauerhaft sichtbar, recherchierbar und transferf\u00e4hig. Das PIZ Hannover an der TIB unterst\u00fctzt Sie bei den gewerblichen Schutzrechten. \u00a0So entsteht eine Br\u00fccke zwischen Forschung, Schutz und Innovation. Sprechen Sie uns gerne an!
","doi":"https://doi.org/10.65527/hbjbb-28825","guid":"https://blog.tib.eu/?p=33999","image":"https://blog.tib.eu/wp-content/uploads/2026/10/2026-dpma.png","language":"de","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1791504000,"rid":"f73k7-csb39","summary":"Open Science und geistiges Eigentum werden oft als Gegens\u00e4tze verstanden: hier der freie Zugang zu Wissen, dort Patente, Urheberrechte und andere Schutzrechte. F\u00fcr den Transfer von Forschung in die Praxis greift diese Gegen\u00fcberstellung zu kurz. Die entscheidenden Fragen sind vielmehr: Was sollte offen sein, was muss gesch\u00fctzt werden \u2013 und wann?","tags":["SERVICES","Lizenz:CC-BY-4.0-INT","TIB AV-Portal","TIB-Portal","Patent"],"title":"Offen forschen, gezielt sch\u00fctzen","updated_at":1791617415,"url":"https://blog.tib.eu/2026/10/09/offen-forschen-gezielt-schuetzen/","version":"v1"}},{"document":{"authors":[{"affiliation":[{"name":"Fachhochschule Potsdam"}],"contributor_roles":[],"family":"Kaden","given":"Ben","url":"https://orcid.org/0000-0002-8021-1785"}],"blog":{"authors":null,"community_id":"00dd7e11-a802-44c1-9584-5c56d1f8d417","created":1706832000,"current_feed_url":null,"description":"Vernetzungs- und Kompetenzstelle Open Access Brandenburg","doi":"https://doi.org/10.59350/oabrandenburg","favicon":"https://rogue-scholar.org/api/communities/00dd7e11-a802-44c1-9584-5c56d1f8d417/logo","feed_format":"application/atom+xml","feed_url":"https://open-access-brandenburg.de/feed/atom","filter":null,"generator":"WordPress","home_page_url":"https://open-access-brandenburg.de/","issn":null,"language":"deu","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"oabrandenburg","status":"active","subfield":"1802","title":"Open Access Brandenburg","updated":1791545849,"use_api":false},"blog_name":"Open Access Brandenburg","blog_slug":"oabrandenburg","content_html":"Es gibt ein relativ neu beschriebenes Ph\u00e4nomen im Open Access bzw. Diamond Open Access, \u00fcber das unl\u00e4ngst Nature berichtete und auf das wir ebenfalls kurz hinweisen wollen. So berichten Lisa Matthias vom Institut f\u00fcr Bibliotheks- und Informationswissenschaft der Humboldt-Universit\u00e4t zu Berlin und ihre Ko-Autoren Juan Pablo Alperin (Simon Fraser University, Vancouver) und Mikael Laakso (Tampere University, Tampere) \u00fcber F\u00e4lle, in denen wissenschaftliche Zeitschriften von einem Diamond-Open-Access-Modell auf APC-basierte Gold-OA, Hybrid-OA oder Subskriptionsmodelle umstellten:
\nLisa Matthias, Juan Pablo Alperin, Mikael Laakso: Diamond Fractures: Tracing Journal Transitions Away from Diamond Open Access. (30.06.2026) DOI: 10.48550/arXiv.2606.31302\n\nInsgesamt konnten sie 440 F\u00e4lle f\u00fcr den Zeitraum von 2009 bis 2026 identifizieren, wobei der gr\u00f6\u00dfte Anteil auf den Wechsel zu Gold Open Access mit Publikationsgeb\u00fchren zu verzeichnen ist. Interessanterweise beschleunigte sich diese Entwicklung in den vergangenen Jahren. Ebenfalls interessant erscheint, dass dies nicht etwa mit Verlags- oder Betreiberwechseln (\"ownership\") zu erkl\u00e4ren ist \u2013 die \u00fcberwiegende Zahl der Titel, 389 von 440 \u2013 blieb beim Ursprungsbetreiber.\nBei einem Teil der F\u00e4lle l\u00e4sst sich Diamond Open Access als \u00dcbergangs- und Etablierungsma\u00dfnahme nach der Neugr\u00fcndung einer Zeitschrift verstehen:
\n\n\n\"we see the pattern of publishers forgoing APC revenue as a promotional strategy, while a new journal builds indexing and reputation, then introduce them once the journal is established\".
\n
Der gr\u00f6\u00dfere Teil der Titel \u00e4ndert sein Betriebsmodell allerdings erst sp\u00e4ter, was mit dieser Deutung adressiert wird:
\n\n\n\"The temptation and relative ease of adopting APCs is particularly evident for these journals, given that such established journals would be strong candidates for collective funding models, such as Subscribe to Open.\"
\n
Besondere Relevanz erh\u00e4lt die Analyse deshalb, weil Diamond Open Access verst\u00e4rkt, wie auch zuletzt auf den Open-Access-Tagen 2026, als ein m\u00f6gliches Alternativmodell besonders zum geb\u00fchrenfinanzierten Gold Open Access angesehen und diskutiert wird und man daher sehr viel \u00fcber die Verschiebung von Gold zu Diamond spricht. Die entgegengesetzte Entwicklung wird dagegen bisher kaum thematisiert. Dank der vorgelegten Auswertung ist \"Diamond Fractures\" im Diskurs und auch in Nature angekommen. Zudem unterstreicht der Befund zus\u00e4tzlich den Bedarf an Diamond-OA-Modelle, die auf strategische Verstetigungen in nicht-kommerziell ausgerichteten Kontexten aufbauen.
\nWer mehr \u00fcber Diamond Fractures erfahren m\u00f6chte, hat im November die Gelegenheit. Denn am 27.11.2026 ab 18 Uhr und virtuell wird Lisa Matthias ihre Forschungen zum Thema \"Diamond Fractures\" im Berliner Bibliothekswissenschaftlichen Kolloquium an der Humboldt-Universit\u00e4t zu Berlin pr\u00e4sentieren: Hybrid und Diamond Open Access unter der Lupe \u2013 Open Access als Forschungsobjekt der Bibliotheks- und Informationswissenschaft.\u00a0
\n\n\nZitierhinweis:
\nKaden, Ben (2026): \"OA-News: Diamond (Open Access) Fractures als Forschungsthema.\" DOI: 10.59350/yq0vn-j2231
\nA new working paper about mass reproducibility in psychological science by Brodeur, Baki, & Fazio finds: \"84% of preregistered articles deviate from their pre-analysis plan, and nearly half of these deviations go unacknowledged.\"
As Brodeur et al. explained:
Among the 50 preregistered reported articles, 42 (84%) deviated from the pre-analysis plan in at least one respect, and 8 (16%) did not. Of the 42 articles with deviations, 25 (59.5%) disclosed some or all of them, while 17 (40.5%) disclosed none. The most common form of deviation was implementing a different procedure in the paper from the preregistration, occurring in 57.143% of preregistered articles with deviations. \u2026. Deviations involving design or reporting scope were most frequent, occurring in 25 articles, followed by estimation or modeling choices, occurring in 23 articles. Deviations related to sample size/data collection and data exclusions/filtering were each identified in 12 article, while deviations concerning variable construction were prevalent in 10.
The 84% deviation rate is close(ish) to my own estimate that \"around three-quarters of preregistered studies included at least one deviation\" (Rubin, 2025, p. 19, based on studies by Cheng, 2022; Claesen et al., 2021; Clayson et al., 2025; Heirene et al., 2024; Singh et al., 2021; TARG Meta-Research Group and Collaborators, 2023). In my article, I noted that this
evidence is unsurprising given that leading proponents of preregistration encourage researchers to treat preregistration as \"a plan, not a prison\" (Chambers, 2019, pp. 188, 189; DeHaven, 2017; Nosek et al., 2019, p. 817)\u2026 [But the problem is that] adjustable plans do not control the Type I error rate (see also Navarro, 2020, p. 8).
Brodeur, A., Baki, G. A., & Fazio, L. (2026). Mass reproducibility in psychological science. I4R Discussion Paper Series, No. 326, Institute for Replication (I4R). https://hdl.handle.net/10419/344024
and colleagues report the results of their study of prospective registration among observational studies of health care interventions. As part of their study, they surveyed 46 authors about the perceived benefits and barriers to registration. They found that 60% of responders expressed positive attitudes towards registration, and that barriers to registration included limited resources (62%, 26/42), lack of tradition (57%, 24/42), and insufficient knowledge of registries (50%, 21/42).
Our findings regarding author attitudes toward registration align with previous similar studies. Although authors typically acknowledge the benefits of registration for improved planning and methodological transparency of observational studies [26, 27], concerns persist regarding limited perceived added value (e.g., unnecessary bureaucratization, increased workload), unintended side-effects (increased costs, and discouragement of exploratory work), and reduced analytical flexibility [26\u201328].
Jespersen, C., Song, Z., Chan, A. W., & Hr\u00f3bjartsson, A. (2026). Registration of observational studies of interventions: Prevalence, author attitudes, and journal policies \u2013 a meta-research study. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-026-00258-3
Torsten Wilholt published an article in which he asks \"Is trust in science a good thing?\"
Part of Wilholt's answer addresses metascience and the open science movement. He notes that the response to the crisis of confidence has been to increase transparency and methodological standards in order to make science more controllable and verifiable. He contrasts this open science approach with Hardwig's (1985) thesis that scientists need to have a certain degree of trust between one another. He points out that:
If we can regulate all actions in the research process by standards and conventions and make them fully explicit, then there is no need to leave any discretionary powers to researchers, and trust can be replaced by control.
However, Wilholt ends up arguing against this position because standards designed to regulate scientific research cannot be applied to research that targets deep ignorance, in which we do not yet have any candidate answers, and radically deep ignorance, in which \"finding the correct answers would require a radical overhaul of our conceptual resources, our styles of reasoning, or our background beliefs.\"
Many of the standards advocated by the reform movement are embedded in frameworks of statistical analysis. This applies to the call to move away from simply calculating p-values and to pay closer attention to other statistical measures such as power, effect sizes, or confidence intervals, as well as, of course, to the more radical proposals to use Bayesian statistical analysis rather than frequentist methods. Statistical methods, whether Bayesian or frequentist, work only against the backdrop of a spectrum of previously identified competing hypotheses. The point also concerns other proposed methodological improvements, such as the pre-registration of studies, which virtually the entire open science movement expects to be highly effective. Questions for which we have no potential answers cannot be made the subject of research in a way that satisfies such frameworks. Yet science also needs research that aims to overcome deep and even radically deep ignorance. It is not at all clear how the substitution of trust with control is supposed to work in such research. Hardwig's thesis remains as valid as ever.
Wilholt's point remind me of points made by others about the difficulties of conducting preregistered confirmatory research with clearly defined alternative hypotheses (H1) and effect sizes. For example, Pek et al. (2024) explained that:
For most psychological research, if the value of H1 were known, there would be little reason to conduct the research. The unknown value of H1 is the very reason Fisher (1955) strongly opposed specifying an alternative H1 value. The high uncertainty in specifying H1 prevents sensible values of Type II error rate \u2026 and power \u2026 from being calculated
Similarly, although Scheel et al. (2021) see value in preregistered confirmatory research, they also cautioned that, in some cases, psychological scientists may not be ready to test hypotheses:
Psychological scientists have not developed methods for specifying the alternative hypothesis in sufficient detail to make it statistically falsifiable (Meehl, 1967; Morey & Lakens, 2016). This problem is not solved with mere practice\u2014forcing researchers to specify what would falsify their hypotheses when they have no theoretical basis for doing so can lead to testing against arbitrary values (Kruschke, 2018) and runs the risk of replacing one mindless ritual with another.
Wilholt concludes that:
to try and substitute control for trust would mean to fail to recognize that research cannot be fully standardized, as shown by the case of research aimed at overcoming deep ignorance. Hardwig's thesis about the irreplaceability of trust within science thus remains valid.
Wilholt, T. (2026). Is trust in science a good thing? European Journal for Philosophy of Science, 16, 74. https://doi.org/10.1007/s13194-026-00781-6
Tomorrow we\nwill have a discussion in our lab about how to use AI as a scientist. I have\nincreasingly started to use AI, having moved from Copilot Chat that our\nuniversity provides last year, to some free credit on GitHub Copilot under\ntheir educational program, to Claude Pro integrated in Visual Studio Code, to a\nClaude 20x max subscription (in part to also make these AI tools available to\nmy lab in case they want to use them).
\nI work with\ncollaborators whose opinions and values are important to me, and several of\nthem are more skeptical about the use of AI. I see their concerns, both in how\nmy students use AI, and how some peers use AI. I am as annoyed by people use write texts where AI has\nmade up references as anyone. I also see how tempting it is to use AI mindlessly. It does\nso much, so quickly, and validating that it has done things well is so\neffortful, that you get sucked into overtrusting AI. Some people don't want to\nuse AI at all, but for me, the question is not whether to use AI, but how to\nuse it well. Of course, I don't see any problems with how I use AI, and like in\nthe rest of my work, I need others to criticize me to point out my own biases.\nSo I am writing down my current thoughts, to make them as explicit as possible.
\nI use smart\nhome tools, and have been using If This Than That (IFTTT) for years. I like to\nuse technology to make my life more efficient. In the last year, I have been\nusing AI to create tools that improve my life. For example, I have asked an AI\nto create a pipeline where I can transform text (books and papers) to audio on\nmy laptop, sync it to my phone, where I have a custom made Android app that I\ncan use to listen to these audio files. When I swipe back, I can mark the\nsection I listen to, and the text (with the use of meta-data files that link\nthe time stamps in the audio to the sentences in the text) of this section gets\nautomatically synced to my Obsidian note-taking tool. If that feels like\noverdoing it to you, and that you might go as far as buying a tablet to read,\nbut that building custom software is a bridge too far, I understand. My\npersonality is such that I like to be efficient. I would predict that the more\nefficiency matters to you in your job, the more you use technology, including\nAI.
AI use also\naligns with work I do. I spend a lot of time behind my computer, interacting\nwith software, working on websites, and I program R packages and tools such as Metacheck. Of course a computer already automates a ton\nof our activities, so AI use is an extension of the tasks we automate. But if\nyou spend less time behind a computer, and especially if you do not code, there\nwill be less use-cases for AI.
\nI taught a\nHuman Factors course, and the book we used has a chapter on\u00a0human-automation interaction. It starts with\nthe question \"Why automate\" and offers 4 reasons: 1) it is impossible of\nhazardous for people to perform these tasks, 2) tasks are difficult or unpleasant,\n3) automation can extend human capability, or 4) we do it just because it is\npossible, even where it has no benefits. To the extent that we see AI as\nautomation, I have never used it to do impossible or hazardous tasks, but I\nthink I have experienced the other 3 categories. Sometimes the use of tools has\nbeen useful, sometimes less so (but learning when AI use is useful is for me\njust part of learning to use new technology).
\nLet's start\nwith unpleasant tasks. I often perform tasks that take a long time, and/or need\nto be monitored, but that require very little cognitive resources, and that an\nAI can do.
\nI teach\nIntroduction to Psychology and Technology in the bachelor this quarter, and\nMetascience in the master. Both courses have weekly assignments that should be\ngraded. The colleague who was supposed to assist with grading fell ill. We use\nthe Canvas learning management system. Many of the assignments are mainly\nintended to activate students \u2013 for example, they have to walk around campus\nand take pictures of things in the built environment to assist blind and/or\ndeaf people to find their way, or they perform an online experiment and have to\ncopy-paste their results. All I do for these assignments is check if they have\ncompleted them, and then I assign them the points. There are also open\nquestions that need actual grading by comparing answers to a rubric.
Canvas has\nan API, but figuring out how it works is too difficult and unpleasant so I\nnever used it in the past. But AI can figure out how the API works. I use AI to\nwrite R scripts that download all assignments, and store the questions and\nanswers in an spreadsheet. I use that spreadsheet to grade, which prevents me\nfrom having to click 20 times for all 160 students in the slow online Canvas\nsystem. When I am done with grading, I use an R script to upload the results to\nCanvas.
I also use\nClaude to check if assignments are completed. In the past, I would give all\nstudents a point for some questions without checking if they actually uploaded\nreal pictures, as the assignment instructed. I knew some students uploaded\nimages unrelated to the assignment in an attempt to make it look like they did\nthe assignment, and I disliked that I did not have the time to grade these\nassignments, but I found it an acceptable trade-off for more engaging class\nassignments. Now, I can use AI to check the assignments I would otherwise just\nassign points for, thereby keeping the more engaging assignments, while also\nchecking that students who try to get point without doing the assignment receive\nno points.
There are a\nnumber of similar use-cases for grading. I have compared the quality of grading\nof low-stakes assignments last year, compared to this year, and the use of AI\nhas improved the accuracy of the grades I assign. To be clear, I was aware that\nmy grading was too lenient, but I think weekly graded assignments are more\neducational than removing them due to a lack of time to grade them carefully.\nFor both courses, the assignments that are graded with AI are a small\npercentage of the final grade, unlikely to determine who passes a course, and\nwho not. The psychology course has an in person final exam and for the\nMetascience course I have switched to (rather labor intensive) oral exams to\nmake that master course AI proof.
The time\nsaved on grading with the use of AI is substantial (approximately 2 hours per\nweek, for 8 weeks), and the quality has increased. At the same time, I invest\nsome of this time back into education, making new and more engaging\nassignments, and more personal assignments and tests. I think it has made my\neducation better.
\nI perform\nmany tasks on a computer that take a long time. In the past, I ran simulations\nthat would last days, and currently, I run automated checks that Metacheck\nperforms on thousands of scientific articles. When a computer runs for days,\nthere are many things that can go wrong. AI helps me to create scripts that are\nmore reliable, that store intermediate results and can restart easily, informs\nme when there are unforeseen problems, and allows me to quickly resolve them.\nClaude allows you to chat with another computer through the browser, which\nmakes it very easy to leave a computer at work on over the weekend, while\nmonitoring its progress, and asking Claude to resolve problems that emerge. For\nexample, if during a run it becomes clear that there is a bug in a script,\nClaude can notice this before the run has continued for days, fix the code, and\nrestart. I remember how on the past I would often rerun the same thing for\nmultiple times across multiple days (often a real source of frustration in my\nwork).
\nI try to\nimprove my research practices, but there is only so much I am willing to learn.\nI have been using GitHub, as an amateur, but at least I managed to made code\nopen and have version control. But I never received training in Git, and I have\nused it suboptimally. Claude makes it much easier to adopt good coding\npractices, as well as cleaning up a repository after I made a mess of things.\nCreating new branches, using Large File Storage, creating public releases,\nmaking code reproducible, and documentation have improved a lot.
I am also\nincreasingly using Docker, which was another tool I knew I had to learn to use,\nbut found difficult. I was able to run containers in the past of there was a\nclear tutorial, but now I can also make docker containers and share them (for\nexample as part of Metacheck's reproducibility_check where we automatically\ncheck if code in repositories is computationally reproducible in a docker\nenvironment where most R or Python packages are pre-installed to speed up the\nchecks).
In an\nongoing project, where I am iteratively improving the code I am using, and\ntherefore the data I am generating is updated, I was losing track of which\nfiles where updated, and which not, with the risk that the manuscript described\nstale results. I knew targets\nwas the tool I should use to prevent this confusion, but I really did not have\nthe time to learn it. With the help of Claude, I reorganized my repo and\nimplemented it anyway.
\nThere are\nonly so many things I can learn, and my lack of knowledge limits the things I\ncan do. This is especially true when it comes to writing software and code.
AI helps me\nto complete ideas I have that I would otherwise never get around to complete.\nFor example, I turned static pictures into interactive visualizations in my html textbook. I want free\nopen textbooks to be better than paid textbooks by commercial publishers, and\ninteractive apps is one way to achieve this. But I don't have the time to learn\nhow to turn my code into an interactive app. AI is very good at this. This is\nalso true for the creation of Shiny apps. It took me a long time to learn Shiny\nat a mediocre level, but with AI, turning code into an R Shiny app is trivially\neasy. I have to load Shiny apps on my server,\nwhich runs on Linux, but I don't know Linux well, and AI is also of great help\nhere. Shiny apps are now more up to date, because maintaining them and updating\nthem takes less time.
One area\nwhere the use of AI becomes more problematic is when 'extending human\ncapabilities' means 'doing more in less time'. Where scientists used to have to\nbike to the university computer, hand in their programming cards, and return a\nday later to get the results, we can now perform statistics on our laptop in\nseconds. Where we used to browse through paper journals, we can now download\nany pdf version of an article we want. These innovations mean we can do things\nmore quickly. Where should we use AI to speed up our work, even if we can also\ndo this work manually? And, how do we know we are extending our capabilities,\ninstead of using AI to create flawed work?
\nWhen I let\nAI write code, I extensively validate the code against ground truth. This is\neasy when I know what results code should return, and not possible when I do\nnot know what code should return. When I program on Metacheck and I develop a\nnew module, for example one that automatically retrieves datafiles from\nrepositories, I use manually coded ground truth to validate the code I create\nwith AI. I know that my code should find repositories, and I can check if it\nactually finds them. Even in this validating step, AI can be very helpful.\nWhere I might manually check 20 repositories, I can use AI to check all\nrepositories.
Here, we\nget to some challenges I experience when using AI. Manual validation against\nground truth sounds like a good golden standard. But it is effortful, so using\nAI to validate AI is tempting. It is less work, and in all fairness, we are\nalso fallible human beings, so sometimes our manually coded ground truth is\nwrong, and AI points this out. I think I spent more time validating code as I\ndid before I used AI, but validation efforts are more extensive and more\nsystematic, and they often occur at a higher level. I can feel a greater\ndistance between the code and data and my validation work, towards an\nevaluation of the output of an entire piece of code.
Still,\nwhere needed, I can easily jump back in, and look at the dataframes my code\ngenerates, to see if the contents match my expectations. This requires coding\nskills, so AI can't replace learning to code (even if it might change how we\nlearn to code). For most of what I am building, AI seems to be doing very well,\nso this is increasingly less necessary \u2013 or maybe I am overtrusting my AI, and\nI think it is not necessary.
I spent a\nhuge amount of time validating code (as I spend less time coding, I spend more\ntime validating, and for many of the modules in Metacheck that I have created I\nspend weeks (often hundreds of hours) validating that they work, and\niteratively improving them. For this reason, I don't like the term\n'vibe-coding'. I feel this is a pejorative term that is used when people throw\ntogether code in a short amount of time without knowing what their code does,\nand whether it works. This is not what I do when I code with AI.
When I\nstarted coding, I did everything manually. Then I moved to copying code from\nstack overflow, then to copy-pasting code from ChatGPT, and now I ask Claude to\ncreate the code. This is an improvement over asking ChatGPT (and if you have\nnot used AI inside your programming software, you don't really known how coding\nwith AI works and can't judge it). Because Claude has access to the code, and\nthe results, it can check the input, code, and output. If you have a good\ndefault prompt (e.g., you tell it to always check everything, never make\nassumptions, etc.) the quality of the code is hundreds of times better (and\nreally not comparable) to asking ChatGPT 'how do I create a dataframe with data\norganized like I want'.
I also\nthink there is a lot to learn about how to set up a workflow where I am even\nmore likely to notice when AI created code that is not in line with what I\nintend to create. I am just learning how to use these tools \u2013 of course in a\ndecade I will be much better at it. When I learned to share my data openly, I\nlinked to a dropbox folder in my scientific articles. Of course these links are\ndead now, and this was really bad practice. I am sure I am making similar\nmistakes when I use AI. But the mistakes might already be smaller than when I\nfigured out new technology (like data sharing) myself.
When there\nis no ground truth, I would be much more careful in using AI. I also use AI to\nwrite reproducible manuscripts where I do not know what the results should be. In\nthese cases, I don't know if a number should be high, or low. I might still ask\nAI to create a first version of the calculation of the descriptive report. But\nthen I have to double check every number, and here, I often intended to compute\nother numbers than AI thinks I wanted to generate. It is still helpful, and the\ncorrections are less work than writing all reproducible analysis code from\nscratch, but the required oversight is much larger, and I really need to look\nat the raw data and code to make sure everything is accurate. Here, I will\nstill use AI to ask how to compute certain output, then I check the code, I\ncorrect it where needed, but then I would also ask AI to check my corrections.\nAfter all, I also make mistakes, and AI can help to catch these more quickly.
So, there\nis a lot of contextual variation in how much I use AI, and what role it plays\nin the work I am doing. I don't spend less time checking my work \u2013 if anything,\nI spend more time on it \u2013 but the nature of the checks have moved to a slightly\nhigher level, often more away from the data and individual code lines, to the\nresults that the code should produce. This is actually not that different from\nwhen I used to code simulations manually. I would also evaluate the results on\na higher level (the results of the simulation) and only jump into the\ndataframes and individual lines if I expected that there was something wrong.\nAt the same time, those simulations were built in smaller steps, and the\nindividual sections of code would receive at least a basic check. Now, these\nlower level checks are still present, but instead of manual checks, I often ask\nAI to write explicit tests (e.g., in this dataframe, column 1 should be a\nnumber between 0 and 1). This is on the one hand a better practice, but\nsometimes also comes closer to letting AI check AI. It's a space where I still\nneed to explore how to create a reliable and efficient workflow.
\nAs\nenthusiastic I am about using AI for code, as disappointing I am in using AI to\nwrite. Maybe it's because I have a lot of experience in writing, but I find AI\nwritten text mostly mediocre. This also holds for Peer Review. The only use I find\nin AI for peer review is (beyond maybe using Claude to automatically reproduce\nyour results) is to remove the Dutch Directness that can creep in some reviews.\n
There are\nsome exceptions. Sometimes AI can improve parts of text I write. I write too\nlong sentences, and sometimes as a non-native English speaker I create weirdly\nformatted sentence constructions, I don't know the right words, or the thoughts\njust appear on paper somewhat unclear. Asking AI to suggest an alternative\nsometimes (but not always) provides inspiration for how to improve the\nsentence.
Another\nexception is low-stakes writing. Sometimes as academics we need to create text\nthat will not be read (or not be read by many people), where we would otherwise\ninvest so little effort that our writing would not be better than what AI\nproduces. Of course, you could also say 'why not reduce bureaucracy in\ninstitutions'? I agree. With many things, AI use amplifies underlying problems.\nIf people use AI to write, maybe what they are writing does not need to exist.\nIf AI increases our reflection on this topic, that is a good thing.
Another\nexception is documentation. If I want to summarize the code in a repository, AI\nwill do so more diligently that I can. Similarly, the Metacheck Manual has large sections that are written\nby AI (although I also rewrote a lot of sections) because it summarizes what\ncode does, and AI can do this relatively well. Ideally I would write every line\nmyself, but I don't have the time, and the entire team is busy making software.\nSo, AI makes documenting things less work, and although it is certainly not as\ngood as what a human would produce, given our priorities and limited resources,\nit suffices. If you prefer manually written documentation, as I do, and you\nhave time, it would be a great way to help with Metacheck.
\nI\npersonally like the ability to automate boring tasks (e.g., asking AI to\nprepare a meeting, copy files from my computer to a harddrive, cleaning up\nmessy folders, look at which of 10 meetings dates is free in my calendar, etc.).\nI also like the voice interface of Claude. Instead of reading Wikipedia (which\nI often do), I can ask Claude questions while am walking around. The answers\nlargely come from public sources like Wikipedia, but I can more quickly switch\nand develop my thoughts about a topic I don't know a lot about. I don't like\nthe use of AI to generate pictures, videos, or presentations. I find them bland\nand uninteresting. But, this drifts away from the specific use of AI as a\nscientist, and this blog is long enough as it is, so I will not go into details\nabout other uses of AI too much.
\nI am regularly\nreflecting on the ethical aspects of using AI. I am not an expert on this, so I\ncan't say sensible things. I hope there will soon be a European Union version\nof Claude for scientists, and I will switch over to one if it exists. I prefer to\nspend my money on non-commercial and open access tools. I am hopeful this will\nsoon become possible. But in the meantime, I think Anthropic is not that\ndifferent from my bank. There are a lot of values I disagree with, but the\nbenefits of having a bank outweigh the problems I have with banks in general. I\nam supportive of the general policy in the EU around the use and development of\nAI, so I hope that in the future I will be happier with the organizations that\nprovide me access to AI than I am with banks. I do not have a drivers license,\nbut if you have, you should probably be more upset about the companies behind\nthe oil you buy than about the company that provides your AI inference, in my\nhumble opinion.
Then there\nis the climate. Data servers use a lot of energy. Here, the question is how\nmuch energy we want to use, and what we want to use it on. I think people using\nAI to generate or adjust pictures and videos should stop using AI. I don't see\nany societal benefit, and it uses quite some energy. I also think we should\nreduce our reliance on data servers where possible. I personally have stopped automatically\nsyncing everything to the cloud. For example, I maintain a local back-up of my\npictures, but I no longer store them in a datacenter. If we want to\u00a0reduce the use of datacenters, I would\npersonally prioritize a reduced use of video streaming. I feel services like\nInstagram reels waste everyone's time and are overly addictive, and I would not\nmind it if people who want to use them need to pay the environmental cost\n(compared to the current free use) of streaming, and I think the world is\nbetter off if these services do not exist. I also think we should stop\nstreaming video with services such as Netflix. As someone who likes vinyl, I\nthink we should own the media we consume. If we want to rely less on\ndatacenters, I would prefer a societal movement away from all streaming\nservices such as Netflix. As video streaming takes up a much larger percentage of the use\nof datacenters,\nI think for my own personal life, I prefer to use energy more on AI, and less\non things like video streaming.
\nThis blog\nis mainly an attempt to make my current behavior and beliefs more explicit, so\nthat they become easier to criticize. I am sure I am getting things wrong, I am\nmissing things I should weigh more, I have not discussed all important aspects,\nand in 10 years I will read this and know better. But I at least feel prepared\nto go into our lab meeting next Monday, and spend some time with the\nold-fashioned use of a keyboard to organize my thoughts. (Yes, this 100% human\nwriting but it would be funny if you thought otherwise).
There's a funny thing that happens to me from time to time. I randomly stumble across an essay or a paper that resonates with me, sometimes one that I can't stop thinking about for days. Then I come to find out the author is a colleague here at my own University of Virginia in the building next door who I never knew of.
is a professor of psychology here at UVA (I don't know him, but hope to meet you soon, Dan, and compare notes). I just came across an essay Dan published back in August. It's good, and you should take the <5 minutes to read it in full. Even if you're not an educator, you probably still have junior folks working for you or may be doing some on-the-job training, and Dan's points are still relevant, so substitute whatever you want for \"students\" in your case.
His argument is that students should only use AI for the things they already know how to do well. Not brainstorm-with-AI-but-draft-yourself vibes, and not summarize-with-AI-but-write-the-essay-or-whatever-yourself vibes, which are what a lot of AI \"policies\" look like in higher ed, and even K-12. E.g., here are the policies from our own Albemarle County Public Schools around us here in Charlottesville Virginia:
It wasn't Dan's conclusion that stuck with me the most. It was the way he set up the argument by contrasting the objectives of the workplace versus education.
In the workplace, the product is the thing. The report, the grant proposal, the manuscript, the R package, the Python notebook, the pitch deck, whatever. The product is the thing that matters, and here we all agree to this ideal that the human should be the lead in the driver's seat, with AI being the associate.1
In education, the product doesn't matter. The product isn't the point at all.
Students produce products, but the product is seldom the point. No one wants to read the papers they write, and the math problems they solve have no practical value. Teachers have them do these tasks because they provide practice in cognitive abilities that we think matter.
The the R/Bioconductor exercises or the journal club-like essays from landmark genomics papers I assign in my class have zero value to the outside world. They exist purely so that a student practices integrating evidence, thinking about how to tackle a problem with code (even if they'll write that code later with an AI like I do), how to build up a persuasive argument.2
Dan's argument isn't a total ban on AI in the classroom, but to only allow AI for things that the students already know well. He draws an analogy to calculators.
Once you've mastered arithmetic operations, there's no benefit to hand calculations when you're solving an algebra problem. But if you're still learning algebra, don't use the algebraic functions.
Dan goes on to steelman a few objections toward the end of the essay. I recommend reading it in full, and I'd wager after you do you'll agree with his conclusion:
Students should only use AI for things that they already know how to do well.
Further reading:
Although please, I beg you, be respectful to yourself and to your colleagues, and don't lower your intellectual value in your workplace to zero by simply being the meat proxy between me and an AI that I'm perfectly capable of interrogating myself. Your colleagues, bosses, direct reports, and students can all smell the Claudish in the parts-per-trillion from a mile away, and no one wants to spend an hour reading something you spent less than one minute \"writing.\" I'm not innocent here, but I'm trying to do better.
The vast majority of what a senior/principal scientist does in industry and academia is persuasion, sometimes in the form of persuading others to believe that our results are real and meaningful, but most often in the form of begging for money to support research and trainees.
I'll be radically honest here - I never really got the idea of \"Test Driven Development\". I'm much more of a \"telnet into production and damn the consequences\" kind of guy. Look, it's much more fun and my cardiologist recommends severe shocks every now and again.
\n\nThe essence of unit testing is pretty simple:
\n\nSend it good data and if it returns \"true\" the test passes. Send it bad data and make sure it returns false. Easy peasy lemon squeezy.
\n\nAs part of my NLnet funded work on ActivityBot, I'm writing some proper tests for my bot framework. I am hopeful that some of them will be useful to other projects doing similar things.
\n\nThere are four major classes of tests - here are the problems I've found while writing them.
\n\nA basic test might be how to validate that an ActivityPub username is valid. Typically it would be something like @edent@mastodon.social - what assumptions do we make with that? It starts with an @ and has another @ somewhere in it, dividing into two parts, the user and the server. The server has to be a valid domain name - but that can include non-ASCII Internationalised Domain Names and Emoji domains.
What about the user part? Is it just A-Z and 0-9? Can it have dots? What about apostrophes like an email address?
\n\nBasically, how easy is it find the specifications for all the various component parts of ActivityPub? Not very!
\n\nWhat does a WebFinger response look like? Which parts are mandatory and what values can they contain?
\n\nWith any message that your project generates, how can you be sure that other ActivityPub servers will understand it?
\n\nSimilarly, when we receive a follow message from an external server, what should it look like?
\n\nThere are some JSON Schemas which can be used to validate some type of messages.
\n\nBut, again, there are many scattered specifications.
\n\nAs I've written about before, HTTP Signatures are difficult. There are two main problems. The first is verifying signatures.
\n\nIn order to do that, you need to gather a bunch of signatures and then find a way to replay them into the test. So I'm in the process of saving signatures which I can then share with others.
\n\nThe second problem is generating signatures. Sure, you can run tests on the signatures - but there's only one real way to test whether they're acceptable\u2026
\n\nPassing tests means nothing if you can't communicate with others and they can't communicate with you.
\n\nSome tests are easier than others. Sending a signed request should be easy and repeatable. But receiving messages isn't easily automated. It requires being able to control an external server and asking that to send messages.
\n\nWith the Mastodon API it is possible to make requests which will interact with your server - but only if it is a public server.
\n\nI'm putting all my tests online. I'm also sharing a collection of messages and signatures which you can use in your tests.
\n\nI'd love to know if you find them useful.
","doi":"https://doi.org/10.59350/nmxzr-19z59","guid":"https://shkspr.mobi/blog/?p=75774","image":"https://shkspr.mobi/blog/wp-content/uploads/2012/09/709ae69b.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1791590400,"rid":"s1wdt-2k111","summary":"I'll be radically honest here - I never really got the idea of \"Test Driven Development\". I'm much more of a \"telnet into production and damn the consequences\" kind of guy. Look, it's much more fun and my cardiologist recommends severe shocks every now and again.","tags":["/etc/","ActivityBot","ActivityPub","Php"],"title":"Some quick thoughts on Unit Testing ActivityPub","updated_at":1791703811,"url":"https://shkspr.mobi/blog/2026/10/some-thoughts-on-unit-testing-activitypub/","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Bekkers","given":"Rene"}],"blog":{"authors":null,"community_id":"c04223e6-4d3d-42ad-a718-878a8fc35d32","created":1739059200,"current_feed_url":null,"description":"Rene Bekkers","doi":"https://doi.org/10.59350/renebekkers","favicon":"https://rogue-scholar.org/api/communities/c04223e6-4d3d-42ad-a718-878a8fc35d32/logo","feed_format":"application/atom+xml","feed_url":"https://renebekkers.wordpress.com/feed/atom","filter":null,"generator":"WordPress.com","home_page_url":"https://renebekkers.wordpress.com","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"renebekkers","status":"active","subfield":"1404","title":"Rene Bekkers","updated":1791665072,"use_api":true},"blog_name":"Rene Bekkers","blog_slug":"renebekkers","content_html":"There seems to be s a lot of enthusiasm for AI powered tools for research these days, if I look at the feeds that the tech industry controls. No doubt that these tools greatly reduce the effort for a literature review that sounds plausible. It’s great to get a nice set of studies as a starting point for a new project in a new research area. But in these cases we usually don’t know what we don’t see. So here’s a cautionary tale from a rare case where we do know the errors of omission in AI powered tools.
\n\n\n\nThe challenge I gave the AI-powered tools is difficult: I asked them to identify replications of previous experiments in the multidisciplinary field of research on charitable giving. Such replications are needles in a field of haystacks. But we found them by hand, with a multidisciplinary team of six scholars in a study on replications of research on charitable giving. We conducted a comprehensive manual search on Google Scholar and identified 48 replications of experiments. How many replications from the set of 48 can AI-powered tools identify?
\n\n\n\nNot many, it turns out. The best tool, Google Scholar Labs, found one third of them. Most tools – even popular ones such as Perplexity, find fewer than 10. LeapSpace, the tool that Elsevier is trying to sell to university libraries in the Netherlands, performed abysmally. Collectively the twelve tools I assessed identified exactly half of all replications that we had found by hand. But it gets worse. Many studies that the AI tools identified as replications were not replications at all. Prompting more elaborately with explanations and examples did not make the results better.
\n\n\n\nThe news is not all bad. The tools also identified a number of new studies that we had not found because we had limited our search to keywords that returned a feasible number of results. The newly found replications used other terms than we had looked for. Another upside: at least the searches came back quickly. The slowest tool, Claude Science, took 35 minutes. The quickest tool returned results within a few seconds. The result: it could not find any of the replications.
\n\n\n\nThe results underscore the necessity of human review of results from AI-powered tools. At face value, the reports produced by the tools seem entirely plausible. However, human review of the results showed a high number of false positives, up to 70% of all results for LeapSpace. Conclusions based on the searches of the AI-powered tools are incomplete and incorrect. Without an accurate and efficient method to identify false positives the speed with which the tools concluded their work gives users the false impression that they can outsource identifying relevant previous work to AI-powered tools.
\n\n\n\nRead the full paper including the prompts and supplementary materials with all details and a link to the data and code here.
\n\n\n\n","doi":"https://doi.org/10.59350/gvwch-yhj37","guid":"https://renebekkers.wordpress.com/?p=4290","image":"https://renebekkers.wordpress.com/wp-content/uploads/2026/10/12aitools.png?w=975","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1791590400,"rid":"wb9gx-88f29","summary":"There seems to be s a lot of enthusiasm for AI powered tools for research these days, if I look at the feeds that the tech industry controls. No doubt that these tools greatly reduce the effort for a literature review that sounds plausible.","tags":["AI","Data","Experiments","Household Giving","Meta Science"],"title":"Fast, but incomplete and inaccurate: twelve AI-powered tools for literature search","updated_at":1791703809,"url":"https://renebekkers.wordpress.com/2026/10/10/fast-but-incomplete-and-inaccurate-twelve-ai-powered-tools-for-literature-search/","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/050qmg959","name":"Singapore Management University"}],"contributor_roles":[],"family":"Tay","given":"Chee Hsien, Aaron","url":"https://orcid.org/0000-0003-0159-013X"}],"blog":{"authors":null,"community_id":"f34e2211-9904-4b58-97ab-0beeb79ef6f7","created":1697068800,"current_feed_url":null,"description":"Aaron Tay's thoughts about academic librarianship","doi":"https://doi.org/10.59350/musings","favicon":"https://rogue-scholar.org/api/communities/f34e2211-9904-4b58-97ab-0beeb79ef6f7/logo","feed_format":"application/rss+xml","feed_url":"https://aarontay.substack.com/feed","filter":null,"generator":"Substack","home_page_url":"https://aarontay.substack.com","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"musings","status":"active","subfield":"3309","title":"Aaron Tay's Musings about Librarianship","updated":1791665977,"use_api":true},"blog_name":"Aaron Tay's Musings about Librarianship","blog_slug":"musings","content_html":"I think I may have discovered the \"evil conspiracy\" behind why academic search vendors are so reluctant to show us what happens on the semantic side of AI search.
Recently, I have been wondering about a strange asymmetry. I noticed that when academic search vendors use AI to expand lexical searches, they are almost always happy to show you what they did.
For example, Web of Science Smart Search will happily show you the Boolean search it used. The same is true of products such as EBSCO AI-assisted Search, scite Assistant, Primo Natural Language Search and many others.
Then you get to the semantic-search part, and suddenly the curtain comes down.
Take Web of Science Smart Search again. We are told it runs a hybrid search, meaning it runs both lexical and semantic searches in parallel, combines the results and then somehow ranks the combined list.
Yet we know almost nothing about what happens in the semantic-search portion, beyond the fact that it was done.
What exactly was searched? Did they expand your query before embedding it? Why all the secrecy?
Similarly, Scopus AI has a copilot that decides \"whether to run a vector search (aka Semantic Search) and/or a keyword search\". Yet again, all the interface displays is the expanded Boolean Search.
Recently, while studying a new AI search system, I accidentally got a look backstage.
And I discovered a shocking \"truth\". The system was generating document-like passages for semantic retrieval rather than simply embedding my original query.
They were not just synonyms or alternative search terms. The generated passages could introduce assumptions that were never part of my question, or even contain outright false statements.
Imagine entering the following query:
Can Google Scholar be used alone for systematic reviews?
And finding out that, behind your back, the AI generates the following for expansion:
Google Scholar is not sufficient as a standalone source for systematic reviews because studies have shown that while it indexes almost all relevant research, it is difficult to retrieve all the relevant papers within the top 1000 results it allows access to.
Hold on a moment. I never said Google Scholar was not sufficient! That's precisely what I'm trying to find out! Why is the system apparently answering my question before it has even searched?
Now the reason for the \"conspiracy\" seemed obvious.
Vendors don't want us to find out that semantic search is being fed hallucinated AI-generated query strings!
Okay, okay, I was just being facetious here. There is NO evil conspiracy.
This is not to say that systems never do something similar to what I have described. They may generate text resembling relevant documents to help with retrieval, but that text could introduce assumptions that do not reflect the user's information need or even contain outright falsehoods. Such additions could potentially introduce bias if used in a traditional search.
But what I have described is not necessarily problematic. In fact, it resembles established techniques such as Query2doc and HyDE for expanding semantic searches.
While it may seem strange, the fact is:
A generated passage that is not entirely accurate or true can still be useful, perhaps even more useful than the original query, when embedded for semantic retrieval.
Confused? Let me start from the beginning.
The system itself was fairly typical of the current generation of AI-powered academic search tools.
There was a query interpretation step, probably involving an LLM. The system identified things such as entities that could be turned into filters, then ran a hybrid search combining lexical and semantic retrieval.
The results were merged and deduplicated, and an LLM was then used to classify them (by prompting) into relevance categories such as \"Very relevant\" and \"Relevant\".
None of this is especially unusual, and you can see parts of this process in many academic search systems. For example, using LLMs to extract relevance criteria and then classify results is something we see in AI2's Asta Find paper. But let's focus on the even more common idea of hybrid search.
Hybrid search is becoming increasingly popular in the search industry. Typically, it involves running both lexical or keyword searches and semantic searches.
The logic is straightforward. Lexical search is good at matching words and phrases. Semantic search can retrieve documents that express similar ideas using different language.
This potentially gives you the best of both worlds. You just need a way to combine both sets of candidate results and rank or rerank them somehow.
So far, so good. But let's look at the interface (which I have mocked up) that was shown to me.
When it comes to lexical search, we are shown quite a lot. We can see that it runs the original query and two expanded, nested Boolean search strategies (probably generated by an LLM).
But on the semantic side, we are told almost nothing, except that two searches were conducted.
Why this distinction?
When I was interviewed by the vendor, I naturally asked for more details about what was happening on the semantic search side.
Intriguingly, the vendor told me that an earlier version of the product had exposed more of the internal search process. However, users found it confusing, so some of the details were removed.
My reaction at the time was predictable. I did not understand this view at all. Surely more transparency would be better?
Perhaps they could even add an advanced mode for those who wanted more information.
But then I discovered some of the details that might have been hidden, and I began to empathise with why the vendor had chosen not to show everything.
While testing the reproducibility of the search, I ran into a completely different problem. No matter what I did, the system seemed determined to cache the results.
While troubleshooting this, on the advice of ChatGPT, I started recording the browser's network traffic using Chrome DevTools. We will revisit this in a future post.
This produces a HAR, or HTTP Archive, file: essentially a log of the requests and responses exchanged between your browser and the servers behind the application.
As a side effect, this exposed considerably more of the search pipeline than the interface itself did.
The logs confirmed that the system was running:
My original lexical query;
Two additional nested Boolean searches;
Two semantic searches.
There were also hints on many details. For example, about how many of the top results from each search were retained and how they were combined. But for now, let's focus on semantic search.
While there was very little information about the actual embedding model used, there was something more interesting.
I could see the actual text being used for semantic retrieval, and it was definitely not my original query.
Roughly speaking, the system appeared to be doing the following:
Original query + an LLM-generated short description of what a relevant document might look like
In short, it appeared to be taking my query and appending a chunk of LLM-generated text before creating the query embedding.
Those with some training or knowledge of information retrieval will recognise this as resembling a family of techniques such as Query2doc-style expansion or HyDE (Hypothetical Document Embeddings).
I spend quite a bit of time testing AI-powered academic search tools and trying to work out what is really happening behind their interfaces. If you find this kind of independent investigation useful, please consider buying me a coffee on Ko-fi. It helps support more investigations like this, which I share freely on this blog.
If you are already familiar with how embeddings and dense retrieval work, feel free to skip this section.
As I have noted many times on my blog, \"semantic search\" is not a single technique. It is better understood as an objective: retrieving or ranking items according to their estimated meaning.
It is typically contrasted with lexical or keyword search, where retrieval and ranking depend primarily on matches between terms in the query and terms in the documents.
The techniques used to achieve semantic search have changed over time. In recent years, semantic search has often, though not always, referred to dense retrieval or dense vector search.
In a typical dense retrieval system, the query is passed through an encoder model, often based on the Transformer architecture, to produce an embedding: a vector representation (a long series of numbers) intended to capture useful aspects of the query's meaning1.
Documents, passages or text chunks are encoded in a similar way, usually in advance during indexing. The query embedding can then be compared with these precomputed document embeddings.
Both query and document embeddings are represented as vectors: long sequences of numbers. The system calculates a similarity score between them, commonly using measures such as cosine similarity or dot product. Documents whose vectors are closer to the query vector are treated as better candidates for relevance.
Important to note: the query and document do not necessarily need to contain the same terms to be considered similar. Rather, the encoder is able to convert texts that are similar in \"meaning\" into representations that end up close together in vector space.
Conceptually, for dense retrieval, what we are doing is this:
Query \u2192 embedding \u2192 find nearby document embeddings
Semantic search via embeddings is inherently something of a black box
While a Boolean search like
(\"artificial intelligence\" OR \"machine learning\") AND libraries
allows you to inspect the search strategy to understand why some documents are retrieved and others are not. Embedding retrieval, however, is much less transparent.
Firstly, it is extremely difficult to interpret the long series of numbers representing each query or document. Secondly, even if I gave you the calculated similarity score between a query embedding and a document embedding, that would still not provide a satisfying, human-readable explanation of why they are close.
I have long expected this to be unavoidable.2 But surely vendors can at least tell us what text they embedded?
So why don't we just use the query as entered? While this can work in practice, one approach that can improve retrieval is to do the following:
Use an LLM to generate a document, or at least part of one, that might be relevant.
Use that generated text to help construct the query representation for retrieval.
The generated pseudo-document or hypothetical document is then used to help with retrieval.Query2doc appends the generated pseudo-document to the original query before passing the combined text through an encoder to create the query embedding3. HyDE directly generates the embeddings of the generated hypothetical documents (without the query) and in some variants will average that embedding with that of the original query embedding.
Another related technique is RAG-Fusion, used by Elsevier's Scopus AI. Rather than generating hypothetical documents as HyDE and Query2doc do, it generates multiple variations of the original query, runs separate vector searches and combines the results using Reciprocal Rank Fusion. Like HyDE and Query2doc, it can potentially introduce assumptions not present in the original query.
Empirically, this can work better than using the original query alone. Why?
One way of thinking about it is that the query itself is typically quite different, in terms of expression, length and so on, from a relevant document containing the answer. The logic, then, is that by generating a document that might be relevant, its embedding may end up closer to the embeddings of the desired documents than the original query embedding would.
Imagine your query is:
Can you use Google Scholar alone for systematic reviews?
And you get a fairly neutral hypothetical passage explaining what systematic reviews are, why comprehensive searching matters, and so on.
I think most of us would be okay with that.
But what if it generates something like this?
Google Scholar should not be used alone for systematic reviews because it only displays approximately 1,000 search results at most and lacks the advanced search functionality needed to retrieve all relevant studies.
This is, to my knowledge, true. But would using this passage for retrieval introduce bias into the results?
Or worse, what if it uses this?
Google Scholar can be used alone for systematic reviews because Tay(2024) has shown that more than 99% of papers eventually included in a SR can be found in Google Scholar's index.
For those unfamiliar with the literature, there is no such Tay (2024) paper, although real studies have reported very high coverage of studies included in systematic reviews within Google Scholar.
That said, the problem is that being indexed is not the same as being retrievable through a search. Google Scholar restricts access to roughly the first 1,000 results, so even if most relevant papers are in its index, there is no guarantee that a search will surface them.
Surely that last hypothetical passage would produce horrible results?
Not necessarily.
Remember, the system is not necessarily treating the generated text passage as a factual answer. It is using the text to construct a representation for retrieval.
Even though the passage contains a fabricated citation and a potentially misleading conclusion, it also contains concepts highly relevant to the original question: Google Scholar, systematic reviews, database coverage, included studies and comprehensiveness.
It is entirely possible that embedding this passage would retrieve useful papers discussing Google Scholar's suitability for systematic reviews, including papers that argue against using it alone.
But what about bias? Could generating a hypothetical document arguing that Google Scholar cannot be used alone for systematic reviews favour papers reaching that conclusion over those arguing the opposite?
Again, not necessarily. Some research has found that neural retrievers, particularly bi-encoders, can be surprisingly poor at distinguishing between statements that differ only in negation. This suggests that simply reversing a claim might have less influence on retrieval than we would intuitively expect.
Still, this does not mean there is no risk of bias. Our hypothetical passages differ in more than just their conclusions. They also emphasise different concepts, which might influence which documents are retrieved and how they are ranked.
Whether that actually happens is something we would need to test empirically. We cannot tell simply by inspecting the generated text.
In fact, it is conceivable that a hypothetical passage containing entirely correct statements could produce worse retrieval results than one containing factual errors. After all, factual accuracy and usefulness for retrieval are not the same thing4. What matters is whether the generated passage helps the system retrieve relevant documents, although factual errors could certainly make that harder.
There is an old search tip that sounds superficially similar to what these systems are doing.
The advice goes something like this: when searching Google, think about what words or phrases a webpage answering your question might contain, then search using those terms.
Suppose I want to know whether Google Scholar can be used alone for systematic reviews.
I might reason that relevant papers would contain terms such as coverage, recall, comprehensiveness, included studies and search limitations, then search using those terms.
This makes sense and has a clear resemblance to what HyDE is doing.
In both cases, we are trying to anticipate what relevant documents might look like and search accordingly5?
There is, however, an important distinction here.
With the Google trick, I am predicting words or phrases that I expect to literally occur in relevant documents. If I guess incorrectly, I may damage the search, which is one reason why using LLMs to generate nested Boolean searches does not always produce the best results6.
With HyDE, even if the generated document is factually wrong, its embedding can still be useful if it ends up closer to the embeddings of relevant documents than the original query embedding would.
The hope is that this richer, document-like representation lands somewhere near real documents discussing those concepts.
The fact that a generated passage reaches the wrong conclusion about Google Scholar does not necessarily prevent that from happening.
Of course, I am not saying that a hallucinated passage cannot push the query embedding further away from relevant documents.
In fact, some literature suggests that such expansion techniques tend to help weaker retrievers but may hurt stronger ones. They may also work less well in areas where the LLM's knowledge is limited. As always in information retrieval, whether a technique improves performance depends on factors such as the retriever, the types of queries and the dataset being searched. Ultimately, these are empirical questions that should be settled through testing (which hopefully the vendor has done) rather than decided a priori.
The point is simply that spotting a hallucination does not, by itself, tell us how serious the problem is.
This brings us back to my supposed conspiracy.
Imagine the interface shows you:
\"Google Scholar\" AND (\"systematic review\" OR \"evidence synthesis\") AND (coverage OR recall OR limitations)
A librarian might question why certain terms were included, but they would still recognise what they were looking at and feel comfortable with it.
Now imagine the interface instead shows:
Google Scholar can be used alone for systematic reviews because Tay (2024) demonstrated that more than 99% of studies eventually included in systematic reviews can be found in Google Scholar's index.
I suspect the reaction would be much stronger.
Why is the system citing a paper that doesn't exist? Why has it apparently decided that Google Scholar is sufficient? Is it now biased towards papers supporting that conclusion?
These are very reasonable questions if you interpret the passage as an answer or a literal statement of the search criteria.
But in the world of semantic embeddings, it is not necessarily either. It may simply be an intermediate representation used to create an embedding.
Perhaps part of the problem is that what happens inside modern semantic search no longer maps neatly onto what librarians traditionally think of as a query.
Modern AI search really does expand your query.
The surprising part is that making things up isn't necessarily the problem.
And this, I think, explains at least part of the strange asymmetry I keep seeing in AI search interfaces. Even if vendors wanted to show such details, many users, including librarians, might find them confusing7.
The more details vendors show about the search process, the greater the chance that they will open a can of worms by raising questions and concerns among users and librarians. From the vendor's perspective, the safer option may be not to show them.
The other part of the issue is that modern information retrieval is largely driven by empirical experimentation. You can propose a perfectly logical change that should improve retrieval, but if the experiments show that it doesn't work, you shouldn't use it. The reverse is also true. Some techniques may seem counterintuitive to librarians but work well empirically8.
While I can understand how vendors might feel, I still don't think they should hide all the details. We have a legitimate reason to want to know what is happening under the hood.
For librarians, the practical implication is this: resist the instinct to judge the semantic side of AI search by the same standards as Boolean. Spotting a hallucination in a query expansion is not evidence that a system is broken. The more useful question to ask is not \"does the generated query look right?\" but \"does the system consistently surface relevant results?\" That requires empirical testing against known relevant papers, not query inspection.
For librarians supporting systematic reviews the stakes are higher of course given the higher standards of transparency and reproducibility required.
What do you think?
Some might argue that showing the expanded query used to generate the query embedding is not particularly important. But the question applies more broadly to all aspects of the search pipeline.
Would you like vendors to show every detail, even if you don't fully understand it or think some of the techniques are strange?
If you found this essay on what AI-powered academic search tools are really doing behind the scenes educational, please consider supporting my work.
How is the encoder model able to generate representations that capture something resembling \"meaning\"? Encoder models learn useful language representations during pretraining on large text corpora, then are typically fine-tuned for retrieval so that relevant query-document pairs receive more similar vector representations than irrelevant ones.
You can get more transparency using multi-vector or late-interaction methods such as ColBERT, or learnt sparse representation methods such as SPLADE, but these are currently not commonly used.
Both techniques use very similar prompts to generate documents that Query2doc calls pseudo-documents and HyDE calls hypothetical documents. In practice, there is little difference between the generated texts themselves. The main distinction is that Query2doc was designed as a general query-expansion method. The query and expanded text can be used for lexical matching as well as retrieval via embeddings. HyDE was designed specifically for embedding-based retrieval.
Factual accuracy and retrieval effectiveness are different properties, but that does not mean they are statistically independent or even negatively correlated! For example, Yoon et al. (2025) found that hypothetical-document expansion improved retrieval mainly when generated passages contained information supported by the actual evidence, raising concerns that some reported gains might reflect knowledge leakage rather than better retrieval.
Notice this technique also risks bias!
I have long argued that using LLMs to generate Boolean is not very productive mostly because they tend to be poor at generating Boolean search strategies and even if they did do it decently it does not help average searchers who already know how to do so. A recent paper by San Diego State University documented how difficult it was to get the LLM to generate reasonable boolean just with prompt engineering.
I can imagine librarians or researchers unfamiliar with information retrieval becoming upset when they see what is happening and assuming that the search is broken.
Of course, it is important that a technique works generally and not just on limited test sets.
I have written a lot of notes that I never saw again. Not because I deleted them. They were still somewhere on my computer, carefully saved inside a folder with a sensible name. The problem was remembering that they existed when I actually needed them. That is the strange thing about most note-taking systems. We spend a lot of time thinking about how to store information and surprisingly little time thinking about how we will find our way back to it.
\nYou read something interesting. You write a note. You choose a folder. Maybe you add a tag. Then you move on. Months later, you are working on something related and vaguely remember, I know I wrote something about this before. So you search, open a handful of notes that are not quite right. Eventually, you either find it after far too much searching or give up and recreate the idea from scratch. The note is still there. The problem is that it just never found its way back into your thinking.
Obsidian approaches note-taking differently. Instead of treating notes as isolated documents that need to be carefully filed away, Obsidian encourages you to think about how ideas relate to one another. Suppose I am reading about confounding and create a note about it. Somewhere in that note, I mention g-computation. I can simply wrap g-computation in double brackets: [[g-computation]]. Those double brackets create a link. If I already have a note called g-computation, clicking the link takes me there. If I do not, Obsidian lets me create one. More importantly, the relationship works in the other direction too. When I eventually open my g-computation note, I can see that the confounding note links back to it. As I keep reading and writing, these connections accumulate and gradually becomes a network of ideas.
There is another reason I became comfortable with Obsidian. Your notes are simply Markdown files stored in an ordinary folder on your computer. They are not locked inside a proprietary database that only one application understands. Open your Obsidian vault in Finder or File Explorer and your notes are right there as .md files. You can open them with another text editor, move them, back them up, or manage them however you want. If you stopped using Obsidian tomorrow, your notes would still be yours and would still be readable.
Obsidian is also free to use. The features that make the application powerful, including plugins, themes, graph view, linking, and multiple vaults, are available without paying for a subscription. There are paid services such as Obsidian Sync for syncing your vault across devices and Obsidian Publish for publishing notes to the web, but they are optional conveniences. Check obsidian.md/pricing for current numbers, since they change.
\nAnd that brings me to this guide. I use Obsidian mainly for academic and research work, so that is the perspective I will use throughout this guide. You will see examples involving papers, methods, concepts, research projects, and ideas I want to revisit later. But there is nothing inherently academic about the system. The same approach can work for work projects, meeting notes, books, personal writing, recipes, or almost anything else you want to keep and connect.
\nI also do not want this to become one of those Obsidian tutorials where we install twenty-seven plugins, create fourteen folders, design an elaborate tagging taxonomy, and spend three hours building a system for taking notes before we have actually taken a note. We are going to start small. We will install Obsidian, create a vault, understand the few pieces of Markdown you actually need, create our first linked notes, and develop a simple structure for organizing them. From there, we can add templates, daily notes, plugins, backups, and some of the more powerful features, but only when there is a reason for them.
\nDownload Obsidian from obsidian.md for macOS, Windows, Linux, iOS, or Android. There is no account to create: the app opens straight into a prompt asking what you want to do. Choose Create new vault, give it a name, and pick where it lives on disk.
A vault is a folder. Inside it are .md text files and a hidden .obsidian/ subfolder holding your settings. That is the entire architecture. Open the folder in Finder or Explorer right now and you will see your notes sitting there as ordinary files.
Open your vault and you will see a mostly blank window. Here is what you are looking at.
\n
On the left, the file explorer lists every note in your vault; folders here are real folders on disk. In the middle, the editor is where you write. On the right, the graph view shows your notes as dots and their links as lines.
A handful of shortcuts cover most of what you will do:
\n| Shortcut | \nWhat it does | \n
|---|---|
Ctrl/Cmd + N | \nNew note | \n
Ctrl/Cmd + O | \nQuick switcher \u2014 jump to any note by name | \n
Ctrl/Cmd + P | \nCommand palette \u2014 every command, searchable | \n
Ctrl/Cmd + E | \nToggle editing / reading view | \n
Ctrl/Cmd + Shift + F | \nSearch across the whole vault | \n
Ctrl/Cmd + click | \nOpen a link in a new pane | \n
If you learn only one, make it Ctrl/Cmd + O.
Type three or four letters and the note you want is usually the top hit. Once you pass a few hundred notes this becomes how you navigate almost exclusively, and the file explorer turns into decoration.
\nObsidian notes are Markdown, but you rarely have to think about syntax. You type a few characters and the formatting appears live.
\n
The syntax you will actually use day to day is small:
\n# Heading 1\n## Heading 2\n\n**bold** *italic* `inline code`\n\n- bullet\n- [ ] unchecked task\n- [x] completed task\n\n> blockquote\n\n[[Link to another note]]\n[[Link to another note|shown as this text]]\n![[Embed another note entirely]]\n\n#tag #nested/tagTwo Obsidian-specific ones are worth calling out. The pipe in [[Note|display text]] lets the link read naturally in a sentence while still pointing at the right file. And ![[Note]] with a leading exclamation mark embeds the whole note inline rather than linking to it (useful for pulling a definition into a summary without duplicating it).
This is the part that matters. Create a note called Confounding and write a couple of sentences. Somewhere in the text, type two opening square brackets: [[.
A dropdown appears. Start typing G-computation. Even though that note does not exist yet, press Enter \u2014 Obsidian creates it the moment you click through.
Now open G-computation and look at the Linked mentions panel.
You typed one link. You got two. That is the whole trick, and it compounds: every time you link a new note to an old one, the old note gets richer without you touching it.
\nNew users spend their first week building an elaborate folder hierarchy and their second week discovering it does not fit. Obsidian gives you three organising tools, and they answer different questions.
\n
Folders answer where does this live? A note sits in exactly one. Use them for broad, stable categories. Three to six top-level folders is plenty. Tags answer what kind of note is this? Type #method or #toread anywhere in a note. A note can carry many, which makes tags good for status and type \u2014 things that cut across folders. Nested tags like #status/draft give you a little hierarchy without much cost. Links answer what is this related to? These do the real work. Folders and tags are coarse; links are specific.
The failure mode is over-investing in the first and under-using the third. A flat vault with good links is far more useful than a beautiful eleven-level folder tree with none.
\nHere is a starter structure:
\nmy-vault/\n\u251c\u2500\u2500 Daily/ # one note per day, capture inbox\n\u251c\u2500\u2500 Notes/ # the permanent, linked notes\n\u251c\u2500\u2500 Projects/ # one note per active project\n\u251c\u2500\u2500 Reading/ # books, papers, articles\n\u2514\u2500\u2500 Templates/ # reusable note skeletonsLet Notes/ grow flat and link heavily inside it.
Go to Settings \u2192 Core plugins and enable Daily notes. In its options, set the folder to Daily/ and the date format to YYYY-MM-DD so the files sort correctly.
The daily note is a capture inbox, not an archive. Anything that occurs to you during the day lands here with no organising: meeting notes, half-ideas, a paper someone recommended, a problem you hit in your analysis. The bar for writing something down should be almost zero.
\nLater, you promote the few things that earned it into real notes in Notes/, with claim-style titles and links. Most of what you capture will not earn it, and that is the point. The inbox absorbs the noise so your permanent notes stay signal.
Turning on the Calendar community plugin gives you the month view shown above, which makes it easy to jump back to \"that Tuesday when the model broke.\"
Enable the Templates core plugin and point it at Templates/. Then create a file there like this:
# {{title}}\n\n**Source:**\n**Date:** {{date}}\n\n## Summary\n\n## Key points\n-\n\n## Links\n-Now Ctrl/Cmd + P \u2192 \"Insert template\" drops that skeleton into any new note.
The payoff is not the typing you save. It is that every literature note has the same shape, so six months later you can skim thirty of them quickly, and a Dataview query can pull fields out of them reliably. Two or three templates is usually the right number: one for literature, one for meetings, one for projects.
\nCommunity plugins live in Settings \u2192 Community plugins \u2192 Browse. You will need to turn off Restricted Mode first.
\nResist installing twenty of them. Each is a thing that can break, conflict, or need migrating, and the most common way to end up with a broken vault and no notes in it is to spend week one configuring instead of writing. Add plugins to solve problems you have actually hit.
\nThe ones that earn their place for most people:
\n| Plugin | \nWhat it solves | \n
|---|---|
| Dataview | \nTurns notes into a queryable database | \n
| Calendar | \nMonth view for daily notes | \n
| Templater | \nTemplates with logic \u2014 dates, prompts, scripting | \n
| Excalidraw | \nHand-drawn diagrams stored inside the vault | \n
| Style Settings | \nTweak theme colours and spacing without CSS | \n
Dataview is the one that changes how the vault feels. Instead of maintaining an index note by hand, you write a query and it stays current.
\n
That query sits in a normal note. Every time you open it, Obsidian scans the Reading/ folder and rebuilds the table, so a paper you add next month appears without you editing anything.
Here is the honest truth about the graph view: it is beautiful, it is motivating, and it is useless for the first month.
\n
A graph of twelve notes tells you nothing you did not already know. A graph of four hundred shows you clusters you never planned, orphan notes you should link or delete, and hub notes that turn out to be the real organising centres of your thinking.
\nBuilding structure before content. You cannot design the right hierarchy for notes you have not written. Start flat and let structure emerge.
Installing too many plugins. Add them to solve problems you have actually hit, not problems a video told you about.
Treating it as an archive. A vault full of clipped articles you never linked or reread is a graveyard, not a knowledge base. One note you wrote in your own words beats ten you pasted.
Splitting into many vaults. Links do not cross vault boundaries. One vault, folders inside it.
Chasing the perfect system. Every month someone publishes a new methodology with an acronym. The people getting real value from Obsidian are mostly running something boring and consistent. Boring and consistent wins.
Over-tagging. If every note carries eight tags, tags stop narrowing anything. A handful you actually filter on beats a taxonomy you admire.
If you want something concrete to follow:
\nDays 1\u20132. Install, create one vault, make the five folders. Write three notes about things you already know well and link them to each other.Days 3\u20137. Turn on daily notes. Capture into them every day, without organising. Do not touch settings.Day 7. First review. Promote two or three captures into real notes. Notice what you keep reaching for.Week 2. Add a literature template. Install Dataview only if you feel the need for an index. Write a note that links to five existing ones.End of week 2. Open the graph for the first time. It will be small, and that is fine.Once the daily loop is automatic, the next steps are richer Dataview queries, Templater for dynamic templates, and if you want to publish, either Obsidian Publish or a static-site generator like Quartz, which renders a vault into a website for free.
\nBut none of that matters until the habit exists. After installation, everything else is elaboration. I hope this helps!
\nOpen Science und geistiges Eigentum werden oft als Gegens\u00e4tze verstanden: hier der freie Zugang zu Wissen, dort Patente, Urheberrechte und andere Schutzrechte. F\u00fcr den Transfer von Forschung in die Praxis greift diese Gegen\u00fcberstellung jedoch zu kurz. Die entscheidende Frage ist vielmehr: Was sollte offen sein, was muss gesch\u00fctzt werden \u2013 und wann?
\nDieser Frage wurde auf der Innovations-Roadshow des Deutschen Patent- und Markenamtes (DPMA) am 8. September 2026 in Hannover an der TIB nachgegangen.
\nForschungsergebnisse sind mehr als wissenschaftliche Publikationen. Sie entstehen als Daten, Software, Modelle, Videos, Prototypen oder technische Erfindungen. Damit stellen sich zugleich Fragen nach Zug\u00e4nglichkeit, Nachnutzung, Urheberrecht und gewerblichem Rechtsschutz.
\nWer Forschung offen zug\u00e4nglich machen und Innovation erm\u00f6glichen will, muss deshalb beides zusammendenken: Offenheit und Schutz.
\nEine wichtige Schnittstelle daf\u00fcr ist das Patentinformationszentrum (PIZ) Hannover an der TIB. Seit 1878 unterst\u00fctzt es dabei, technisches Wissen und Informationen \u00fcber Schutzrechte zug\u00e4nglich und nutzbar zu machen. Heute ist das PIZ das einzige Patentinformationszentrum in Niedersachsen und Kooperationspartner des Deutschen Patent- und Markenamts.
\nZu den kostenfreien Basisdiensten geh\u00f6ren Informationen und Fachliteratur zu Patenten und gewerblichen Schutzrechten, Unterst\u00fctzung bei Recherchen in Patent- und Schutzrechtsdatenbanken sowie Informationen zu nationalen und internationalen Schutzrechtsverfahren. Veranstaltungen und Sensibilisierungsangebote vermitteln zudem Grundlagen des gewerblichen Rechtsschutzes.
\nDabei versteht sich das PIZ Hannover als neutrale Lotsenstelle: Die kostenfreie Erfindererstberatung kann bei Bedarf zu IHK oder Patentanwaltschaft weiterf\u00fchren. F\u00fcr spezialisierte Recherchen, Analysen oder Patentstatistiken werden geeignete Anbieter und Einrichtungen vermittelt. Auch bei Fragen zu Verwertung, Schutzrechtsmanagement oder F\u00f6rderm\u00f6glichkeiten verweist das PIZ auf die jeweils zust\u00e4ndigen Stellen.
\nSo bietet das PIZ einen kostenfreien, neutralen und niedrigschwelligen Zugang zu Informationen und Orientierung im Netzwerk der Schutzrechts- und F\u00f6rderangebote.
\nF\u00fcr Forschende beginnt der Mehrwert oft schon vor einer Patentanmeldung: Welche L\u00f6sungen gibt es bereits? Was wurde schon ver\u00f6ffentlicht? Welche Entwicklungen sind neu \u2013 und wo gibt es Ankn\u00fcpfungspunkte f\u00fcr die eigene Forschung?
\nEine Patentrecherche kann dabei auch helfen, Unternehmen, Forschungseinrichtungen oder andere Akteure zu identifizieren, die an \u00e4hnlichen Technologien arbeiten und als m\u00f6gliche Kooperationspartner infrage kommen.
\nPatentinformationen k\u00f6nnen damit eine Br\u00fccke zwischen wissenschaftlicher Erkenntnis und technologischer Entwicklung bilden.
\nEin Beispiel f\u00fcr diese Verbindung ist die Patentdatenbank Orbit Intelligence. Die Plattform kann aus wissenschaftlichen Texten technische Merkmale extrahieren und dazu passende Patentdokumente identifizieren. Lizenziert f\u00fcr Nutzende der TIB. DBIS \u2013 Orbit Intelligence
\nSo l\u00e4sst sich etwa pr\u00fcfen, in welchen Patenten bestimmte Komponenten, Verfahren oder technische Zusammenh\u00e4nge beschrieben sind. Orbit unterst\u00fctzt damit die Recherche und Analyse \u2013 ersetzt aber keine rechtliche Pr\u00fcfung von Neuheit oder erfinderischer T\u00e4tigkeit.
\nF\u00fcr Forschende entsteht so eine praktische Br\u00fccke zwischen wissenschaftlicher Publikation und Patentinformation.
\nWie aus Patentinformation und Forschungsergebnissen konkreter Technologietransfer entstehen kann, zeigt das Beispiel von Bardia Akaberi der Firma Goldhoop.
\nAusgangspunkt war die Recherche nach dem Stand der Technik, auf die dann der Schutz der Erfindung als Gebrauchsmuster DE 20 2025 003 427 folgte. Doch der Transfer endete nicht mit dem Schutzrecht. Ein funktionierender Prototyp wurde umgesetzt und in einem Video dokumentiert. Denn ein Film sagt mehr als 1.000 Bilder: Die technische Idee wird anschaulich, verst\u00e4ndlich und f\u00fcr potenzielle Kooperations- und Transferpartner unmittelbar erlebbar.
\n\u00dcber das TIB AV-Portal (https://doi.org/10.5446/73021) bleibt die Dokumentation dauerhaft auffindbar und zitierf\u00e4hig. Im TIB-Portal \u00a0k\u00f6nnen Patent, Video und wissenschaftliche Informationen miteinander verkn\u00fcpft werden.
\nEin Schutzrecht allein macht eine Erfindung noch nicht sichtbar. Erst wenn eine technische Entwicklung verst\u00e4ndlich, auffindbar und dauerhaft dokumentiert ist, kann daraus Interesse entstehen.
\nSichtbarkeit \u2192 Verst\u00e4ndnis \u2192 Interesse \u2192 Kontakt \u2192 Technologietransfer
\nSo wird aus gesch\u00fctztem Wissen ein anschlussf\u00e4higes Forschungsergebnis \u2013 und aus Forschung kann Innovation entstehen.
\nDie TIB macht mit Ihren Services Digitale Dienste und Dienstleistungen der TIB \u00a0Wissen dauerhaft sichtbar, recherchierbar und transferf\u00e4hig. Das PIZ Hannover an der TIB unterst\u00fctzt Sie bei den gewerblichen Schutzrechten. \u00a0So entsteht eine Br\u00fccke zwischen Forschung, Schutz und Innovation. Sprechen Sie uns gerne an!
","doi":"https://doi.org/10.65527/hbjbb-28825","guid":"https://blog.tib.eu/?p=33999","image":"https://blog.tib.eu/wp-content/uploads/2026/10/2026-dpma.png","language":"de","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1791504000,"rid":"f73k7-csb39","summary":"Open Science und geistiges Eigentum werden oft als Gegens\u00e4tze verstanden: hier der freie Zugang zu Wissen, dort Patente, Urheberrechte und andere Schutzrechte. F\u00fcr den Transfer von Forschung in die Praxis greift diese Gegen\u00fcberstellung zu kurz. Die entscheidenden Fragen sind vielmehr: Was sollte offen sein, was muss gesch\u00fctzt werden \u2013 und wann?","tags":["SERVICES","Lizenz:CC-BY-4.0-INT","TIB AV-Portal","TIB-Portal","Patent"],"title":"Offen forschen, gezielt sch\u00fctzen","updated_at":1791617415,"url":"https://blog.tib.eu/2026/10/09/offen-forschen-gezielt-schuetzen/","version":"v1"},{"authors":[{"affiliation":[{"name":"Fachhochschule Potsdam"}],"contributor_roles":[],"family":"Kaden","given":"Ben","url":"https://orcid.org/0000-0002-8021-1785"}],"blog":{"authors":null,"community_id":"00dd7e11-a802-44c1-9584-5c56d1f8d417","created":1706832000,"current_feed_url":null,"description":"Vernetzungs- und Kompetenzstelle Open Access Brandenburg","doi":"https://doi.org/10.59350/oabrandenburg","favicon":"https://rogue-scholar.org/api/communities/00dd7e11-a802-44c1-9584-5c56d1f8d417/logo","feed_format":"application/atom+xml","feed_url":"https://open-access-brandenburg.de/feed/atom","filter":null,"generator":"WordPress","home_page_url":"https://open-access-brandenburg.de/","issn":null,"language":"deu","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"oabrandenburg","status":"active","subfield":"1802","title":"Open Access Brandenburg","updated":1791545849,"use_api":false},"blog_name":"Open Access Brandenburg","blog_slug":"oabrandenburg","content_html":"Es gibt ein relativ neu beschriebenes Ph\u00e4nomen im Open Access bzw. Diamond Open Access, \u00fcber das unl\u00e4ngst Nature berichtete und auf das wir ebenfalls kurz hinweisen wollen. So berichten Lisa Matthias vom Institut f\u00fcr Bibliotheks- und Informationswissenschaft der Humboldt-Universit\u00e4t zu Berlin und ihre Ko-Autoren Juan Pablo Alperin (Simon Fraser University, Vancouver) und Mikael Laakso (Tampere University, Tampere) \u00fcber F\u00e4lle, in denen wissenschaftliche Zeitschriften von einem Diamond-Open-Access-Modell auf APC-basierte Gold-OA, Hybrid-OA oder Subskriptionsmodelle umstellten:
\nLisa Matthias, Juan Pablo Alperin, Mikael Laakso: Diamond Fractures: Tracing Journal Transitions Away from Diamond Open Access. (30.06.2026) DOI: 10.48550/arXiv.2606.31302\n\nInsgesamt konnten sie 440 F\u00e4lle f\u00fcr den Zeitraum von 2009 bis 2026 identifizieren, wobei der gr\u00f6\u00dfte Anteil auf den Wechsel zu Gold Open Access mit Publikationsgeb\u00fchren zu verzeichnen ist. Interessanterweise beschleunigte sich diese Entwicklung in den vergangenen Jahren. Ebenfalls interessant erscheint, dass dies nicht etwa mit Verlags- oder Betreiberwechseln (\"ownership\") zu erkl\u00e4ren ist \u2013 die \u00fcberwiegende Zahl der Titel, 389 von 440 \u2013 blieb beim Ursprungsbetreiber.\nBei einem Teil der F\u00e4lle l\u00e4sst sich Diamond Open Access als \u00dcbergangs- und Etablierungsma\u00dfnahme nach der Neugr\u00fcndung einer Zeitschrift verstehen:
\n\n\n\"we see the pattern of publishers forgoing APC revenue as a promotional strategy, while a new journal builds indexing and reputation, then introduce them once the journal is established\".
\n
Der gr\u00f6\u00dfere Teil der Titel \u00e4ndert sein Betriebsmodell allerdings erst sp\u00e4ter, was mit dieser Deutung adressiert wird:
\n\n\n\"The temptation and relative ease of adopting APCs is particularly evident for these journals, given that such established journals would be strong candidates for collective funding models, such as Subscribe to Open.\"
\n
Besondere Relevanz erh\u00e4lt die Analyse deshalb, weil Diamond Open Access verst\u00e4rkt, wie auch zuletzt auf den Open-Access-Tagen 2026, als ein m\u00f6gliches Alternativmodell besonders zum geb\u00fchrenfinanzierten Gold Open Access angesehen und diskutiert wird und man daher sehr viel \u00fcber die Verschiebung von Gold zu Diamond spricht. Die entgegengesetzte Entwicklung wird dagegen bisher kaum thematisiert. Dank der vorgelegten Auswertung ist \"Diamond Fractures\" im Diskurs und auch in Nature angekommen. Zudem unterstreicht der Befund zus\u00e4tzlich den Bedarf an Diamond-OA-Modelle, die auf strategische Verstetigungen in nicht-kommerziell ausgerichteten Kontexten aufbauen.
\nWer mehr \u00fcber Diamond Fractures erfahren m\u00f6chte, hat im November die Gelegenheit. Denn am 27.11.2026 ab 18 Uhr und virtuell wird Lisa Matthias ihre Forschungen zum Thema \"Diamond Fractures\" im Berliner Bibliothekswissenschaftlichen Kolloquium an der Humboldt-Universit\u00e4t zu Berlin pr\u00e4sentieren: Hybrid und Diamond Open Access unter der Lupe \u2013 Open Access als Forschungsobjekt der Bibliotheks- und Informationswissenschaft.\u00a0
\n\n\nZitierhinweis:
\nKaden, Ben (2026): \"OA-News: Diamond (Open Access) Fractures als Forschungsthema.\" DOI: 10.59350/yq0vn-j2231
\n