![\"\"](\"https://substackcdn.com/image/fetch/$s_!ciEX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb0530b4d-0b78-4409-adee-31a9dd7e9389_1134x651.png\")
This post is part of a \"hot takes\" series in which I make sharper claims than I usually do. I do not intend to offend, and I am not trying to tar every librarian with the same brush \u2014 the patterns I describe and perceive may be a function of my own local context.
In my last hot takes post, I argued that while designing a search system to maximise learning gains may not always align with designing a search system that scores the best for relevancy, unplanned friction in learning aka poor relevancy is never a good idea.
In this post, I consider the idea of tools that are considered flawed because they can't match the performance of an older tool in some way1.
Imagine you are a librarian in 2004, when Google Scholar launches in beta.
You have read the studies. The coverage gaps are real. The metadata is wretched. Nobody, including Google, can tell you exactly what is indexed2.
You decide it is the dumbest thing on the planet and declare that it could never be useful to anyone, despite a vocal minority of users insisting otherwise.
Fast forward to 2024. Google Scholar has become the default academic search starting point for many researchers and is widely regarded as the most comprehensive free academic search engine3. Your first assumption might be that Google fixed all the problems.
But that assumption would be mostly wrong.
Sure, Google Scholar improved, especially in coverage. Its metadata may have arguably also have improved. But you still cannot consult a stable, auditable list of indexed sources or records. Even Google's own guidance effectively tells users to test coverage by sampling titles rather than by checking a definitive inventory. One of the fundamental weaknesses librarians diagnosed in 2004 is still there.
And yet the librarians and researchers of 2024 are not idiots.
What happened is simple. Users learnt to compensate. They used Scholar for what it was good at and routed around its flaws or used another tool for other use cases (e.g. Evidence synthesis). They learnt when its metadata could not be trusted, when its coverage was opaque, and when to complement with another database was needed. The tool did not become perfect. It became useful enough, along side other tools.
![\"\"](\"https://substackcdn.com/image/fetch/$s_!_kqs!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F50eeb872-ee6d-44db-b0e5-9f73776df674_999x632.png\")
That trajectory is suprisingly common with new technology and one I think about when I read arguments thay say AI or any technology tools can never be useful because of some \"fundamental flaw\".
![\"\"](\"https://substackcdn.com/image/fetch/$s_!z3Ha!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4127aecf-6c4d-4cd2-80e4-29958b76726f_1758x894.png\")
Some of the most prominent sceptics have already said as much.
Gary Marcus, cognitive scientist, author of Rebooting AI, and one of the most consistent public critics of AI hype, has repeatedly acknowledged that LLMs can be useful, especially for coding, brainstorming and writing, while arguing that they are unreliable and not a route to AGI alone.
Margaret Mitchell, a co-author of the influential \"Stochastic Parrots\" paper, has been even more explicit: LLMs can be \"extremely useful\".
Emily M. Bender has likewise clarified that \"stochastic parrot\" is a description or metaphor, not an argument that these systems have no possible utility4.
Mike Caulfield, of SIFT fame, is actively studying and using these tools. These are not AI boosters. If even they concede that the tools can be useful, the basic question has already been settled.
You can oppose LLM use on environmental grounds, labour grounds, epistemic grounds, or any number of other defensible grounds.
But the claim that these tools can never be useful to anyone has moved past argument into something closer to an unfalsifiable position. No demonstration of utility, no improvement in the tools, and no evidence of successful use by researchers seems able to update it.
For the rest of this post, when I say \"AI tools\", I mean AI-powered academic search tools. I work in this space, so I will stay in my lane. The argument may extend elsewhere, but I am not pretending to make that case here.
It is also worth being precise about what \"AI search\" means. The term covers several different things: changing what gets retrieved, reranking results, summarising content, and generating direct answers to questions. These are not the same capability and do not carry the same risks.
A librarian who objects to generative answer synthesis is making a different argument from one who objects to AI-assisted reranking. Conflating the two muddles the debate. Before objecting to \"AI search\", it is worth saying which part concerns you, and why.
The Google Scholar analogy maps most cleanly onto AI-assisted retrieval and reranking: helping surface relevant results that users might otherwise miss. It also maps reasonably well onto \"tip-of-the-tongue\" search, one of the limited uses Bender has acknowledged as potentially useful.
It maps less directly onto generative answer synthesis, where hallucination risks are sharper. I am not arguing that all uses of AI in search carry equal risk. I am arguing that even the riskiest versions clear the \"never useful\" bar. The appropriate response to different risk profiles is differentiated teaching, not blanket rejection.
The people who insisted in 2004 that Google Scholar could never be useful until Google published full holdings lists were sure they were right. But they were eventually proven wrong to conclude that a tool could never be useful without that.
A tool need not be perfect to be useful. This sounds obvious, but it is the point that keeps getting lost.
One objection is that the Scholar analogy fails because LLM errors are different. Google Scholar had messy metadata and opaque coverage. LLMs produce overconfident hallucinations.
That objection has force, but notice what it actually supports. It supports teaching verification skills. It supports scaffolding. It supports appropriate scepticism. It does not support the early conclusion that the tool is useless.
A tool that requires careful handling is not the same as a tool that cannot be useful. The verification argument also cuts both ways. Uncritical acceptance of search results, including Google Scholar results, has always been the failure mode librarians teach against.
Perhaps the scaffolding still will not be enough. But I am sure many librarians who rejected Google Scholar in 2004 were pretty sure too.
Nor does it help that some of the most vocal sceptics seem not to have engaged seriously with these tools since 2023. They appear to underestimate how much the systems, and the harnesses around them, have changed even just three years. But even that point is secondary. The Scholar parallel does not depend on the exact pace of improvement5.
I currently believe from some experience setting up and testing agents, that between the use of code to constrain the model and aggressive multi-validation checks, you can reduce the probability of error/hallucatins down to very low levels comparable to the human.
The lesson is not that tools improve. The lesson is that what counts as good enough is not always obvious at first.
Some librarians argue that the Google Scholar comparison to AI breaks down on three grounds: librarians do not actively promote Scholar (but librarians actively promote AI); users genuinely want Scholar rather than having it pushed on them; and Scholar is free, whereas many AI tools are commercial products.
None of these objections does the work required.
On promotion, many librarians do promote Google Scholar. It appears in LibGuides, instruction sessions and one-on-one consultations. The claim that \"we do not promote it\" is a polite fiction. Beyond what is said publicly, plenty of academic librarians reach for Google, Scholar or Wikipedia first in their own work when the situation calls for it.
On user demand, users clearly want AI tools. Whatever one thinks of that demand, pretending it does not exist is not a serious position.
On cost, being free or paid is a separate question from whether a tool can be useful. There are genuine concerns about commercial AI: vendor lock-in, inequity of access, environmental cost, labour implications, surveillance, and the commercial capture of scholarly infrastructure. Those concerns deserve serious engagement.
But they are arguments about adoption, governance and institutional support. They are not arguments that the tools can never be useful.
And let us be honest about the comparison. Library databases have plenty of flaws: idiosyncratic interfaces, uneven indexing, opaque relevance ranking, and sometimes weak metadata. We still pay substantial sums for them and promote them as a matter of course. The objection to AI tools cannot simply be that they are commercial and imperfect, because by that standard half the collection budget becomes difficult to defend.
There is a related claim that deserves a direct response: librarians who teach users how to use AI search tools are abusing professional trust because the tools are imperfect and can lead to errors.
This is a bad argument dressed up as an ethical one. It rests entirely on the premise that AI search tools are imperfect, as though that distinguished them from anything else we teach.
e.g. Name the flawless tool, we libraries promote?
Our job, properly understood, is to teach people to use imperfect tools well, with appropriate scepticism and a clear understanding of what each tool can and cannot do. Refusing to teach AI search tools because they are flawed is not an act of professional integrity. It is an abdication of the actual job.
It also leaves users to figure these tools out on their own. That is the worse outcome by every measure.
A common argument among librarians goes something like this: Emily Bender says LLMs are stochastic parrots; therefore, LLMs can never be useful.
There is an immediate problem with that argument. Even if you accept the \"stochastic parrot\" description, it does not tell you whether LLMs combined with other technologies can be useful. It says nothing directly about retrieval-augmented generation, tool use, calculators, citation validators, structured workflows, human review, or other harnesses wrapped around the model.
The more damaging problem is that Bender herself has clarified that \"stochastic parrot\" is not an argument that LLMs are useless. In her account, it is not even an empirical hypothesis. It is a description or metaphor for systems that generate fluent linguistic form without grounding in communicative intent, a model of the world, or a model of the reader's state of mind6.
This does not mean Bender thinks LLMs are broadly useful. Her position is far more sceptical than that. She has warned that synthetic text is not an information source, and that using chatbots as reliable sources of knowledge is a serious category mistake.
But she has acknowledged limited possible uses, including \"tip-of-the-tongue\" search, language-learning dialogue partners, non-player characters in games, and non-generative uses of language models in classification, speech recognition and machine translation. She treats summarisation more cautiously, because it can introduce material not present in the source.
Nor does this mean Bender has retreated from the stronger \"form versus meaning\" argument. In Bender and Koller's 2020 paper, understanding is defined as mapping language to something outside language. Their claim is that a system trained only on linguistic form has no basis for learning that mapping, because it has access only to patterns in text, not to the extra-linguistic world those texts are about.
That is a serious argument. But it should not be flattened into the much weaker claim that LLMs can never be useful.
So the better conclusion is not \"stochastic parrots can never be useful\" (though she is currently very skeptical). It is: do not mistake fluent synthetic text for grounded understanding or reliable information. That is a much narrower, stronger, and useful warning but does not address the question on usefulness.
But it leaves room to ask the question that actually matters for librarians: under what conditions, with what scaffolding, for which tasks, and with what verification, can LLM-based systems be made useful rather than misleading?
The lesson from Google Scholar is not that librarians should embrace every flawed tool users like. It is that \"flawed\" and \"useless\" are not synonyms.
It is hard to compare like for like, but I think it is fair to say that the practical gains in LLM-powered tools from 2023 to 2026 have been faster and larger than Google Scholar's improvements across its first decade. But the more important point is not the scale of improvement. It is that Google Scholar's improvement did not fundamentally fix its transparency problem. Instead, users learnt that the flaw was either less fatal than it first appeared, or manageable with the right habits.
That is the lesson librarians need to take seriously now.
If the objection is environmental cost, make the environmental argument. If it is labour exploitation, make the labour argument. If it is vendor lock-in, inequity, surveillance, weak governance, or commercial capture, make those arguments. They are serious enough to stand on their own.
They do not need a backdoor return to the claim that AI tools cannot really be useful.
That move is increasingly unconvincing. It usually begins with a reluctant concession: of course AI can sometimes be useful. Then, when the discussion turns to teaching, adoption or institutional support, the old premise quietly reappears. The tools are flawed, so using them must be irresponsible.
But that is not how librarians treat tools.
We teach flawed systems all the time. We teach Google Scholar while warning about coverage and metadata. We teach Scopus and Web of Science while explaining their selectivity. We teach discovery layers while knowing their indexing and ranking are imperfect.
The professional act is not pretending tools are flawless. It is teaching people where they help, where they fail, and how to verify what matters.
So reject an AI tool because the cost is too high, the governance is too weak, the evidence is too thin, or the institutional incentives are wrong.
Just say that.
Do not dress those objections up as proof that the tool can never be useful. That argument has already lost.
The question now is not whether AI search tools can be useful. It is which uses are worth the cost, which are not, and what role librarians should play in helping users tell the difference.
There are many similarities to the Innovator's Dilemma argument. Early users may value dimensions that experts discount, and a tool that performs poorly against established professional criteria may still become useful enough to reshape practice. But unlike the Innovator's Dilemma, I refer to cases where the alternatives co-exist.
Google Scholar offered no stable, auditable list of indexed sources or records, and even Google's own guidance effectively tells users to test coverage by sampling titles rather than consult a definitive inventory.
As confirmed by many studies. The amount of full-text indexed by Google Scholar is also believed by many to be unmatched.
To be clear, Bender is not being cited here as making the same claim as Marcus or Mitchell that LLMs are broadly useful. Her position is much narrower and more sceptical. In interviews, She has said that safe and beneficial uses of synthetic text are hard to identify, but has offered tentative examples especially \"tip of the tongue\" search, where a user describes something in order to recover the name of it and can then verify it through ordinary search. She also distinguishes text generation from other uses of language models, saying that language models can have positive uses in classification, automatic speech recognition, and machine translation, while treating summarisation as more borderline because it can introduce material not present in the source. The point here is therefore not that Bender endorses LLMs as broadly useful, but that \"stochastic parrot\", in her own account, is not an empirical hypothesis or an argument that LLMs have no possible utility. It is a description or metaphor for language-mimicking systems, and the 2021 paper was about the risks and harms of pursuing ever-larger language models, not a general paper about \"AI\".
The improvement of LLMs between 2022 to 2026 is far larger than from 2004 to 2024! This improvement comes from both improvements in the models as well as the use of harnesses like Claude Code to combine deterministic code with LLM flexibility.
To be clear, this does not mean Bender has retreated from the stronger \"form versus meaning\" argument. In her account, the closest thing to an argument in this area is Bender and Koller's 2020 paper, which defines understanding as mapping language to something outside language. Their claim is that a system trained only on linguistic form has no basis for learning that mapping, because it only has access to patterns in text, not to the extra-linguistic world those texts are about. This is separate from the \"stochastic parrots\" phrase itself, which Bender describes as a metaphor rather than an empirical hypothesis. She also notes that multimodal systems complicate the picture: image-text models may meet the Bender and Koller definition of understanding in a very thin sense, because they can map between linguistic strings and images. But she argues that the stochastic-parrot framing remains relevant to such systems and to systems built around them.
Note to readers: some of these ideas made it into a commentary I wrote with Christine Cuskley:
\n\n\n\nDingemanse, Mark & Cuskley, Christine (in press). For robust research, center values, not technology. Behavioral and Brain Sciences. Preprint doi: 10.5281/zenodo.18944023
\n\n\n\nOne topic that often comes up when discussing LLM technology in relation to research integrity is one that I will describe as seeking permission. When looking at the ethical, legal, and societal harms imposed by LLMs (and there are many), sometimes people feel the message ends up altogether too negative. How about this use case I heard of? Aren’t some people getting something useful out of it? Surely we can’t ban the tech outright?
\n\n\n\nOften this is phrased as a concern about messaging (people won’t accept it if we tell them how bad it is; you need to sugarcoat it by also mentioning something nice). Sometimes it is phrased as a majority argument: it’s already here, everyone is using it, surely it can’t be that bad? (Smoking would like a word.) Sometimes it is a concern about missing the boat: these are skills we need, telling people not to use it is like telling them to go back to quill pens. 1
\n\n\n\nWhat I think is going on in these kinds of cases is that the central thrust of the argument is being missed. When it comes to research integrity, the key is a values-first perspective rather than a tech-first perspective.
\n\n\n\nA values-first perspective asks: how can we best uphold the values and standards that make our research robust, reproducible and future proof? A tech-first perspective asks: yeah, but how can I use this technology? It puts technology above values. It seeks permission but sidesteps the question of values.
\n\n\n\nOne example that came up in a recent session started with literature review: surely it’s not too harmful, a questioner said, to use an LLM for a first stab at a literature review? Multiple participants pushed back on this, saying that actually, LLMs don’t reliably summarise. Also, they provide only the most average consensus view; we don’t know what’s being left out. And LLMs by nature regurgitate without understanding; can we actually identify confidently produced bullshit in a field that we don’t master fully? Further, reading is a hard-won skill: tracing arguments, spending time with papers, separating substance and rhetorics; surely we don’t want to lose this skillset.
\n\n\n\nA retreat followed: well then, maybe literature review was not the best example, but surely there is some other kind of use that is okay? Someone mentioned programming. Folks brought up deskilling, the security risks of vibe-coding, technological debt. Oh well, that’s not the kind of programming I meant. And so on.
\n\n\n\nWe can play this game all day: seek permission for an edge case, retreat to another case when trouble arises, appeal to amiability or tolerance. Two things to note when a conversation gets to this point. First, it’s a misrepresentation: reluctance to grant permission is portrayed as an intolerant move, a wish to take the other’s toys, when it is nothing like that. What do you need my permission for, anyway? Second, it’s a distraction: it moves the conversation away from values and towards a tech-first perspective.
\n\n\n\nThis is where a values-first perspective can help cut through the knots. The approach is not to recommend or forbid particular tech products, or particular uses; it is to provide a matrix for mindful choice. For any attested or conceivable use, you can ask: how would this help or hinder my upholding of high standards of research integrity? If we consider the pushback voiced in response to the ‘literature review’ use case, you can see they appeal to the same core values:
\n\n\n\nThese values do not come out of thin air; they’re straight from a widely adopted code of conduct for research integrity (NCCRI, 2018). They are also not controversial; most scientists will recognise them as principles that characterize robust research. I’ve written about them before, e.g. in my guidance on GenAI and in my post on why synthetic text has no place in science blogging.
\n\n\n\nIf you use these values as a compass to steer by, it’s easier to navigate the landscape of technology use. On the other hand, if you find yourself seeking permission, one useful thing to do is to step back and inspect the underlying value conflict.
\n\n\n\nAs you move from a tech-first to a values-first perspective, the question shifts from “won’t you give me permission?” to “how do I do the best science possible?”. And that, to me, is a question worth asking.
\n\n\n\nIn language, semantics describe the names and meanings of words. The\nbioinformatics community has aptly adopted biosemantics as a concept that\nencompasses the issues with the names and meanings of biological entities,\nusually in natural language processing and data integration. However, semantics\ndoes not capture the context of words, and biosemantics fails to describe the\nbiological context and complex relationships between biological entities.
\n![\"Semantics](\"https://pediaa.com/wp-content/uploads/2018/08/Difference-Between-Semantics-and-Pragmatics_Figure-1.png\"/)
Pragmatics goes beyond semantics and describes the context of words. Because of\nthis parallelism, I've begun to use the term biopragmatics to describe the\nfamily of computational approaches aimed at identifying and contextualizing the\ncontext of biological entities.
","doi":"https://doi.org/10.59350/j4vty-rrr02","guid":"https://cthoyt.com/2020/01/22/biosemantics-versus-biopragmatics","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1579651200,"rid":"6qadr-jvy42","summary":"In language, semantics describe the names and meanings of words. The bioinformatics community has aptly adopted biosemantics as a concept that encompasses the issues with the names and meanings of biological entities, usually in natural language processing and data integration. However, semantics does not capture the context of words, and biosemantics fails to describe the biological context and complex relationships between biological entities.","tags":["Semantics","Meta"],"title":"Biosemantics vs. Biopragmatics","updated_at":1781539932,"url":"https://cthoyt.com/2020/01/22/biosemantics-versus-biopragmatics.html","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"How many molecular biology papers have you read today? This week? This month? If\nyou're like me, its not so many, and we're falling behind very quickly. Here's a\nchart made by the new PubMed that summarizes how many papers were published\nmentioning RAS in the last 50 years.
\n![\"RAS](\"https://cthoyt.com/img/ras_pubmed_history.png\"/)
There were 4,483 publications listed in 2019. We can't read that much, and even\nif we did, we couldn't remember it all. That's why we need to take the knowledge\nout of the unstructured text and store it in a structured form that can be read\nand stored in computers. This way, we can easily share it, query it, and write\nalgorithms that can help us reason about the incredible amount of biological\nknowledge out there.
\nThere are several formats in which this kind of information can be stored on a\ncontinuum between directly representing mechanistic biology to representing the\nknowledge itself. In the popular middle ground are BioPAX and BEL, which I'll\ncome back to in future posts.
\nIt's important to keep in mind that knowledge needs to be curated - this can\neither be manual, through natural language processing, or a mixture of both.\nI've written\na paper\non such a process, but for now this post should motivate a few following ones\ndescribing what it takes to deal with nomenclature, build ontologies, and then\nstart extracting mechanistic biology from the literature.
","doi":"https://doi.org/10.59350/jt101-b1374","guid":"https://cthoyt.com/2020/01/23/encoding-biology-in-kgs","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1579737600,"rid":"qyhqj-cs605","summary":"How many molecular biology papers have you read today? This week? This month? If you're like me, its not so many, and we're falling behind very quickly. Here's a chart made by the new PubMed that summarizes how many papers were published mentioning RAS in the last 50 years.","tags":["Knowledge Graphs"],"title":"Encoding Biology in Knowledge Graphs","updated_at":1781539915,"url":"https://cthoyt.com/2020/01/23/encoding-biology-in-kgs.html","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"The other day I saw a tweet lamenting the drag that is literature review during\npreparation for writing your thesis.
\n\nI just love writing 15 page literature reviews for graduate school courses on literally any topic except my thesis topic.
\u2014 PhD Diaries (@thoughtsofaphd) February 7, 2020
I agree. I felt the same pain last fall when I wrote\nmy doctoral thesis. Luckily, I had\na strategy that made it a bit easier.
\nI learned it from one of my professors when I was doing my master's degree in\nLife Science Informatics. Each semester, we had a seminar course in which each\nstudent was assigned research articles to read and present to the class with a\nshort slide deck. Later, I joined his research group and realized that this\ncourse served as a literature review for him just as much as us.
\nSo later when I was a Ph.D. student, I volunteered to run the seminar. I\nco-opted the concept, and planned the course to cover many of the topics I found\ninteresting for my thesis. I already knew some of the papers very well, and a\nfew were ones I had always been meaning to read. I tried to pick the most recent\npapers for topics when possible, but also threw in a few classics as well.
\nOn the first day of the seminar, I shared the following course information. I\nthought it was important to make clear what my expectations were for students in\nterms of their prior knowledge. Since they all came from the same master's\nprogram, I thought it was enough that they had passed one of the first semester\nlectures called \"Biological Databases\" which was about many of the resources and\ndatabases used in the systems and networks biology community. I also outlined\nwhat was the content for the course, what was expected, etc. then shared this\nall as a Google Doc so they could read it over and add comments.
\nI also made a list of possible papers and a tentative schedule that students\ncould look over and decide which papers they found most interesting. The topics\nwere arranged in a logical order to tell the story of my thesis, and for each\nsection there were a few papers that I thought were very important, and a few\nextras just in case there was a lot of interest. During the first day of the\nseminar, I also went through the list of all papers and explained the topics to\nthe students. I gave them this list via Google Docs as well, and they were able\nto claim papers for their presentations. Below, I've listed the final list of\npapers and the order in which they were presented. We were able to come to\nagreements for all students to present the papers I found most important. Maybe\n40% of the class found a paper interesting and picked one the first day and the\nrest took the next week to decide, ask questions, or propose new papers.
\nAnother consideration I had when picking this paper list was to choose work done\nby my colleagues that I found interesting and helpful. After, I invited them to\ncome listen to the seminar and mediate discussion after. We were able to invite\none of my collaborators Mehdi Ali (he's a really good guy!) to discuss his work\non using deep learning for relation extraction in natural language processing. I\nthink that might have been the most engaging day of the whole series.
\nI added one aspect to this course compared to the previous seminar that I had\nattended: each student was not only responsible for presenting the paper that\nhad been assigned from my list, but they were also responsible for finding a\nrelevant pre-print (in the same or similar topic) and submitting a peer review\nthrough the pre-print system. When I was a student, I noticed many students did\nnot read the references of the paper they were assigned in our seminars, and\nalso had not considered other similar research to their paper. Asking them to\nfind their own papers was a way to make this a more creative and fun process,\nand would directly prepare them to answer questions at the end of the\npresentation like \"what will the authors do next?\" or \"how will this research be\nused by others?\"
\nOne of the funny things that happened during the pre-print presentations is the\nstudents found several of mine and presented those. I suppose this was\ninevitable given the contemporary nature of my work in the context of the topics\nchosen. I would actually explicitly encourage students to check out my\npre-prints the next time I host a seminar, because I know the work very well and\ncould mediate a nice discussion.
\nI learned a lot through the process of preparing this seminar. Its outline\nbecame the outline for my thesis, and a lot of the discussions became points\nthat I addressed explicitly in my writing. I wouldn't say that I was taking\nadvantage of the students in this process - we all benefited from the\nexperience. I hope you get some ideas about how you might be able to do this\nyourself, whether you're a doctoral student, a postdoc, or something else!
\nStudents should be comfortable with the material presented in the Biological\nDatabases lecture during the first semester of the LSI curriculum.
\nStudents will have the opportunity to practice reading, presenting, and\ndiscussing recent biomedical literature on the topics of knowledge assembly,\ndata integration, and modeling in systems and networks biology.
\nStudents will be assigned papers and present on the holistic process of\nknowledge discovery in systems and networks biology that focus on the topics of\nknowledge assembly (e.g., natural language processing, modeling formalisms and\nformats, reasoning techniques), data integration (e.g., practical scenarios\nfocusing on techniques on the data level, knowledge level, and analytical\nlevels), and modeling strategies (e.g., rule-based modeling, agent-based\nmodeling, mathematical modeling, hypothesis generation with knowledge-based\napproaches).
\nStudents will be assigned an article to read and present during a thirty (30)\nminute lecture. One goal of this lecture is to show an understanding of not only\nthe material presented in the article, but also the relevant background\ninformation - this may entail following the references and reading other\narticles. Another goal is to not only educate, but entertain the audience.\nStudents will also be expected to find a relevant pre-print article on arXiv,\nbioRxiv, or other pre-print server and post a peer-review for the author on the\ncorresponding service. Following the presentation of their assigned article,\nstudents should include slides (1-3) briefly explaining the relevance of the\npre-print that they found.
\nStudents will be assessed on the understanding of their assigned topic, the\nquality of their presentation, and their participation. Students missing more\nthan 2 seminars will not pass the course without a doctor's note.
\nThis week there will a short discussion of the syllabus and no presentation. For\nthose in Bonn that aren't aware of this wonderful tradition, welcome to Syllabus\nWeek.
\nMubassher Leser, U., & Hakenberg, J. (2005).\nWhat makes a gene name? Named entity recognition in the biomedical literature.\nBriefings in Bioinformatics, 6(4), 357\u2013369.
\nPre-print: https://www.biorxiv.org/content/early/2017/03/08/115022
\nBachman, J. A., Gyori, B. M., & Sorger, P. K. (2018).\nFamPlex: A resource for entity recognition and relationship resolution of human protein families and complexes in biomedical text mining.\nBMC Bioinformatics, 19(1), 1\u201314.
\nPre-print: https://www.biorxiv.org/content/early/2018/07/29/379446
\nLaibe, C., & Le Nov\u00e8re, N. (2007).\nMIRIAM Resources: tools to generate and resolve robust cross-references in Systems' Biology.\nBMC Systems Biology, 1, 58.
\nPre-print: https://www.biorxiv.org/content/early/2017/07/24/167619
\nJuty, N., Le Nover\u0300e, N., & Laibe, C. (2012).\nIdentifiers.org and MIRIAM Registry: Community resources to provide persistent identification.\nNucleic Acids Research, 40(D1), 580\u2013586.
\nPre-print: https://www.biorxiv.org/content/early/2018/02/14/101279
\nNovichkova, S., et al. (2003).\nMedScan, a natural language processing engine for MEDLINE abstracts.\nBioinformatics, 19(13), 1699\u20131706.
\nPre-print: https://www.biorxiv.org/content/early/2018/08/29/403667
\nAli, M., et al. (2017).\nAutomatic Extraction of BEL-Statements based on Neural Networks.\nProceedings of BioCreative VI Challenge and Workshop, (October).
\nPre-print : https://osf.io/j76y3/
\nDemir, E., et al. (2010).\nThe BioPAX community standard for pathway data sharing.\nNature Biotechnology, 28(12), 1308\u20131308.
\nPre-print: https://www.biorxiv.org/content/10.1101/192856v1
\nHucka, M., et al. (2003).\nThe systems biology markup language (SBML): a medium for representation and exchange of biochemical network models.\nBioinformatics (Oxford, England), 19(4), 524\u201331.
\nPre-print: https://www.biorxiv.org/content/early/2018/01/23/246470
\nLe Nov\u00e8re, et al. (2009).\nThe Systems Biology Graphical Notation.\nNature Biotechnology, 27(8), 735\u201341.
\nPre-print: https://www.biorxiv.org/content/early/2018/01/30/256750
\nCarbon, S., et al. (2017).\nExpansion of the gene ontology knowledgebase and resources: The gene ontology consortium.\nNucleic Acids Research, 45(D1), D331\u2013D338.
\nPre-print: https://www.biorxiv.org/content/early/2018/10/07/437020
\nCroft, D., et al. (2014).\nThe Reactome pathway knowledgebase.\nNucleic Acids Research, 42(D1), D472\u2013D477. AND Fabregat, A., et al.\n(2018).\nThe Reactome Pathway Knowledgebase.\nNucleic Acids Research, 46(D1), D649\u2013D655.
\nPre-print: https://www.biorxiv.org/content/early/2018/10/09/375097
\nCerami, E. G., et al. (2011).\nPathway Commons, a web resource for biological pathway data.\nNucleic Acids Research, 39(SUPPL. 1), 685\u2013690.
\nPre-print: https://www.biorxiv.org/content/early/2018/09/03/353235
\nKhatri, P., Sirota, M., & Butte, A. J. (2012).\nTen years of pathway analysis: Current approaches and outstanding challenges.\nPLoS Computational Biology, 8(2).
\nPre-print: https://www.biorxiv.org/content/early/2018/09/13/416131
\nGligorijevi\u0107, V., & Pr\u017eulj, N. (2015).\nMethods for biological data integration: perspectives and challenges.\nJournal of The Royal Society Interface, 12(112), 20150571.
\nPre-print: https://www.biorxiv.org/content/early/2018/12/13/358390
\nSaqi, M., et al. (2018).\nNavigating the disease landscape: knowledge representations for contextualizing molecular signatures.\nBriefings In Bioinformatics, (May), 1\u201315.
\nPre-print: https://www.biorxiv.org/content/early/2018/11/23/475202
\nHimmelstein, D. S., et al. (2017).\nSystematic integration of biomedical knowledge prioritizes drugs for repurposing.\nELife, 6.
\nPre-print: https://www.biorxiv.org/content/early/2018/10/13/442640
\nLopez, C. F., et al. (2013).\nProgramming biological models in Python using PySB.\nMolecular Systems Biology, 9(646), 646.
\nPre-print: https://www.biorxiv.org/content/early/2018/12/23/503359
\nGyori, B. M., et al. (2017).\nFrom word models to executable models of signaling networks using automated assembly.\nMolecular Systems Biology, 13(11), 954.
\nPre-print: https://www.biorxiv.org/content/early/2018/05/15/322156
","doi":"https://doi.org/10.59350/4t3sf-mab09","guid":"https://cthoyt.com/2020/02/09/seminar-for-thesis-writing","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1581206400,"rid":"9h7sp-08z86","summary":"The other day I saw a tweet lamenting the drag that is literature review during preparation for writing your thesis.","tags":["Doctoral Thesis","Teaching"],"title":"Host a Graduate Seminar Before Writing Your Thesis","updated_at":1781539913,"url":"https://cthoyt.com/2020/02/09/seminar-for-thesis-writing.html","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"We've all been there. You started a new branch from master. You had a very\nspecific goal in mind, The Original Goal. You made a pull request (PR) to go\nwith it, too, The Original Pull Request. But then, you had an idea! And\nalso, someone on your team asked you to solve another problem! Now the original\ncode you wrote to address The Original Goal relies on that code \u2026 and now\nyou've got dozens of files changed, hundreds of lines of diff, and nobody\n(including you) can understand what you've done. Like I said, we've all been\nthere. Here's what you can do to fix it:
\nDon't do anything rash. Git is a pain to use, and you're going to have to rely\non it to keep a history for you of what you've done.
\nFirst, you're going to have to take a big step back. Write a summary of all the\nthings you've done in The Original Pull Request. This should be about what\nthe PR does and why it does it. Of course it could vary depending on the\nsituation, but this summary shouldn't be about exactly how the PR does it,\nbecause the implementation details are likely what lead to this situation in the\nfirst place.
\nKeep in mind that every PR has a box at the top that's used to describe what's\nin it. This is where you will put your summary.
\nOf all the things that The Original Pull Request, some of them are\nself-contained, and some of them rely on each other. It was probably the case\nthat to accomplish The Original Goal, you had to address lots of smaller\ngoals. You probably also had to change lots of code and write new code too.
\nWouldn't it have been nice if all of these implementations were already done,\nbecause then you could have just solved The Original Goal directly by\nusing/applying previous code. That's what we're going to aim for.
\nBut first, you need to figure out which things you did relied on which other\nones, because you're going to break The Original Pull Request up until it\nexactly matches up to addressing The Original Goal. don't have any
\nAfter you understand which parts of The Original Pull Request depend on each\nother, pick one independent part of the code that accomplishes one sub-goal.\nSince you're not doing this to be a martyr, and we all know git is too\ncomplicated to Do It Right, you're going to copy/paste the files that are\nrelated to this change to your desktop*.
\nBefore continuing, you're going to make sure all of the code in your big messy\nbranch for The Original Pull Request is committed and pushed. Even though we\nwant to supersede what's there, it never hurts to keep track of your descent\ninto madness.
\nAfter there's nothing lying around, switch back to master. If your team has\ntaken good care of your repository, the master branch should be undisturbed by\nthe chaos you've created in The Original Pull Request. Make a new branch\nfrom master, and name it appropriately for fixing the one sub-goal, from here\nout known as The Sub-Goal that you identified in Step 4. Now you can start\nupdating the relevant files in your repository based on the files you copied to\nyour desktop. I suggest you don't copy/paste the contents of the whole files,\nbecause you might have forgotten about something else you changed in them. After\nall, you're reading my guide because this was a mess.
\nOnce you've finished making the new branch for your independent part of code\nthat solves The Sub-Goal, you can make The New Pull Request.
\nYou will now go through the entire process of writing a good summary of this\nbranch for your co-developers, you will get their feedback, you will make\nupdates, pass flake8, and so on. They will thank you for having code that\naccomplishes one thing, and can be described simply. They will thank you for not\nhaving too big of a diff, and for the things in the diff all being relevant and\nimportant. Then you can merge this branch into master.
\nThroughout transferring the code for The New Pull Request you have probably\nrealized there are some things you did back in The Original Pull Request\nthat you could do better, and made some updates in the code in The New Pull\nRequest to reflect the wisdom you've gained along the way. That's great!\nCongratulations!
\nAfter your team has approved The New Pull Request, you can merge it into\nmaster and both delete the branch locally and on the remote. Then you should\nswitch back to the master branch. You can pull from master, and see your code\nthat solved The Sub-Goal reflected here.
\nThis is the hard part. Now you have to switch back to the branch for The\nOriginal Pull Request. Now you have to update this branch from master. It's\ngoing to be hard because now you've probably made different changes in The New\nPull Request than in The Original Pull Request so there will likely be\nconflicts.
\nThis is not a tutorial on how to solve merge conflicts. Use google to figure\nthat out
\nI can't understate: do this part really well. If you don't, then the history\nin the original branch will be even more incomprehensible, and you won't be able\nto tell if you lost any of your original work. Please, please, please do this\nwell.
\nP.S. Like I said before, don't be a martyr. Use tools like GitHub Desktop and\nPyCharm to help you merge. I heard that the git CLI was allegedly created by\nLinus Torvalds to slow other developers down.
\nWhy are we going through all of this pain, rather than just pushing your team to\nlet you merge The Original Pull Request? The reason you have to do this is\nbecause now all of the changes that addressed The Sub-Goal are part of\nmaster, and are no longer part of the diff of The Original Pull Request.
\nNow you're one step closer to your team being able to understand, review, and\neventually merge The Original Pull Request.
\nThis is the frustrating part. After you've gone through all of that work to\nsplit a tiny part of The Original Pull Request into a smaller, independent\npull request, you're not done. You will probably have to repeat steps 4-8 a few\ntimes. You'll be tempted to throw away the branch for The Original Pull\nRequest and maybe start over.
\nDon't do that.
\nIf you do, the same disorganization that lead to the mess of The Original Pull\nRequest might just slip back into whatever you do next. Even worse, nobody\nelse will be able to follow what you've done until now.
\nSo relax. This is going to take a few days. You're going to have to wait in\nbetween several iterations for feedback. That's good. You need feedback. I need\nfeedback. We all need to practice getting it and giving it. Embrace the\nopportunity to have your team help you improve your code, gain wisdom, and make\nyour contributions sustainable.
\nEventually after several iterations of 4-9, you will have excised all of the\ncode that was important for The Original Pull Request, but not directly\naccomplishing The Original Goal. As you removed independent parts, new parts\nbecame independent themselves. Eventually, The Original Pull Request will\nindeed match up exactly to The Original Goal, then you will be able to come\nback to it for review and merging.
\nI understand this is a frustrating process. The purpose of these steps were to\nhelp you think through a large piece of work you've done. You should be proud\nthat you've solved a complex problem with many intricate parts. It was a lot of\nextra work to break it into many pull requests, and it might have taken more of\nyour time the first time working through this process, but in the future, this\nmight help you to start with small tasks rather than addressing The Original\nGoal all at once. GitHub, for example, has an issue tracker that is very\nhelpful for this. I imagine that each issue should correspond to a Sub-Goal,\nand that each should have exactly one PR that addresses it. The Original\nGoal also deserves its own issue that points to all of the issues for its\nsub-goals. Eventually you will address this with a beautiful PR as well. Happy\ncoding!
\n*If you're thinking, why don't I use cherry picking? If you know what cherry\npicking is in the context of git (and also how to use it) then you probably\nwon't have the issue that prompted this blog post. But also, you should go\noutside and pick some apples instead. Thanksgiving is never more than a few\nhundred days away. It pays to be ready.
\nIt might be illustrative to see where an example of where this was done in\npractice, so I'll share some work I did with a text mining tool from Harvard\nMedical School, Gilda. It's a simple yet\npowerful system for grounding of named entities based on dictionary lookup.\nUnfortunately, it didn't include some dictionaries I wanted, and it didn't have\na UI to go with its web API.
\nSo I set out on figuring out how it generated dictionaries, where it stored\nthem, and how it loaded them to make the web app. I ended up making several\nmodifications to accomplish this goal, but it was a huge PR. I've definitely\nannoyed the author, @bgyori, with PRs that are too\nbig before, which he was ultimately not able to understand or merge.
\nKeep in mind, in your team, your teammates might be obligated to help you\nbecause you're working towards a common goal, getting paid, etc. When you're in\nthe open source world, nobody really owes you anything, so you it's in your best\ninterest to make things as easy as possible on the package's maintainer(s).
\nSo I made a few different pull requests that were all totally independent:
\nMaybe you're seeing a theme here. I was improving lots of different bits of\nGilda so I could reuse the package in new code later. The next incremental\nincrease was:
\nAnd finally with these in place, I realized that adding a web interface was\nparallel to my original goal, but not the core. What was really important was\nthat throughout all of the Gilda functionality, I could load my own synonym list\n(which I'd generate using the HPO, EFO, and DOID). I was able to address the UI\nwith:
\nAt the time of writing, we're still working through this PR. But all of it is\nleading up to the point where I can load my own files into this web interface.\nIt will seem so obvious to Ben when I send this PR next (but after giving him\nsome space\u2026 I did just bombard him with 5 PRs in a few days) what I am trying\nto accomplish and why.
\nWant to see what happens when you try and do all of this in one PR? You will\ncorrectly guess that the PR is a total mess, impossible to understand, and\nriddled with questions that are really too big to answer when your head is\nalready so far in the sand. Behold, in all its infamy, my failed PR from last\nsummer (#4). At this point, you\ncan't even see what a mess it was from the linked web page but if you go back\nthrough the version history before I broke it into 5 smaller PRs (using the\nworkflow described above) it was a monolith.
","doi":"https://doi.org/10.59350/3cn95-h9y94","guid":"https://cthoyt.com/2020/03/20/how-to-fix-your-monolithic-pull-request","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1584662400,"rid":"w8wqq-y9x71","summary":"We've all been there. You started a new branch from master. You had a very specific goal in mind, The Original Goal. You made a pull request (PR) to go with it, too, The Original Pull Request. But then, you had an idea! And also, someone on your team asked you to solve another problem!","tags":["Code With Me"],"title":"How to Fix Your Monolithic Pull Request","updated_at":1781539911,"url":"https://cthoyt.com/2020/03/20/how-to-fix-your-monolithic-pull-request.html","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"A few months ago, the question was posed on science Twitter: \"How many people\nhave published on ChemRxiv?\"
\n\nmakes me wonder about the stats at @ChemRxiv https://t.co/Ml5X8F4ckJ
\u2014 Egon Willigh\u24d0gen (@egonwillighagen) January 20, 2020
It was a good day for me, which meant I was in the mood to take up the first\nchallenged posed on Twitter. I found that Fran\u00e7ois-Xavier Coudert\n(@fxcoudert) has previously written a\npython client\nfor ChemRxiv. I made a pair of pull requests\n(fxcoudert/tools#9 and\nfxcoudert/tools#10) to fix some\nbugs and make it importable from other python modules.
\nUnlike BioRxiv, the pre-print server for biology, ChemRxiv is implemented with\nFigShare. It turns out that all FigShare \"institutions\"\nlike ChemRxiv are actually accessible through the main\nFigShare API. I think this is pretty cool, and\nmade sure that the ChemRxiv client that I had updated was actually able to be\nrun for any institution. Fun fact: the institution code for ChemRxiv is 259.
I got to work writing my\nown repository to wrap the\nclient, take care of downloading all of the bibliographic information available,\nand generating some pretty pictures. I originally ran the scripts and generated\npictures on January 20th, 2020 (the day Egon posed the question). Since the\npandemic has got the whole science community introspecting, I came back to this\ntoday and thought it might be worth writing up as a blog post.
\nWithout further ado, here are the most recent charts I've generated to answer\nthree main questions. I've linked the images in such a way that the charts will\nbe automatically updated with my GitHub repository. This also implicitly means\nthat there's a history of each image, but because two of them are plotting time\ncourse information, the history is already conveyed within the chart.
\nHow many papers were submitted each month to ChemRxiv? Keep in mind that the\ncurrent month is likely not complete.
\n![\"Articles](\"https://raw.githubusercontent.com/cthoyt/chemrxiv-summarize/master/figshare/chemrxiv/articles_per_month.png\"/)
This only counts using the ORCID iDs of the first authors; it's pretty\ninconsistent what other identifying information is included in the metadata for\neach article.
\n![\"Unique](\"https://raw.githubusercontent.com/cthoyt/chemrxiv-summarize/master/figshare/chemrxiv/unique_authors_per_month.png\"/)
How many authors submitted more than once per month? This chart shows spikes in\nAugust, which I will guess is when most people are submitting before their\nsummer breaks :)
\n![\"Percent](\"https://raw.githubusercontent.com/cthoyt/chemrxiv-summarize/master/figshare/chemrxiv/percent_duplicate_authors_per_month.png\"/)
![\"First](\"https://raw.githubusercontent.com/cthoyt/chemrxiv-summarize/master/figshare/chemrxiv/first_time_first_authors_per_month.png\"/)
How many first authors have historically contributed to ChemRxiv at each month?\nWe can take the first date of authorship for each author then count at each\nmonth how many unique first time authors there are. Then, we can use a\ncumulative sum to show how many authors have contributed to ChemRxiv at any\npoint in time.
\n![\"Historical](\"https://raw.githubusercontent.com/cthoyt/chemrxiv-summarize/master/figshare/chemrxiv/historical_authorship.png\"/)
If we aggregate the data, we can ask how many authors have submitted lots of\narticles:
\n![\"Author](\"https://raw.githubusercontent.com/cthoyt/chemrxiv-summarize/master/figshare/chemrxiv/author_prolificness.png\"/)
The following chart shows the popularity of different licenses over time. The\nCC BY-NC-ND 4.0 license is\na resounding victor. You can learn about Creative Commons (CC) licenses\nhere.
\n![\"Historical](\"https://raw.githubusercontent.com/cthoyt/chemrxiv-summarize/master/figshare/chemrxiv/historical_licenses.png\"/)
If you're interested to regenerate these charts yourself, you're welcome to do\nso with the following code:
\ngit clone https://github.com/cthoyt/chemrxiv-summarize\ncd chemrxiv-summarize\npython 01_download.py\npython 02_process.py\npython 03_visualize.py\nDownloading takes a bit of time (about 40 minutes) but there's a tqdm bar to\nkeep you entertained in the mean time. Normally I package all of my code, but\nthe one off scripts here didn't seem to warrant it.
As a final note, I'd like to shout out to Marshall Brennan\n(@Organometallica) for being an excellent\nspokesperson and public face of ChemRxiv. Also, throughout this process I\nrealized he also was a chemistry major in his bachelor's at Northeastern\nUniversity like me. Go huskies!
\nMay 2020 Update: Fran\u00e7ois-Xavier Coudert created the\nChemRxiv-Dashboard, which makes some\nsimilar summaries to this. Check it out!
\n\nI made a dashboard for @ChemRxiv, fed by the @figshare
\u2014 FX Coudert (@fxcoudert) May 19, 2020
metadata API.https://t.co/rKyAOGkrVO pic.twitter.com/fLfjEabraz
November 2020 Update: I added a license chart and made some changes to enable\nthis repo to be much more easily used for other FigShare institutions. If you've\nfound this post from @figshare's\ntweet and want help\nmaking these charts for your FigShare institution, please feel free to @ me on\nTwitter or send me an email.
","doi":"https://doi.org/10.59350/n7rjr-90f02","guid":"https://cthoyt.com/2020/04/15/summarizing-chemrxiv","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1586908800,"rid":"3ky5b-cne42","summary":"A few months ago, the question was posed on science Twitter: \"How many people have published on ChemRxiv?\"","tags":["Bibliometrics"],"title":"Summarizing ChemRxiv","updated_at":1781539909,"url":"https://cthoyt.com/2020/04/15/summarizing-chemrxiv.html","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"We have a big problem in the bioinformatics community with namespaces,\nidentifiers, and names. And nobody's posed the question better than\nRihanna herself.
\nDuring my Ph.D. at Fraunhofer, one of the old text miners reminisced to me about\nthe late 90's and early naughties when they had to curate their own dictionaries\nof synonyms for entities. I was lucky enough to have joined the bioinformatics\ncommunity after excellent nomenclature resources like\nChEBI and the HGNC\nwere established and accepted by the community as gospel.
\nI consider these sources excellent because it's quite easy to get a list of the\nidentifiers and corresponding names that they maintain (TSV, etc.). There are\nother nomenclatures, like the\nExPASy Enzyme Classes, that\nare stored as text files in non-standard formats.
\nThe Open Biomedical Ontology (OBO) format and\nOBO Foundry were first published in\n2007 as a solution for standardizing\na growing set of biomedical ontologies that few shared semantics. Many ontology\nmaintainers adopted their format, or at least used the OWL to OBO converter\ntools to include their ontologies in a reusable format. However, there remain\nsome notable holdouts like the\nCell Line Ontology that have not begun to\ndistribute their content as OBO.
\nIn parallel, the Ontology Lookup Service (OLS) was\npublished as one of many front-ends for exploring this growing list of\nresources. In comparison, it may have been one of the first tools to provide a\nnice user experience that included a search engine (powered by\nsolr, because they're living in the Java world).
\nBoth are lacking - there does not exist a solid OBO ecosystem (though Martin\nLarralde's pronto may well soon change\nthat) and even worse, the content in OBO loosely follows the standard, at best.\nOn the other hand, the OLS has both an over-engineered interface that isn't\nquite user friendly. For example, if you want to look up programmed cell death\n(GO:0012501), you have to know the internal OLS key for the namespace and the\nPURL for the identifier, which is not so obvious. Then you can finally hit the\nAPI.
\nAnd still, both of them lack some of my favorite, and arguably most important\nnamespaces, like HGNC, RGD, MGI, UniProt, Entrez Gene, and PubChem. As an aside,\ndealing with PubChem is for people operating on a whole different level, so I'm\nnot blaming anyone for dropping the ball on that one. Later, I will confess to\ndoing the same.
\nEven worse, the OBO Foundry and OLS can't even agree on what to call some\nnamespaces. A great example is the NCBI taxonomy database. On the NCBI site,\nthey say that the namespace is called NCBI and compact uniform identifiers\n(CURIEs) should look like NCBI:txid175694, OBO Foundry says the namespace is\nNCBITaxon (one of the few notable mixed-case namespace names) and CURIEs\nshould look like NCBITaxon:175694.
Identifiers.org came along to solve some of these ambiguities with a curated\ndatabase, but it's missing lots of the things in OBO Foundry and OLS, and it\neven disagrees on others. They call the NCBI taxonomy namespace taxonomy and\nsay that identifiers should look like taxonomy:175694. Exhausting!
![\"Registry](\"https://cthoyt.com/img/registry_comparison.svg\"/)
One more issue is the GOGO problem. Many OBO ontologies use local identifiers\nthat also include the prefix because a given ontology might contain terms\nimported from other ones. However, this means that ontologies that originated\nfrom the OBO world have redundant identifiers, like from GO (e.g.,\nGO:GO:0012501). I know what you're wondering: is Dr. Claw in charge? Maybe.
\nThe reason I went down this rabbit hole is because I want to support people to\ndo better curation. This means I want them to use identifiers instead of ever\nchanging names. For example, it turns out the half life of an HGNC gene symbol\nis very short -\nthousands of them change every year.\nHowever, if I want people to use identifiers instead of names in their\ndatabases, their papers, and other writing, there need to be really good tools\nfor looking up the names that go with each identifier and the cross-references\n(equivalences) to other databases that are talking about the same thing.
\nSo I built PyOBO. It includes tools for\nreading the OBO Foundry and getting all of the OBO resources that are available\n(as well as many manual fixes for incorrect metadata), it uses Daniel\nHimmelstein's Obonet for parsing and\nstoring pre-parsed files for fast loading, and it applies a swath of rule-based\nnormalization that I've\nmanually curated\nby personally reading all of the OBO files, their identifiers, their\ncross-references, relationships, properties, and everything else. When it comes\nto data, there really is no way around getting your hands dirty.
\nI also went ahead and\nwrote parsers and converters\nfor lots of other databases like Entrez, ComplexPortal, InterPro, and others so\nthey could play nice with the rest of the ecosystem. Of course, this is an\nongoing process. There are always more databases to include, and when it comes\nto super-sized ones like PubChem, the paradigms I used might not hold up anymore\n(though I did write parser/converter for it and you're welcome to use it).
\nAfter this long journey of a blog post, I think we're ready to address Rihanna's\nperrenial question: what's my name? Until now, there really didn't exist a\nservice that let you look up the name for an entity by its CURIE. The link I\ngave for the OLS is the closest I have found, and that just doesn't cut it.
\nAfter all of this coding, I wrote a script (just run obo ooh-na-na) that takes\nall of the available sources, normalizes their namespaces, normalizes their\nidentifiers, and dumps them as a big 'ol TSV file. 3 columns - namespace,\nidentifier, and name. No nonsense. Probably legal! Get it at\n![\"DOI\"](\"https://zenodo.org/badge/DOI/10.5281/zenodo.3756206.svg\"/){kind=icon}
.\nI'll make updates periodically as I add more sources, such as if/when I feel\ncomfortable with including the PubChem dump - the\nCID-Title.gz\nfile is about 1.3 gigabytes, which means this will significantly increase the\nsize, but not so much that it's unreasonable.
I can imagine that most people probably won't want to download this file, or\nload it in memory (un-gzipped) every time they want to use it. I wrote a simple\nweb service that wraps this dataset\nincluded in PyOBO.\nIt should be as easy as running with the shell with\npython -m pyobo.apps.resolver then running the following python code:
import requests\n\n# This is an exact match\nsuccessful_request = requests.get('http://localhost:5000/resolve/DOID:14330').json()\n# {\"identifier\": \"14330\", \"name\": \"Parkinson's disease\", \"prefix\": \"doid\", \"query\": \"DOID:14330\", \"success\": True}\n\n# This one remaps the prefix if you get it slightly wrong\nsuccessful_remapped_request = requests.get('http://localhost:5000/resolve/DO:14330').json()\n# {\"identifier\": \"14330\", \"name\": \"Parkinson's disease\", \"prefix\": \"doid\", \"query\": \"DO:14330\", \"success\": True}\n\n# This one can't find the identifier.\nunsuccessful_request = requests.get('http://localhost:5000/resolve/DO:00000').json()\n# {\"identifier\": \"00000\", \"message\": \"Could not look up identifier\", \"prefix\": \"doid\", \"query\": \"DO:00000\", \"success\": False}\n\n# Keep in mind, the point of this service isn't to validate identifiers.\nunsuccessful_crazy_request = requests.get('http://localhost:5000/resolve/DO:thisIsNotRightAtAll').json()\n# {\"identifier\": \"thisIsNotRightAtAll\", \"message\": \"Could not look up identifier\", \"prefix\": \"doid\", \"query\": \"DO:thisIsNotRightAtAll\", \"success\": False}\n\n# No mercy for bad prefixes\nunsuccessful_prefix_lookup = requests.get('http://localhost:5000/resolve/notanamespace:0000').json()\n# {\"message\": \"Could not identify prefix\", \"query\": \"notanamespace:0000\", \"success\": False}\nIt's especially important that the service normalizes curies first, so both\nDOID:14330, doid:14330, and DO:14330 can all be resolved to their name,\nParkinson's disease. Because I did extensive manual curation of namespaces and\ntheir synonyms, NCBITaxon and taxonomy are both acceptable as well. However,\nthis service doesn't load from the aforementioned TSV, but rather takes\nadvantage of PyOBO's internal code for looking up mappings. I can imagine lots\nof ways I might re-write this service to directly take advantage of this dump (I\nalso invite you to do the same, however best suits you) such as loading it into\nEdgeDB and auto-generating a GraphQL endpoint.
The last thing that I'm looking into getting this service hosted so everyone can\nbenefit from it without doing dev-ops in their own organizations. Then I will\ncontinue to obfuscate all usage and documentation with references to pop\nculture. Enjoy!
","doi":"https://doi.org/10.59350/wmj3y-04914","guid":"https://cthoyt.com/2020/04/18/ooh-na-na","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1587168000,"rid":"81da2-mtb29","summary":"We have a big problem in the bioinformatics community with namespaces, identifiers, and names. And nobody's posed the question better than Rihanna herself.","tags":["OBO","Lexica"],"title":"Ooh Na Na, What's My Name?","updated_at":1781539907,"url":"https://cthoyt.com/2020/04/18/ooh-na-na.html","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"On top the issue of resolving identifiers to their\nnames, the bioinformatics community has a\nhard time figuring out when two identifiers from different databases are\nequivalent. You know who else has the same problem? Inspector Javert. Get ready\nfor a Les Miserables-themed post on how to address this long-standing problem.
\nI have to start my tale of woes by disclosing my source material. I loved both\nthe 1985 and 1987 recordings from the respective original London and Broadway\ncasts. But, for the purposes of this post, I will assume that you've seen the\nexcellent 2012 film adaptation of Alain Boublil, Jean-Marc Natel, and Herbert\nKretzmer's musical adaptation of Victor Hugo's novel Les Miserables and tell\nthe story through that perspective. I also want to you to know that I enjoyed\nRussell Crowe's Inspector Javert very much.
\nLes Miserables begins with the Work Song, in which the protagonist,\nprisoner:24601 is confronted by Inspector Javert while doing some\u2026 work. He\ninsists he has a name, Jean Valjean and his identifier in his\nhome village's fictional\ndatabase (that I just retconned) was faverolles:2468. Javert isn't interested\nin his name. It's enough that he has a cross-reference between faverolles:2468\nis equivalent to prisoner:24601. He was only there to inform Jean Valjean that\nhis parole has begun and issues him a passeport jaune (yellow ticket) for the\ncommune of Pontarlier.
I'm sure this passport also had an identifier on it. I'm going to take a bit of\ncreative freedom and say it was pontarlier:25791. It probably also had Jean\nValjean's prisoner number on it so everybody knew he was in the 1800's fictional\nFrench convict database. The fictional 1800's French took maintaining\ncross-references very seriously.
Jean Valjean never made it to Pontarlier. Instead, he broke his parole, forged\nsome new documents, and went to Montreuil-sur-Mer under the new name of Monsieur\nMadeleine. It's probably the case that his identifier for the Montreuil-sur-Mer\ncity database was montreuil-sur-mer:1357, or something like this (more\nretcons!). It must have been a good fake, because even the king of France\nrecognized him (note: this plot point did not appear in the film).
Javert figured out Jean Valjean broke his parole basically immediately and set\nout on his quest to find and capture prisoner:24601 once again. Until this\npoint, Javert has access to the prisoner registry and yellow tickets. He knows\nprisoner:24601 is the same as both faverolles:2468 and pontarlier:25791.
The part that will hit close to home for many bioinformaticians is that when\nJavert goes to Montreuil-sur-Mer, he meets Monsieur Madeleine. He is unaware\nthat it is Jean Valjean. There is no cross-reference between faverolles:2468\nand pontarlier:25791 or montreuil-sur-mer:1357. If there were a\ncross-reference in the fictional French 1800's inspector database, Javert could\nhave arrested Jean Valjean on sight. Instead, Javert had to the hard work of\ncurating cross-references himself and finding out who was the same in the\nmontreuil-sur-mer database as prisoner:24601. Admittedly, he probably would\nhave called this inspecting.
The next part that will hit even closer to home for many bioinformaticians is\nthat after his inspecting, Javert actually identified the wrong guy! This lead\nto one of the my favorite songs in musical theater ever\n(Who Am I?), where Monsieur\nMadeleine (montreuil-sur-mer:1357, also actually Jean Valjean, but Javert\ndidn't yet realize this) admits that he is actually prisoner:24601. In this\nextended metaphor of a blog post, Jean Valjean's confession in \"Who Am I\" is\neffectively the same as a database providing its own cross-references to other\ndatabase. Would be nice if everyone did this, and did it well, huh?
You should know that Javert is a powerful cross-reference reasoning machine. He\nalready knew faverolles:2468 was the same as prisoner:24601. Now he knew\nthat montreuil-sur-mer:1357 was the same as prisoner:24601. This way, he\ncould infer that montreuil-sur-mer:1357 (Monsieur Madeleine) is actually\nfaverolles:2468 (Jean Valjean). One of the nice properties of cross-references\nis that they're transitive through any number of connections. We'll take\nadvantage of this fact later. You'll also have to excuse the fact that\nthroughout this post, I'm operating under the assumption that \"cross-references\"\nand \"equivalences\" are the same thing. That's not always true, and sometimes it\ncan even get you in trouble. For example, provenance can be a cross-reference,\ndisease-gene associations are considererd as cross-references in MONDO (I\nthink), and OBO even gives specific semantics for when you should consider this\nassumption valid. We'll just have to live with it for now.
Javert might have got lucky that Jean Valjean revealed himself once, but the\nshow must go on! Jean Valjean had many more songs to sing and thus had to escape\nfrom Montreuil-sur-Mer to Paris. This meant that Javert has to find another\nmapping to Jean Valjean's new paris identifier. And we already know that the\nFrench 1800's inspector database of cross-references was not being maintained.\nExhausting!
In the bioinformatics community, we have a very similar problem to Inspector\nJavert. There are lots of databases that are talking about the same things, but\nonly a few of them provide mappings between each other. This means that we\neither have to curate our own cross-references, do our best to infer new\ncross-references based on ones we already have, or throw our hands in the air.
\nLuckily, we have a few standardized resources to fall back on. In addition to\nstandardizing the storage of identifier/name pairs, the OBO format standardizes\nthe way cross-references are stored and the OBO Foundry already contains quite a\nfew cross-references imported from the ontologies that it covers.
\nOne of the most difficult entity types to map from database to database are\nphenotypes because of the variety of language used to describe each, the\ndifferences in semantics of how each is defined, and the sheer number of\ndatabases. Unfortunately, some of the most popular like MeSH and to an extent,\nUMLS, NCIT, SNOMED-CT, and ICD (seemingly the culprits are mostly American!?)\nprovide very little accessible information. Some are even paid, so the ony\ncross-references that exist are externally curated ones from other laudable\nsources like HP, DOID, and EFO. In fact, dealing with phenotypes is such a pain,\nthat there is a project called the\nMonarch Initiative that has a huge staff\ntrying to solve exactly this problem and publish the results through the\nMonarch Disease Ontology (MONDO).\nNormally, I would reference\nthis XKCD comic about making new standards when hearing\nabout something like this. But these are dire times, and one of my opinions is\nthat you should always trust curators who love what they do.
\nThere are also lots of cross-references available from databases that don't\nmaintain their nomenclature as an ontology. One example is\nBioGRID,\nwhich assigns proteins internal accession numbers, but almost all of them\ncross-reference out to Entrez Gene (I counted less than 15 that didn't, and 3 of\nthem are COVID-related, so cut them some slack). As an aside, I don't really\nunderstand why BioGRID would go through the effort of maintaining their own\naccession numbers. In the literal handful of cases where they can't reference\nEntrez Gene, I think it would be better to email the maintainers and work with\nthem to make improvements.
\nIt's also worth noting that excellent resources like HGNC, MGI, RGD, SGD,\nEnsembl, UniProt, and others in the genome (and gene product) nomenclature do a\nstellar job at maintaining cross-references. So to all of the curators and\nmaintainers who work there, I would like to sincerely thank you.
\nThere are also community-curated cross-references sources. One of the notable\nones is from Harvard Medical School, that's mapping MeSH identifiers to gene\nidentifiers in the\nGilda GitHub repository.\nI think this is really a good time to point out that MeSH contains a bit of\neverything, is ubiquitous throughout the bioinformatics community, and in my\nopnion is is doing a huge disservice by not providing these kinds of mappings\nitself. Or, alternatively, it is, and both the Harvard guys and I have never\nfound it. It's not impossible, but we're all very motivated, so I think we would\nhave found if it did. If any MeSH maintainers are reading this and want help\nmaking this happen, I would be elated to donate my time to you to help solve\nthis problem.
\nWith all these data source in mind, I built an extensible pipeline in\nPyOBO\nfor extracting cross-references from entries in OBO Foundry and other\ncross-reference sources. Throughout the process, I realized that these sources\nhave an incredible variety in how they name prefixes and how the OBO format\nitself has been (ab)used. I made lots of improvements, wrote extensible code\nthat allowed the specification of new rules through external files (and thus\nless code writing in the future), and did lots more curation. I won't get into\nthe technical part of that here, since you can read the code (if you dare).
\nAfter all that this coding, I wrote a script (just run obo javerts-xrefs) that\ntakes all available cross-references, normalizes their namespaces, normalizes\ntheir identifiers, and dumps them in a big 'ol TSV file. 5 columns - source\nnamespace, source identifier, target namespace, target identifier, and\nprovenance (ontology name or URL). No nonsense. Get it at\n![\"DOI\"](\"https://zenodo.org/badge/DOI/10.5281/zenodo.3757266.svg\"/){kind=icon}
.\nI'll make updates periodically as I add new sources.
Once you have a database of cross-references, you have actually built an\nundirected graph. Equivalences go both ways, and they are transitive. This means\nthat every connected component in an equivalence graph represents a set of\nentities that are mutually equivalent. In other words, if a path exists between\ntwo nodes in an equivalent graph, then they are equivalent.
\nEven better, you don't have to materialize all of the possible inferred\nequivalences when you have an equivalence graph because identifying all of the\nnodes in a connected component can be done in linear time with respect to the\nsize of the connected component, which is usually pretty small, by using a\nbreadth-first or depth-first search.
\nBased off of that, one application of an equivalence graph is to identify all of\nthe nodes that are equivalent to a given node. You can also get a little tricker\nand identify the paths through which the traversal must go if you want to\nestablish an equivalency. You could even go further and weight edges based on\nhow much you trust the source from which they came to identify how much you\nshould believe in a mapping. For example, if you have a percent confidence in\neach mapping being right, then the confidence in the whole pathway would be the\nproduct of the confidences.
\nThe actual problem I set out to solve was given a set of entities, remap all of\nthem based on a prioritized list. For example, I might have a set of entities\nthat contains HGNC genes, Entrez Genes, and OMIM genes. If my favorite\nnomenclature consortium is Entrez, my second favorite is HGNC, and my third\nfavorite is OMIM and I have an equivalence database, I might want to remap all\nof my identifiers. This is very important during the curation of mechanistic\nbiology (such as with BEL), since curators will likely use all sorts of\nidentifiers with no clear guidelines or rules. This means that the same entity\nmight appear twice with different identifiers in the same curated data!
\nGiven a priority list, you can even transform an equivalence graph into a\ndirected graph where each identifier has a single out edge pointing towards the\nidentifier that is the best mapping. Then, each connected component would become\na star graph. There's actually a better data structure for this, since each\nentity points to exactly one thing - a mapping. This is a more efficient data\nstructure for storage, and if your graph is implemented as an adjacency\ndictionary (becuase you're using networkx, right?), then you basically already\nhave this.
I've provided an implementation for all of these in PyOBO. They can be run as a\nweb API with python -m pyobo.apps.mapper. There's a keyword argument to allow\nyou to load the TSV from Inspector Javert's Xref Database directly, or if you're\nfeeling lucky, to regenerate it yourself. Below I will give a few examples of\nhow to use it. Later, I would also like to host this service for anyone to use.
pip install git+https://github.com/pyobo/pyobo.gitpython -m pyobo.apps.mapper -x inspector_javerts_xrefs.tsv.gzimport requests\n\n# Get all entities mapped to MAPT, including through chains of xrefs\nsuccessful_request = requests.get('http://localhost:5000/mappings/hgnc:6893').json()\n\"\"\"\n{\n \"orphanet:123144\": [\n {\n \"provenance\": \"hgnc\",\n \"source\": \"hgnc:6893\",\n \"target\": \"orphanet:123144\"\n }\n ],\n \"pr:P10636\": [\n {\n \"provenance\": \"hgnc\",\n \"source\": \"hgnc:6893\",\n \"target\": \"uniprot:P10636\"\n },\n {\n \"provenance\": \"pr\",\n \"source\": \"uniprot:P10636\",\n \"target\": \"pr:P10636\"\n }\n ],\n ...\n}\n\"\"\"\n\n# Keep in mind this isn't a validation service\nunsuccessful_request = requests.get('http://localhost:5000/mappings/hgnc:0000').json()\n# {\"message\": \"could not find curie\", \"query\": {\"curie\": \"hgnc:0000\"}, \"success\": False}\n\n# Get all paths mapping MAPT in HGNC to Ensembl. Returns a list of paths (which are lists of xrefs)\npath_request = requests.get('http://localhost:5000/mappings/hgnc:6893/ensembl:ENSG00000186868').json()\n\"\"\"\n[\n [\n {\n \"provenance\": \"hgnc\",\n \"source\": \"hgnc:6893\",\n \"target\": \"ensembl:ENSG00000186868\"\n }\n ]\n]\n\"\"\"\n\n# Get the priority identifier for MAPT identified by Ensembl\nprioritize_request = requests.get('http://localhost:5000/prioritize/cosmic:MAPT').json()\n# {\"found\": True, \"query\": \"cosmic:MAPT\", \"result\": \"hgnc:6893\"}\n\n# What happens when a CURIE can't be found for prioritization\nunsuccessful_prioritize_request = requests.get('http://localhost:5000/prioritize/cosmic:NOPE').json()\n# {\"found\": False, \"query\": \"cosmic:NOPE\"}\nI'd like to give a big thanks to my high school music teacher, Ken Tedeschi, for\nhelping me (and basically everyone else) fall in love with Les Mis in high\nschool. Writing about my work was so much more fun in extended metaphor. I would\nalso like to thank Hugh Jackman. You know, for being Hugh Jackman.
\nI have some random afterthoughts that I think might be worth including, that I'm\nadding after originally posting this.
\nYou might be wondering why I didn't get into a discussion about the\nOntology Mapping Service (OXO)\nfrom the EBI. It looks to me like this project has been abandoned. Even if not,\nit's API has most of the same issues that I described in a previous\npost.
\nI'm also aware of BridgeDB, from which I think I\nwill be able to take inspiration to include more xrefs later. However, I think\nthey're limited in scope, and PyOBO is more about standardizing data so nobody\nhas to figure out databases\u2026 again and again and again.
\nOne glaring omission from this work is WikiData mappings. I have a plan to\ninclude curated information in the PyOBO metaregistry that links databases to\ntheir WikiData properties. That will allow me to build an automated framework\nfor downloading these mappings, given the curation of the properties.
","doi":"https://doi.org/10.59350/r3qzt-z0d08","guid":"https://cthoyt.com/2020/04/19/inspector-javerts-xref-database","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1587254400,"rid":"vqbr3-bcb03","summary":"On top the issue of resolving identifiers to their names, the bioinformatics community has a hard time figuring out when two identifiers from different databases are equivalent. You know who else has the same problem? Inspector Javert. Get ready for a Les Miserables-themed post on how to address this long-standing problem.","tags":["Mappings"],"title":"Inspector Javert's Xref Database","updated_at":1781539905,"url":"https://cthoyt.com/2020/04/19/inspector-javerts-xref-database.html","version":"v1"}},{"document":{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"As scientists, we place huge importance on the communication of our results. We\nspend lots of time on editing, revising, and formatting so people can understand\nwhat we did. We also write a lot of code, so why aren't we investing the same\namount of love? Enter, flake8.
\nIt's incredibly important that we write following community standards so when\nother people read our work, they don't have to think about how it's organized.\nFor scientific prose, this usually means the IMRD\n(introduction-methods-results-discussion) format. In Python, my current favorite\nprogramming language for science, this means using a standardized number of\nspaces for indents (4), using triple-double quotes for docstrings in the\nbeginning of each module, class, and function, and lots more.
\nIt's pretty intimidating to figure out style. For english prose, Strunk and\nWhite wrote\nThe Elements of Style.\nFor Python, Guido van Rossum wrote\nPEP-8 and Raymond Hettinger\npresented Beyond PEP-8. Even with\nthese resources, it's still hard to learn which are rules and which\nare more like guidelines.
\nThis post is a short explanation of how I use flake8 to keep a consistent\nstyle in the code in my Python projects. There's a similar command line tool for\nfixing the style in R projects that's already built into most operating\nsystems - rm -rf *, but I won't get more into that here.
It's pretty easy to get up and running with flake8 - just run\npip install flake8 then use it from the shell on a python file like\nflake8 my_file.py or flake8 my_directory/. Then, it outputs a list of\nproblems that need to be fixed on a line-by-line basis in your code.
![\"Flake8](\"https://cthoyt.com/img/flake8_output.png\"/)
You can also install plugins with pip like that extend the kinds of things it\nchecks. A few that I install are:
id.flake8 output (explanation how to set up is below)print(). The literal only exception is when using print to get\ntext into a file with print(..., file=...)% or .format() formatting.\nException being for logging.In each of my repositories, I put all of the information on how to install\nflake8 and its plugins then run them in a tox configuration under the\n[testenv:flake8] header so they can easily reproducibly run with\ntox -e flake8. An example of part of one of my tox.ini files (which always\nlives in the root of the repository) is below:
[testenv:flake8]\nskip_install = true\ndeps =\n flake8\n flake8-bandit\n flake8-builtins\n flake8-bugbear\n flake8-colors\n flake8-commas\n flake8-comprehensions\n flake8-docstrings\n flake8-import-order\n flake8-print\n flake8-use-fstring\n pep8-naming\n pydocstyle\ncommands =\n flake8 src/pybel/ tests/ setup.py\ndescription = Run the flake8 tool with several plugins (bandit, docstrings, import order, pep8 naming).\nAnother configuration file you can set up in the root of the repository is\n.flake8. Unfortunately, the Python configuration file reader doesn't allow\nsome of the crazy characters that I want to use for the colors so this can't be\nincluded in setup.cfg or tox.ini like most of your other configuration.
[flake8]\nignore =\n # line break before binary operator\n W503\nexclude =\n .tox,\n .git,\n __pycache__,\n docs/source/conf.py,\n build,\n dist,\n tests/fixtures/*,\n *.pyc,\n *.egg-info,\n .cache,\n .eggs\nmax-line-length = 120\nimport-order-style = pycharm\napplication-import-names =\n pybel\n bel_resources\n tests\nformat = ${cyan}%(path)s${reset}:${yellow_bold}%(row)d${reset}:${green_bold}%(col)d${reset}: ${red_bold}%(code)s${reset} %(text)s\nFirst thing you'll notice is the ignore list. This isn't here to turn flake8\noff because you're feeling lazy. If somebody includes a change in this list in\ntheir PR, you have to explain to them that compliance is not optional, then help\nthem work through the problem that they obviously gave up on solving. It's\nactually there for you, as the project maintainer, to enumerate the flake8\nrules that you don't agree with. For example, I totally disagree with the W503\nline break before operator rule. I want to write long conditionals with and\nstatements on the first line, like this:
if (\n condition_1\n and condition_2\n and condition_3\n):\n print('all true')\nOne of the benefits of this style is you can add more lines with only single\nline diffs. The other is that the reader always sees the operation that goes\nwith each line. Same could be done with arithmatic that could incorporate not\nonly + but also -.
Next is the exclude block. Just copy/paste this each time, since it has lots\nof garbage you don't want flake8 to bother with. One of the checkers in\nflake8 is for function \"cyclomatic\" complexity. You can make the maximum\nnumber higher with max-complexity. Normally, you want this to be enforced, but\nsometimes there's no way around a complex function. For this, you can add a code\ncomment noqa followed by the error code like # noqa:W123. Again, adding tags\nto ignore bad style just to pass flake8 is against the point.
The max-line-length is a very contentious setting. I think 120 is fine. Some\npeople think 78, 79, or 80 is best because of the standard sizes of old computer\nscreens or punch cards\u2026 When I get older and I can't read my computer screen,\nI'll probably make the text bigger and change my mind about this. If you find\nyourself breaking up lines in a totally non-sensical, unstyled way, then you're\nconforming too tightly to the rules. Sorry about the mixed messages!
import-order-style = pycharm\napplication-import-names =\n pybel\n bel_resources\n tests\nI copied this again because this part is really important. You have to tell\nflake8 what rules you use for import order. I use the pycharm rules, which\ngroup python builtin packages, then 3rd party packages, then my packages. The\napplication-import-names is a place to list what are your packages.
Last is the format entry, which gives the nice colorful output. Copy paste\nthis! I borrowed mine from Scott Colby.
After all of that, I set up Travis CI to run tox every time code is pushed to\nthe repository. If you're working in a team, you probably do something like the\nfork/pull request or branch/pull request workflow on GitHub to support doing\ncode review before merging new code. The best part is that there's a big box on\neach pull request that checks if flake8 passed (among other tests), which\nmeans that there were no errors detected.
I encourage my teammates to make pull requests as soon as they start working on\ncode. GitHub even has a \"draft pull request\" mode now. However, before asking\nanyone to review your code, it has to pass flake8. And obviously, no code that\nisn't passing flake8 can be merged.
This is a very painful process to get people used to. I've done it with many\ngroups of people and always got pushback. However, everyone who has gone through\nthe process with me has come out the other side happy that they did it. It's\nimportant that when you start enforcing coding rules on other people that you\nare a resource for them - when somebody is frustrated by a flake8 error code\nthey have never seen, they will likely forget how to use Google. They will\nprobably ask you for help. You have to resist the urge to send\nlmgtfy links to them and be patient. Because eventually,\nthey will do it on their own, and spread the gospel of flake8.
While a good arsenal of flake8 plugins provides a solid foundation, it's not\nall that needs to be done to make your code readable and look good. Just like\nwith reading and speaking, the best way to develop a sense of style is by\nreading lots of code (with the caveat that reading poorly written code\nprobably won't teach you much). Within the rules imposed by flake8, there is\nlots of space for style. If you watch lectures from David Beazley, you'll notice\na very different style from Raymond Hettinger, and also from me.
Now that you've made it to the end of this short guide, I wish you the best of\nluck on developing your own style!
\nAre you working with people who are particularly unsusceptible to Travis CI\nemails or checking the big red box on pull requests? You could try getting them\nset up with pre-commit hooks, which run the style\nchecks locally any time they try and push (even if it's to a branch) and it will\ngive them the message in the console.
\nIs style not your thing at all / you're not ready to let go of your identity as\na Java/Perl developer? Maybe consider Black,\nwhich actually re-writes your code in a deterministic style. I don't live by it,\nbut it's a great tool to run on a code base that's never been loved before going\nback and stylizing it.
","doi":"https://doi.org/10.59350/cfxtw-0ma23","guid":"https://cthoyt.com/2020/04/25/how-to-code-with-me-flake8","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1587772800,"rid":"4y55t-c1k96","summary":"As scientists, we place huge importance on the communication of our results. We spend lots of time on editing, revising, and formatting so people can understand what we did. We also write a lot of code, so why aren't we investing the same amount of love? Enter, flake8.","tags":["Code With Me"],"title":"How to Code with Me - Flake8 Hell","updated_at":1781539903,"url":"https://cthoyt.com/2020/04/25/how-to-code-with-me-flake8.html","version":"v1"}}],"items":[{"authors":[{"affiliation":[{"id":"https://ror.org/050qmg959","name":"Singapore Management University"}],"contributor_roles":[],"family":"Tay","given":"Aaron","url":"https://orcid.org/0000-0003-0159-013X"}],"blog":{"authors":null,"community_id":"f34e2211-9904-4b58-97ab-0beeb79ef6f7","created":1697068800,"current_feed_url":null,"description":"Aaron Tay's thoughts about academic librarianship","favicon":"https://rogue-scholar.org/api/communities/f34e2211-9904-4b58-97ab-0beeb79ef6f7/logo","feed_format":"application/rss+xml","feed_url":"https://aarontay.substack.com/feed","filter":null,"generator":"Substack","home_page_url":"https://aarontay.substack.com","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"musings","status":"active","subfield":"3309","title":"Aaron Tay's Musings about Librarianship","updated":1781540136,"use_api":true},"blog_name":"Aaron Tay's Musings about Librarianship","blog_slug":"musings","content_html":"This post is part of a \"hot takes\" series in which I make sharper claims than I usually do. I do not intend to offend, and I am not trying to tar every librarian with the same brush \u2014 the patterns I describe and perceive may be a function of my own local context.
In my last hot takes post, I argued that while designing a search system to maximise learning gains may not always align with designing a search system that scores the best for relevancy, unplanned friction in learning aka poor relevancy is never a good idea.
In this post, I consider the idea of tools that are considered flawed because they can't match the performance of an older tool in some way1.
Imagine you are a librarian in 2004, when Google Scholar launches in beta.
You have read the studies. The coverage gaps are real. The metadata is wretched. Nobody, including Google, can tell you exactly what is indexed2.
You decide it is the dumbest thing on the planet and declare that it could never be useful to anyone, despite a vocal minority of users insisting otherwise.
Fast forward to 2024. Google Scholar has become the default academic search starting point for many researchers and is widely regarded as the most comprehensive free academic search engine3. Your first assumption might be that Google fixed all the problems.
But that assumption would be mostly wrong.
Sure, Google Scholar improved, especially in coverage. Its metadata may have arguably also have improved. But you still cannot consult a stable, auditable list of indexed sources or records. Even Google's own guidance effectively tells users to test coverage by sampling titles rather than by checking a definitive inventory. One of the fundamental weaknesses librarians diagnosed in 2004 is still there.
And yet the librarians and researchers of 2024 are not idiots.
What happened is simple. Users learnt to compensate. They used Scholar for what it was good at and routed around its flaws or used another tool for other use cases (e.g. Evidence synthesis). They learnt when its metadata could not be trusted, when its coverage was opaque, and when to complement with another database was needed. The tool did not become perfect. It became useful enough, along side other tools.
That trajectory is suprisingly common with new technology and one I think about when I read arguments thay say AI or any technology tools can never be useful because of some \"fundamental flaw\".
Some of the most prominent sceptics have already said as much.
Gary Marcus, cognitive scientist, author of Rebooting AI, and one of the most consistent public critics of AI hype, has repeatedly acknowledged that LLMs can be useful, especially for coding, brainstorming and writing, while arguing that they are unreliable and not a route to AGI alone.
Margaret Mitchell, a co-author of the influential \"Stochastic Parrots\" paper, has been even more explicit: LLMs can be \"extremely useful\".
Emily M. Bender has likewise clarified that \"stochastic parrot\" is a description or metaphor, not an argument that these systems have no possible utility4.
Mike Caulfield, of SIFT fame, is actively studying and using these tools. These are not AI boosters. If even they concede that the tools can be useful, the basic question has already been settled.
You can oppose LLM use on environmental grounds, labour grounds, epistemic grounds, or any number of other defensible grounds.
But the claim that these tools can never be useful to anyone has moved past argument into something closer to an unfalsifiable position. No demonstration of utility, no improvement in the tools, and no evidence of successful use by researchers seems able to update it.
For the rest of this post, when I say \"AI tools\", I mean AI-powered academic search tools. I work in this space, so I will stay in my lane. The argument may extend elsewhere, but I am not pretending to make that case here.
It is also worth being precise about what \"AI search\" means. The term covers several different things: changing what gets retrieved, reranking results, summarising content, and generating direct answers to questions. These are not the same capability and do not carry the same risks.
A librarian who objects to generative answer synthesis is making a different argument from one who objects to AI-assisted reranking. Conflating the two muddles the debate. Before objecting to \"AI search\", it is worth saying which part concerns you, and why.
The Google Scholar analogy maps most cleanly onto AI-assisted retrieval and reranking: helping surface relevant results that users might otherwise miss. It also maps reasonably well onto \"tip-of-the-tongue\" search, one of the limited uses Bender has acknowledged as potentially useful.
It maps less directly onto generative answer synthesis, where hallucination risks are sharper. I am not arguing that all uses of AI in search carry equal risk. I am arguing that even the riskiest versions clear the \"never useful\" bar. The appropriate response to different risk profiles is differentiated teaching, not blanket rejection.
The people who insisted in 2004 that Google Scholar could never be useful until Google published full holdings lists were sure they were right. But they were eventually proven wrong to conclude that a tool could never be useful without that.
A tool need not be perfect to be useful. This sounds obvious, but it is the point that keeps getting lost.
One objection is that the Scholar analogy fails because LLM errors are different. Google Scholar had messy metadata and opaque coverage. LLMs produce overconfident hallucinations.
That objection has force, but notice what it actually supports. It supports teaching verification skills. It supports scaffolding. It supports appropriate scepticism. It does not support the early conclusion that the tool is useless.
A tool that requires careful handling is not the same as a tool that cannot be useful. The verification argument also cuts both ways. Uncritical acceptance of search results, including Google Scholar results, has always been the failure mode librarians teach against.
Perhaps the scaffolding still will not be enough. But I am sure many librarians who rejected Google Scholar in 2004 were pretty sure too.
Nor does it help that some of the most vocal sceptics seem not to have engaged seriously with these tools since 2023. They appear to underestimate how much the systems, and the harnesses around them, have changed even just three years. But even that point is secondary. The Scholar parallel does not depend on the exact pace of improvement5.
I currently believe from some experience setting up and testing agents, that between the use of code to constrain the model and aggressive multi-validation checks, you can reduce the probability of error/hallucatins down to very low levels comparable to the human.
The lesson is not that tools improve. The lesson is that what counts as good enough is not always obvious at first.
Some librarians argue that the Google Scholar comparison to AI breaks down on three grounds: librarians do not actively promote Scholar (but librarians actively promote AI); users genuinely want Scholar rather than having it pushed on them; and Scholar is free, whereas many AI tools are commercial products.
None of these objections does the work required.
On promotion, many librarians do promote Google Scholar. It appears in LibGuides, instruction sessions and one-on-one consultations. The claim that \"we do not promote it\" is a polite fiction. Beyond what is said publicly, plenty of academic librarians reach for Google, Scholar or Wikipedia first in their own work when the situation calls for it.
On user demand, users clearly want AI tools. Whatever one thinks of that demand, pretending it does not exist is not a serious position.
On cost, being free or paid is a separate question from whether a tool can be useful. There are genuine concerns about commercial AI: vendor lock-in, inequity of access, environmental cost, labour implications, surveillance, and the commercial capture of scholarly infrastructure. Those concerns deserve serious engagement.
But they are arguments about adoption, governance and institutional support. They are not arguments that the tools can never be useful.
And let us be honest about the comparison. Library databases have plenty of flaws: idiosyncratic interfaces, uneven indexing, opaque relevance ranking, and sometimes weak metadata. We still pay substantial sums for them and promote them as a matter of course. The objection to AI tools cannot simply be that they are commercial and imperfect, because by that standard half the collection budget becomes difficult to defend.
There is a related claim that deserves a direct response: librarians who teach users how to use AI search tools are abusing professional trust because the tools are imperfect and can lead to errors.
This is a bad argument dressed up as an ethical one. It rests entirely on the premise that AI search tools are imperfect, as though that distinguished them from anything else we teach.
e.g. Name the flawless tool, we libraries promote?
Our job, properly understood, is to teach people to use imperfect tools well, with appropriate scepticism and a clear understanding of what each tool can and cannot do. Refusing to teach AI search tools because they are flawed is not an act of professional integrity. It is an abdication of the actual job.
It also leaves users to figure these tools out on their own. That is the worse outcome by every measure.
A common argument among librarians goes something like this: Emily Bender says LLMs are stochastic parrots; therefore, LLMs can never be useful.
There is an immediate problem with that argument. Even if you accept the \"stochastic parrot\" description, it does not tell you whether LLMs combined with other technologies can be useful. It says nothing directly about retrieval-augmented generation, tool use, calculators, citation validators, structured workflows, human review, or other harnesses wrapped around the model.
The more damaging problem is that Bender herself has clarified that \"stochastic parrot\" is not an argument that LLMs are useless. In her account, it is not even an empirical hypothesis. It is a description or metaphor for systems that generate fluent linguistic form without grounding in communicative intent, a model of the world, or a model of the reader's state of mind6.
This does not mean Bender thinks LLMs are broadly useful. Her position is far more sceptical than that. She has warned that synthetic text is not an information source, and that using chatbots as reliable sources of knowledge is a serious category mistake.
But she has acknowledged limited possible uses, including \"tip-of-the-tongue\" search, language-learning dialogue partners, non-player characters in games, and non-generative uses of language models in classification, speech recognition and machine translation. She treats summarisation more cautiously, because it can introduce material not present in the source.
Nor does this mean Bender has retreated from the stronger \"form versus meaning\" argument. In Bender and Koller's 2020 paper, understanding is defined as mapping language to something outside language. Their claim is that a system trained only on linguistic form has no basis for learning that mapping, because it has access only to patterns in text, not to the extra-linguistic world those texts are about.
That is a serious argument. But it should not be flattened into the much weaker claim that LLMs can never be useful.
So the better conclusion is not \"stochastic parrots can never be useful\" (though she is currently very skeptical). It is: do not mistake fluent synthetic text for grounded understanding or reliable information. That is a much narrower, stronger, and useful warning but does not address the question on usefulness.
But it leaves room to ask the question that actually matters for librarians: under what conditions, with what scaffolding, for which tasks, and with what verification, can LLM-based systems be made useful rather than misleading?
The lesson from Google Scholar is not that librarians should embrace every flawed tool users like. It is that \"flawed\" and \"useless\" are not synonyms.
It is hard to compare like for like, but I think it is fair to say that the practical gains in LLM-powered tools from 2023 to 2026 have been faster and larger than Google Scholar's improvements across its first decade. But the more important point is not the scale of improvement. It is that Google Scholar's improvement did not fundamentally fix its transparency problem. Instead, users learnt that the flaw was either less fatal than it first appeared, or manageable with the right habits.
That is the lesson librarians need to take seriously now.
If the objection is environmental cost, make the environmental argument. If it is labour exploitation, make the labour argument. If it is vendor lock-in, inequity, surveillance, weak governance, or commercial capture, make those arguments. They are serious enough to stand on their own.
They do not need a backdoor return to the claim that AI tools cannot really be useful.
That move is increasingly unconvincing. It usually begins with a reluctant concession: of course AI can sometimes be useful. Then, when the discussion turns to teaching, adoption or institutional support, the old premise quietly reappears. The tools are flawed, so using them must be irresponsible.
But that is not how librarians treat tools.
We teach flawed systems all the time. We teach Google Scholar while warning about coverage and metadata. We teach Scopus and Web of Science while explaining their selectivity. We teach discovery layers while knowing their indexing and ranking are imperfect.
The professional act is not pretending tools are flawless. It is teaching people where they help, where they fail, and how to verify what matters.
So reject an AI tool because the cost is too high, the governance is too weak, the evidence is too thin, or the institutional incentives are wrong.
Just say that.
Do not dress those objections up as proof that the tool can never be useful. That argument has already lost.
The question now is not whether AI search tools can be useful. It is which uses are worth the cost, which are not, and what role librarians should play in helping users tell the difference.
There are many similarities to the Innovator's Dilemma argument. Early users may value dimensions that experts discount, and a tool that performs poorly against established professional criteria may still become useful enough to reshape practice. But unlike the Innovator's Dilemma, I refer to cases where the alternatives co-exist.
Google Scholar offered no stable, auditable list of indexed sources or records, and even Google's own guidance effectively tells users to test coverage by sampling titles rather than consult a definitive inventory.
As confirmed by many studies. The amount of full-text indexed by Google Scholar is also believed by many to be unmatched.
To be clear, Bender is not being cited here as making the same claim as Marcus or Mitchell that LLMs are broadly useful. Her position is much narrower and more sceptical. In interviews, She has said that safe and beneficial uses of synthetic text are hard to identify, but has offered tentative examples especially \"tip of the tongue\" search, where a user describes something in order to recover the name of it and can then verify it through ordinary search. She also distinguishes text generation from other uses of language models, saying that language models can have positive uses in classification, automatic speech recognition, and machine translation, while treating summarisation as more borderline because it can introduce material not present in the source. The point here is therefore not that Bender endorses LLMs as broadly useful, but that \"stochastic parrot\", in her own account, is not an empirical hypothesis or an argument that LLMs have no possible utility. It is a description or metaphor for language-mimicking systems, and the 2021 paper was about the risks and harms of pursuing ever-larger language models, not a general paper about \"AI\".
The improvement of LLMs between 2022 to 2026 is far larger than from 2004 to 2024! This improvement comes from both improvements in the models as well as the use of harnesses like Claude Code to combine deterministic code with LLM flexibility.
To be clear, this does not mean Bender has retreated from the stronger \"form versus meaning\" argument. In her account, the closest thing to an argument in this area is Bender and Koller's 2020 paper, which defines understanding as mapping language to something outside language. Their claim is that a system trained only on linguistic form has no basis for learning that mapping, because it only has access to patterns in text, not to the extra-linguistic world those texts are about. This is separate from the \"stochastic parrots\" phrase itself, which Bender describes as a metaphor rather than an empirical hypothesis. She also notes that multimodal systems complicate the picture: image-text models may meet the Bender and Koller definition of understanding in a very thin sense, because they can map between linguistic strings and images. But she argues that the stochastic-parrot framing remains relevant to such systems and to systems built around them.
Note to readers: some of these ideas made it into a commentary I wrote with Christine Cuskley:
\n\n\n\nDingemanse, Mark & Cuskley, Christine (in press). For robust research, center values, not technology. Behavioral and Brain Sciences. Preprint doi: 10.5281/zenodo.18944023
\n\n\n\nOne topic that often comes up when discussing LLM technology in relation to research integrity is one that I will describe as seeking permission. When looking at the ethical, legal, and societal harms imposed by LLMs (and there are many), sometimes people feel the message ends up altogether too negative. How about this use case I heard of? Aren’t some people getting something useful out of it? Surely we can’t ban the tech outright?
\n\n\n\nOften this is phrased as a concern about messaging (people won’t accept it if we tell them how bad it is; you need to sugarcoat it by also mentioning something nice). Sometimes it is phrased as a majority argument: it’s already here, everyone is using it, surely it can’t be that bad? (Smoking would like a word.) Sometimes it is a concern about missing the boat: these are skills we need, telling people not to use it is like telling them to go back to quill pens. 1
\n\n\n\nWhat I think is going on in these kinds of cases is that the central thrust of the argument is being missed. When it comes to research integrity, the key is a values-first perspective rather than a tech-first perspective.
\n\n\n\nA values-first perspective asks: how can we best uphold the values and standards that make our research robust, reproducible and future proof? A tech-first perspective asks: yeah, but how can I use this technology? It puts technology above values. It seeks permission but sidesteps the question of values.
\n\n\n\nOne example that came up in a recent session started with literature review: surely it’s not too harmful, a questioner said, to use an LLM for a first stab at a literature review? Multiple participants pushed back on this, saying that actually, LLMs don’t reliably summarise. Also, they provide only the most average consensus view; we don’t know what’s being left out. And LLMs by nature regurgitate without understanding; can we actually identify confidently produced bullshit in a field that we don’t master fully? Further, reading is a hard-won skill: tracing arguments, spending time with papers, separating substance and rhetorics; surely we don’t want to lose this skillset.
\n\n\n\nA retreat followed: well then, maybe literature review was not the best example, but surely there is some other kind of use that is okay? Someone mentioned programming. Folks brought up deskilling, the security risks of vibe-coding, technological debt. Oh well, that’s not the kind of programming I meant. And so on.
\n\n\n\nWe can play this game all day: seek permission for an edge case, retreat to another case when trouble arises, appeal to amiability or tolerance. Two things to note when a conversation gets to this point. First, it’s a misrepresentation: reluctance to grant permission is portrayed as an intolerant move, a wish to take the other’s toys, when it is nothing like that. What do you need my permission for, anyway? Second, it’s a distraction: it moves the conversation away from values and towards a tech-first perspective.
\n\n\n\nThis is where a values-first perspective can help cut through the knots. The approach is not to recommend or forbid particular tech products, or particular uses; it is to provide a matrix for mindful choice. For any attested or conceivable use, you can ask: how would this help or hinder my upholding of high standards of research integrity? If we consider the pushback voiced in response to the ‘literature review’ use case, you can see they appeal to the same core values:
\n\n\n\nThese values do not come out of thin air; they’re straight from a widely adopted code of conduct for research integrity (NCCRI, 2018). They are also not controversial; most scientists will recognise them as principles that characterize robust research. I’ve written about them before, e.g. in my guidance on GenAI and in my post on why synthetic text has no place in science blogging.
\n\n\n\nIf you use these values as a compass to steer by, it’s easier to navigate the landscape of technology use. On the other hand, if you find yourself seeking permission, one useful thing to do is to step back and inspect the underlying value conflict.
\n\n\n\nAs you move from a tech-first to a values-first perspective, the question shifts from “won’t you give me permission?” to “how do I do the best science possible?”. And that, to me, is a question worth asking.
\n\n\n\nIn language, semantics describe the names and meanings of words. The\nbioinformatics community has aptly adopted biosemantics as a concept that\nencompasses the issues with the names and meanings of biological entities,\nusually in natural language processing and data integration. However, semantics\ndoes not capture the context of words, and biosemantics fails to describe the\nbiological context and complex relationships between biological entities.
\n
Pragmatics goes beyond semantics and describes the context of words. Because of\nthis parallelism, I've begun to use the term biopragmatics to describe the\nfamily of computational approaches aimed at identifying and contextualizing the\ncontext of biological entities.
","doi":"https://doi.org/10.59350/j4vty-rrr02","guid":"https://cthoyt.com/2020/01/22/biosemantics-versus-biopragmatics","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1579651200,"rid":"6qadr-jvy42","summary":"In language, semantics describe the names and meanings of words. The bioinformatics community has aptly adopted biosemantics as a concept that encompasses the issues with the names and meanings of biological entities, usually in natural language processing and data integration. However, semantics does not capture the context of words, and biosemantics fails to describe the biological context and complex relationships between biological entities.","tags":["Semantics","Meta"],"title":"Biosemantics vs. Biopragmatics","updated_at":1781539932,"url":"https://cthoyt.com/2020/01/22/biosemantics-versus-biopragmatics.html","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"How many molecular biology papers have you read today? This week? This month? If\nyou're like me, its not so many, and we're falling behind very quickly. Here's a\nchart made by the new PubMed that summarizes how many papers were published\nmentioning RAS in the last 50 years.
\n
There were 4,483 publications listed in 2019. We can't read that much, and even\nif we did, we couldn't remember it all. That's why we need to take the knowledge\nout of the unstructured text and store it in a structured form that can be read\nand stored in computers. This way, we can easily share it, query it, and write\nalgorithms that can help us reason about the incredible amount of biological\nknowledge out there.
\nThere are several formats in which this kind of information can be stored on a\ncontinuum between directly representing mechanistic biology to representing the\nknowledge itself. In the popular middle ground are BioPAX and BEL, which I'll\ncome back to in future posts.
\nIt's important to keep in mind that knowledge needs to be curated - this can\neither be manual, through natural language processing, or a mixture of both.\nI've written\na paper\non such a process, but for now this post should motivate a few following ones\ndescribing what it takes to deal with nomenclature, build ontologies, and then\nstart extracting mechanistic biology from the literature.
","doi":"https://doi.org/10.59350/jt101-b1374","guid":"https://cthoyt.com/2020/01/23/encoding-biology-in-kgs","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1579737600,"rid":"qyhqj-cs605","summary":"How many molecular biology papers have you read today? This week? This month? If you're like me, its not so many, and we're falling behind very quickly. Here's a chart made by the new PubMed that summarizes how many papers were published mentioning RAS in the last 50 years.","tags":["Knowledge Graphs"],"title":"Encoding Biology in Knowledge Graphs","updated_at":1781539915,"url":"https://cthoyt.com/2020/01/23/encoding-biology-in-kgs.html","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"The other day I saw a tweet lamenting the drag that is literature review during\npreparation for writing your thesis.
\n\nI just love writing 15 page literature reviews for graduate school courses on literally any topic except my thesis topic.
\u2014 PhD Diaries (@thoughtsofaphd) February 7, 2020
I agree. I felt the same pain last fall when I wrote\nmy doctoral thesis. Luckily, I had\na strategy that made it a bit easier.
\nI learned it from one of my professors when I was doing my master's degree in\nLife Science Informatics. Each semester, we had a seminar course in which each\nstudent was assigned research articles to read and present to the class with a\nshort slide deck. Later, I joined his research group and realized that this\ncourse served as a literature review for him just as much as us.
\nSo later when I was a Ph.D. student, I volunteered to run the seminar. I\nco-opted the concept, and planned the course to cover many of the topics I found\ninteresting for my thesis. I already knew some of the papers very well, and a\nfew were ones I had always been meaning to read. I tried to pick the most recent\npapers for topics when possible, but also threw in a few classics as well.
\nOn the first day of the seminar, I shared the following course information. I\nthought it was important to make clear what my expectations were for students in\nterms of their prior knowledge. Since they all came from the same master's\nprogram, I thought it was enough that they had passed one of the first semester\nlectures called \"Biological Databases\" which was about many of the resources and\ndatabases used in the systems and networks biology community. I also outlined\nwhat was the content for the course, what was expected, etc. then shared this\nall as a Google Doc so they could read it over and add comments.
\nI also made a list of possible papers and a tentative schedule that students\ncould look over and decide which papers they found most interesting. The topics\nwere arranged in a logical order to tell the story of my thesis, and for each\nsection there were a few papers that I thought were very important, and a few\nextras just in case there was a lot of interest. During the first day of the\nseminar, I also went through the list of all papers and explained the topics to\nthe students. I gave them this list via Google Docs as well, and they were able\nto claim papers for their presentations. Below, I've listed the final list of\npapers and the order in which they were presented. We were able to come to\nagreements for all students to present the papers I found most important. Maybe\n40% of the class found a paper interesting and picked one the first day and the\nrest took the next week to decide, ask questions, or propose new papers.
\nAnother consideration I had when picking this paper list was to choose work done\nby my colleagues that I found interesting and helpful. After, I invited them to\ncome listen to the seminar and mediate discussion after. We were able to invite\none of my collaborators Mehdi Ali (he's a really good guy!) to discuss his work\non using deep learning for relation extraction in natural language processing. I\nthink that might have been the most engaging day of the whole series.
\nI added one aspect to this course compared to the previous seminar that I had\nattended: each student was not only responsible for presenting the paper that\nhad been assigned from my list, but they were also responsible for finding a\nrelevant pre-print (in the same or similar topic) and submitting a peer review\nthrough the pre-print system. When I was a student, I noticed many students did\nnot read the references of the paper they were assigned in our seminars, and\nalso had not considered other similar research to their paper. Asking them to\nfind their own papers was a way to make this a more creative and fun process,\nand would directly prepare them to answer questions at the end of the\npresentation like \"what will the authors do next?\" or \"how will this research be\nused by others?\"
\nOne of the funny things that happened during the pre-print presentations is the\nstudents found several of mine and presented those. I suppose this was\ninevitable given the contemporary nature of my work in the context of the topics\nchosen. I would actually explicitly encourage students to check out my\npre-prints the next time I host a seminar, because I know the work very well and\ncould mediate a nice discussion.
\nI learned a lot through the process of preparing this seminar. Its outline\nbecame the outline for my thesis, and a lot of the discussions became points\nthat I addressed explicitly in my writing. I wouldn't say that I was taking\nadvantage of the students in this process - we all benefited from the\nexperience. I hope you get some ideas about how you might be able to do this\nyourself, whether you're a doctoral student, a postdoc, or something else!
\nStudents should be comfortable with the material presented in the Biological\nDatabases lecture during the first semester of the LSI curriculum.
\nStudents will have the opportunity to practice reading, presenting, and\ndiscussing recent biomedical literature on the topics of knowledge assembly,\ndata integration, and modeling in systems and networks biology.
\nStudents will be assigned papers and present on the holistic process of\nknowledge discovery in systems and networks biology that focus on the topics of\nknowledge assembly (e.g., natural language processing, modeling formalisms and\nformats, reasoning techniques), data integration (e.g., practical scenarios\nfocusing on techniques on the data level, knowledge level, and analytical\nlevels), and modeling strategies (e.g., rule-based modeling, agent-based\nmodeling, mathematical modeling, hypothesis generation with knowledge-based\napproaches).
\nStudents will be assigned an article to read and present during a thirty (30)\nminute lecture. One goal of this lecture is to show an understanding of not only\nthe material presented in the article, but also the relevant background\ninformation - this may entail following the references and reading other\narticles. Another goal is to not only educate, but entertain the audience.\nStudents will also be expected to find a relevant pre-print article on arXiv,\nbioRxiv, or other pre-print server and post a peer-review for the author on the\ncorresponding service. Following the presentation of their assigned article,\nstudents should include slides (1-3) briefly explaining the relevance of the\npre-print that they found.
\nStudents will be assessed on the understanding of their assigned topic, the\nquality of their presentation, and their participation. Students missing more\nthan 2 seminars will not pass the course without a doctor's note.
\nThis week there will a short discussion of the syllabus and no presentation. For\nthose in Bonn that aren't aware of this wonderful tradition, welcome to Syllabus\nWeek.
\nMubassher Leser, U., & Hakenberg, J. (2005).\nWhat makes a gene name? Named entity recognition in the biomedical literature.\nBriefings in Bioinformatics, 6(4), 357\u2013369.
\nPre-print: https://www.biorxiv.org/content/early/2017/03/08/115022
\nBachman, J. A., Gyori, B. M., & Sorger, P. K. (2018).\nFamPlex: A resource for entity recognition and relationship resolution of human protein families and complexes in biomedical text mining.\nBMC Bioinformatics, 19(1), 1\u201314.
\nPre-print: https://www.biorxiv.org/content/early/2018/07/29/379446
\nLaibe, C., & Le Nov\u00e8re, N. (2007).\nMIRIAM Resources: tools to generate and resolve robust cross-references in Systems' Biology.\nBMC Systems Biology, 1, 58.
\nPre-print: https://www.biorxiv.org/content/early/2017/07/24/167619
\nJuty, N., Le Nover\u0300e, N., & Laibe, C. (2012).\nIdentifiers.org and MIRIAM Registry: Community resources to provide persistent identification.\nNucleic Acids Research, 40(D1), 580\u2013586.
\nPre-print: https://www.biorxiv.org/content/early/2018/02/14/101279
\nNovichkova, S., et al. (2003).\nMedScan, a natural language processing engine for MEDLINE abstracts.\nBioinformatics, 19(13), 1699\u20131706.
\nPre-print: https://www.biorxiv.org/content/early/2018/08/29/403667
\nAli, M., et al. (2017).\nAutomatic Extraction of BEL-Statements based on Neural Networks.\nProceedings of BioCreative VI Challenge and Workshop, (October).
\nPre-print : https://osf.io/j76y3/
\nDemir, E., et al. (2010).\nThe BioPAX community standard for pathway data sharing.\nNature Biotechnology, 28(12), 1308\u20131308.
\nPre-print: https://www.biorxiv.org/content/10.1101/192856v1
\nHucka, M., et al. (2003).\nThe systems biology markup language (SBML): a medium for representation and exchange of biochemical network models.\nBioinformatics (Oxford, England), 19(4), 524\u201331.
\nPre-print: https://www.biorxiv.org/content/early/2018/01/23/246470
\nLe Nov\u00e8re, et al. (2009).\nThe Systems Biology Graphical Notation.\nNature Biotechnology, 27(8), 735\u201341.
\nPre-print: https://www.biorxiv.org/content/early/2018/01/30/256750
\nCarbon, S., et al. (2017).\nExpansion of the gene ontology knowledgebase and resources: The gene ontology consortium.\nNucleic Acids Research, 45(D1), D331\u2013D338.
\nPre-print: https://www.biorxiv.org/content/early/2018/10/07/437020
\nCroft, D., et al. (2014).\nThe Reactome pathway knowledgebase.\nNucleic Acids Research, 42(D1), D472\u2013D477. AND Fabregat, A., et al.\n(2018).\nThe Reactome Pathway Knowledgebase.\nNucleic Acids Research, 46(D1), D649\u2013D655.
\nPre-print: https://www.biorxiv.org/content/early/2018/10/09/375097
\nCerami, E. G., et al. (2011).\nPathway Commons, a web resource for biological pathway data.\nNucleic Acids Research, 39(SUPPL. 1), 685\u2013690.
\nPre-print: https://www.biorxiv.org/content/early/2018/09/03/353235
\nKhatri, P., Sirota, M., & Butte, A. J. (2012).\nTen years of pathway analysis: Current approaches and outstanding challenges.\nPLoS Computational Biology, 8(2).
\nPre-print: https://www.biorxiv.org/content/early/2018/09/13/416131
\nGligorijevi\u0107, V., & Pr\u017eulj, N. (2015).\nMethods for biological data integration: perspectives and challenges.\nJournal of The Royal Society Interface, 12(112), 20150571.
\nPre-print: https://www.biorxiv.org/content/early/2018/12/13/358390
\nSaqi, M., et al. (2018).\nNavigating the disease landscape: knowledge representations for contextualizing molecular signatures.\nBriefings In Bioinformatics, (May), 1\u201315.
\nPre-print: https://www.biorxiv.org/content/early/2018/11/23/475202
\nHimmelstein, D. S., et al. (2017).\nSystematic integration of biomedical knowledge prioritizes drugs for repurposing.\nELife, 6.
\nPre-print: https://www.biorxiv.org/content/early/2018/10/13/442640
\nLopez, C. F., et al. (2013).\nProgramming biological models in Python using PySB.\nMolecular Systems Biology, 9(646), 646.
\nPre-print: https://www.biorxiv.org/content/early/2018/12/23/503359
\nGyori, B. M., et al. (2017).\nFrom word models to executable models of signaling networks using automated assembly.\nMolecular Systems Biology, 13(11), 954.
\nPre-print: https://www.biorxiv.org/content/early/2018/05/15/322156
","doi":"https://doi.org/10.59350/4t3sf-mab09","guid":"https://cthoyt.com/2020/02/09/seminar-for-thesis-writing","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1581206400,"rid":"9h7sp-08z86","summary":"The other day I saw a tweet lamenting the drag that is literature review during preparation for writing your thesis.","tags":["Doctoral Thesis","Teaching"],"title":"Host a Graduate Seminar Before Writing Your Thesis","updated_at":1781539913,"url":"https://cthoyt.com/2020/02/09/seminar-for-thesis-writing.html","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"We've all been there. You started a new branch from master. You had a very\nspecific goal in mind, The Original Goal. You made a pull request (PR) to go\nwith it, too, The Original Pull Request. But then, you had an idea! And\nalso, someone on your team asked you to solve another problem! Now the original\ncode you wrote to address The Original Goal relies on that code \u2026 and now\nyou've got dozens of files changed, hundreds of lines of diff, and nobody\n(including you) can understand what you've done. Like I said, we've all been\nthere. Here's what you can do to fix it:
\nDon't do anything rash. Git is a pain to use, and you're going to have to rely\non it to keep a history for you of what you've done.
\nFirst, you're going to have to take a big step back. Write a summary of all the\nthings you've done in The Original Pull Request. This should be about what\nthe PR does and why it does it. Of course it could vary depending on the\nsituation, but this summary shouldn't be about exactly how the PR does it,\nbecause the implementation details are likely what lead to this situation in the\nfirst place.
\nKeep in mind that every PR has a box at the top that's used to describe what's\nin it. This is where you will put your summary.
\nOf all the things that The Original Pull Request, some of them are\nself-contained, and some of them rely on each other. It was probably the case\nthat to accomplish The Original Goal, you had to address lots of smaller\ngoals. You probably also had to change lots of code and write new code too.
\nWouldn't it have been nice if all of these implementations were already done,\nbecause then you could have just solved The Original Goal directly by\nusing/applying previous code. That's what we're going to aim for.
\nBut first, you need to figure out which things you did relied on which other\nones, because you're going to break The Original Pull Request up until it\nexactly matches up to addressing The Original Goal. don't have any
\nAfter you understand which parts of The Original Pull Request depend on each\nother, pick one independent part of the code that accomplishes one sub-goal.\nSince you're not doing this to be a martyr, and we all know git is too\ncomplicated to Do It Right, you're going to copy/paste the files that are\nrelated to this change to your desktop*.
\nBefore continuing, you're going to make sure all of the code in your big messy\nbranch for The Original Pull Request is committed and pushed. Even though we\nwant to supersede what's there, it never hurts to keep track of your descent\ninto madness.
\nAfter there's nothing lying around, switch back to master. If your team has\ntaken good care of your repository, the master branch should be undisturbed by\nthe chaos you've created in The Original Pull Request. Make a new branch\nfrom master, and name it appropriately for fixing the one sub-goal, from here\nout known as The Sub-Goal that you identified in Step 4. Now you can start\nupdating the relevant files in your repository based on the files you copied to\nyour desktop. I suggest you don't copy/paste the contents of the whole files,\nbecause you might have forgotten about something else you changed in them. After\nall, you're reading my guide because this was a mess.
\nOnce you've finished making the new branch for your independent part of code\nthat solves The Sub-Goal, you can make The New Pull Request.
\nYou will now go through the entire process of writing a good summary of this\nbranch for your co-developers, you will get their feedback, you will make\nupdates, pass flake8, and so on. They will thank you for having code that\naccomplishes one thing, and can be described simply. They will thank you for not\nhaving too big of a diff, and for the things in the diff all being relevant and\nimportant. Then you can merge this branch into master.
\nThroughout transferring the code for The New Pull Request you have probably\nrealized there are some things you did back in The Original Pull Request\nthat you could do better, and made some updates in the code in The New Pull\nRequest to reflect the wisdom you've gained along the way. That's great!\nCongratulations!
\nAfter your team has approved The New Pull Request, you can merge it into\nmaster and both delete the branch locally and on the remote. Then you should\nswitch back to the master branch. You can pull from master, and see your code\nthat solved The Sub-Goal reflected here.
\nThis is the hard part. Now you have to switch back to the branch for The\nOriginal Pull Request. Now you have to update this branch from master. It's\ngoing to be hard because now you've probably made different changes in The New\nPull Request than in The Original Pull Request so there will likely be\nconflicts.
\nThis is not a tutorial on how to solve merge conflicts. Use google to figure\nthat out
\nI can't understate: do this part really well. If you don't, then the history\nin the original branch will be even more incomprehensible, and you won't be able\nto tell if you lost any of your original work. Please, please, please do this\nwell.
\nP.S. Like I said before, don't be a martyr. Use tools like GitHub Desktop and\nPyCharm to help you merge. I heard that the git CLI was allegedly created by\nLinus Torvalds to slow other developers down.
\nWhy are we going through all of this pain, rather than just pushing your team to\nlet you merge The Original Pull Request? The reason you have to do this is\nbecause now all of the changes that addressed The Sub-Goal are part of\nmaster, and are no longer part of the diff of The Original Pull Request.
\nNow you're one step closer to your team being able to understand, review, and\neventually merge The Original Pull Request.
\nThis is the frustrating part. After you've gone through all of that work to\nsplit a tiny part of The Original Pull Request into a smaller, independent\npull request, you're not done. You will probably have to repeat steps 4-8 a few\ntimes. You'll be tempted to throw away the branch for The Original Pull\nRequest and maybe start over.
\nDon't do that.
\nIf you do, the same disorganization that lead to the mess of The Original Pull\nRequest might just slip back into whatever you do next. Even worse, nobody\nelse will be able to follow what you've done until now.
\nSo relax. This is going to take a few days. You're going to have to wait in\nbetween several iterations for feedback. That's good. You need feedback. I need\nfeedback. We all need to practice getting it and giving it. Embrace the\nopportunity to have your team help you improve your code, gain wisdom, and make\nyour contributions sustainable.
\nEventually after several iterations of 4-9, you will have excised all of the\ncode that was important for The Original Pull Request, but not directly\naccomplishing The Original Goal. As you removed independent parts, new parts\nbecame independent themselves. Eventually, The Original Pull Request will\nindeed match up exactly to The Original Goal, then you will be able to come\nback to it for review and merging.
\nI understand this is a frustrating process. The purpose of these steps were to\nhelp you think through a large piece of work you've done. You should be proud\nthat you've solved a complex problem with many intricate parts. It was a lot of\nextra work to break it into many pull requests, and it might have taken more of\nyour time the first time working through this process, but in the future, this\nmight help you to start with small tasks rather than addressing The Original\nGoal all at once. GitHub, for example, has an issue tracker that is very\nhelpful for this. I imagine that each issue should correspond to a Sub-Goal,\nand that each should have exactly one PR that addresses it. The Original\nGoal also deserves its own issue that points to all of the issues for its\nsub-goals. Eventually you will address this with a beautiful PR as well. Happy\ncoding!
\n*If you're thinking, why don't I use cherry picking? If you know what cherry\npicking is in the context of git (and also how to use it) then you probably\nwon't have the issue that prompted this blog post. But also, you should go\noutside and pick some apples instead. Thanksgiving is never more than a few\nhundred days away. It pays to be ready.
\nIt might be illustrative to see where an example of where this was done in\npractice, so I'll share some work I did with a text mining tool from Harvard\nMedical School, Gilda. It's a simple yet\npowerful system for grounding of named entities based on dictionary lookup.\nUnfortunately, it didn't include some dictionaries I wanted, and it didn't have\na UI to go with its web API.
\nSo I set out on figuring out how it generated dictionaries, where it stored\nthem, and how it loaded them to make the web app. I ended up making several\nmodifications to accomplish this goal, but it was a huge PR. I've definitely\nannoyed the author, @bgyori, with PRs that are too\nbig before, which he was ultimately not able to understand or merge.
\nKeep in mind, in your team, your teammates might be obligated to help you\nbecause you're working towards a common goal, getting paid, etc. When you're in\nthe open source world, nobody really owes you anything, so you it's in your best\ninterest to make things as easy as possible on the package's maintainer(s).
\nSo I made a few different pull requests that were all totally independent:
\nMaybe you're seeing a theme here. I was improving lots of different bits of\nGilda so I could reuse the package in new code later. The next incremental\nincrease was:
\nAnd finally with these in place, I realized that adding a web interface was\nparallel to my original goal, but not the core. What was really important was\nthat throughout all of the Gilda functionality, I could load my own synonym list\n(which I'd generate using the HPO, EFO, and DOID). I was able to address the UI\nwith:
\nAt the time of writing, we're still working through this PR. But all of it is\nleading up to the point where I can load my own files into this web interface.\nIt will seem so obvious to Ben when I send this PR next (but after giving him\nsome space\u2026 I did just bombard him with 5 PRs in a few days) what I am trying\nto accomplish and why.
\nWant to see what happens when you try and do all of this in one PR? You will\ncorrectly guess that the PR is a total mess, impossible to understand, and\nriddled with questions that are really too big to answer when your head is\nalready so far in the sand. Behold, in all its infamy, my failed PR from last\nsummer (#4). At this point, you\ncan't even see what a mess it was from the linked web page but if you go back\nthrough the version history before I broke it into 5 smaller PRs (using the\nworkflow described above) it was a monolith.
","doi":"https://doi.org/10.59350/3cn95-h9y94","guid":"https://cthoyt.com/2020/03/20/how-to-fix-your-monolithic-pull-request","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1584662400,"rid":"w8wqq-y9x71","summary":"We've all been there. You started a new branch from master. You had a very specific goal in mind, The Original Goal. You made a pull request (PR) to go with it, too, The Original Pull Request. But then, you had an idea! And also, someone on your team asked you to solve another problem!","tags":["Code With Me"],"title":"How to Fix Your Monolithic Pull Request","updated_at":1781539911,"url":"https://cthoyt.com/2020/03/20/how-to-fix-your-monolithic-pull-request.html","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"A few months ago, the question was posed on science Twitter: \"How many people\nhave published on ChemRxiv?\"
\n\nmakes me wonder about the stats at @ChemRxiv https://t.co/Ml5X8F4ckJ
\u2014 Egon Willigh\u24d0gen (@egonwillighagen) January 20, 2020
It was a good day for me, which meant I was in the mood to take up the first\nchallenged posed on Twitter. I found that Fran\u00e7ois-Xavier Coudert\n(@fxcoudert) has previously written a\npython client\nfor ChemRxiv. I made a pair of pull requests\n(fxcoudert/tools#9 and\nfxcoudert/tools#10) to fix some\nbugs and make it importable from other python modules.
\nUnlike BioRxiv, the pre-print server for biology, ChemRxiv is implemented with\nFigShare. It turns out that all FigShare \"institutions\"\nlike ChemRxiv are actually accessible through the main\nFigShare API. I think this is pretty cool, and\nmade sure that the ChemRxiv client that I had updated was actually able to be\nrun for any institution. Fun fact: the institution code for ChemRxiv is 259.
I got to work writing my\nown repository to wrap the\nclient, take care of downloading all of the bibliographic information available,\nand generating some pretty pictures. I originally ran the scripts and generated\npictures on January 20th, 2020 (the day Egon posed the question). Since the\npandemic has got the whole science community introspecting, I came back to this\ntoday and thought it might be worth writing up as a blog post.
\nWithout further ado, here are the most recent charts I've generated to answer\nthree main questions. I've linked the images in such a way that the charts will\nbe automatically updated with my GitHub repository. This also implicitly means\nthat there's a history of each image, but because two of them are plotting time\ncourse information, the history is already conveyed within the chart.
\nHow many papers were submitted each month to ChemRxiv? Keep in mind that the\ncurrent month is likely not complete.
\n
This only counts using the ORCID iDs of the first authors; it's pretty\ninconsistent what other identifying information is included in the metadata for\neach article.
\n
How many authors submitted more than once per month? This chart shows spikes in\nAugust, which I will guess is when most people are submitting before their\nsummer breaks :)
\n
How many first authors have historically contributed to ChemRxiv at each month?\nWe can take the first date of authorship for each author then count at each\nmonth how many unique first time authors there are. Then, we can use a\ncumulative sum to show how many authors have contributed to ChemRxiv at any\npoint in time.
\n
If we aggregate the data, we can ask how many authors have submitted lots of\narticles:
\n
The following chart shows the popularity of different licenses over time. The\nCC BY-NC-ND 4.0 license is\na resounding victor. You can learn about Creative Commons (CC) licenses\nhere.
\n
If you're interested to regenerate these charts yourself, you're welcome to do\nso with the following code:
\ngit clone https://github.com/cthoyt/chemrxiv-summarize\ncd chemrxiv-summarize\npython 01_download.py\npython 02_process.py\npython 03_visualize.py\nDownloading takes a bit of time (about 40 minutes) but there's a tqdm bar to\nkeep you entertained in the mean time. Normally I package all of my code, but\nthe one off scripts here didn't seem to warrant it.
As a final note, I'd like to shout out to Marshall Brennan\n(@Organometallica) for being an excellent\nspokesperson and public face of ChemRxiv. Also, throughout this process I\nrealized he also was a chemistry major in his bachelor's at Northeastern\nUniversity like me. Go huskies!
\nMay 2020 Update: Fran\u00e7ois-Xavier Coudert created the\nChemRxiv-Dashboard, which makes some\nsimilar summaries to this. Check it out!
\n\nI made a dashboard for @ChemRxiv, fed by the @figshare
\u2014 FX Coudert (@fxcoudert) May 19, 2020
metadata API.https://t.co/rKyAOGkrVO pic.twitter.com/fLfjEabraz
November 2020 Update: I added a license chart and made some changes to enable\nthis repo to be much more easily used for other FigShare institutions. If you've\nfound this post from @figshare's\ntweet and want help\nmaking these charts for your FigShare institution, please feel free to @ me on\nTwitter or send me an email.
","doi":"https://doi.org/10.59350/n7rjr-90f02","guid":"https://cthoyt.com/2020/04/15/summarizing-chemrxiv","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1586908800,"rid":"3ky5b-cne42","summary":"A few months ago, the question was posed on science Twitter: \"How many people have published on ChemRxiv?\"","tags":["Bibliometrics"],"title":"Summarizing ChemRxiv","updated_at":1781539909,"url":"https://cthoyt.com/2020/04/15/summarizing-chemrxiv.html","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"We have a big problem in the bioinformatics community with namespaces,\nidentifiers, and names. And nobody's posed the question better than\nRihanna herself.
\nDuring my Ph.D. at Fraunhofer, one of the old text miners reminisced to me about\nthe late 90's and early naughties when they had to curate their own dictionaries\nof synonyms for entities. I was lucky enough to have joined the bioinformatics\ncommunity after excellent nomenclature resources like\nChEBI and the HGNC\nwere established and accepted by the community as gospel.
\nI consider these sources excellent because it's quite easy to get a list of the\nidentifiers and corresponding names that they maintain (TSV, etc.). There are\nother nomenclatures, like the\nExPASy Enzyme Classes, that\nare stored as text files in non-standard formats.
\nThe Open Biomedical Ontology (OBO) format and\nOBO Foundry were first published in\n2007 as a solution for standardizing\na growing set of biomedical ontologies that few shared semantics. Many ontology\nmaintainers adopted their format, or at least used the OWL to OBO converter\ntools to include their ontologies in a reusable format. However, there remain\nsome notable holdouts like the\nCell Line Ontology that have not begun to\ndistribute their content as OBO.
\nIn parallel, the Ontology Lookup Service (OLS) was\npublished as one of many front-ends for exploring this growing list of\nresources. In comparison, it may have been one of the first tools to provide a\nnice user experience that included a search engine (powered by\nsolr, because they're living in the Java world).
\nBoth are lacking - there does not exist a solid OBO ecosystem (though Martin\nLarralde's pronto may well soon change\nthat) and even worse, the content in OBO loosely follows the standard, at best.\nOn the other hand, the OLS has both an over-engineered interface that isn't\nquite user friendly. For example, if you want to look up programmed cell death\n(GO:0012501), you have to know the internal OLS key for the namespace and the\nPURL for the identifier, which is not so obvious. Then you can finally hit the\nAPI.
\nAnd still, both of them lack some of my favorite, and arguably most important\nnamespaces, like HGNC, RGD, MGI, UniProt, Entrez Gene, and PubChem. As an aside,\ndealing with PubChem is for people operating on a whole different level, so I'm\nnot blaming anyone for dropping the ball on that one. Later, I will confess to\ndoing the same.
\nEven worse, the OBO Foundry and OLS can't even agree on what to call some\nnamespaces. A great example is the NCBI taxonomy database. On the NCBI site,\nthey say that the namespace is called NCBI and compact uniform identifiers\n(CURIEs) should look like NCBI:txid175694, OBO Foundry says the namespace is\nNCBITaxon (one of the few notable mixed-case namespace names) and CURIEs\nshould look like NCBITaxon:175694.
Identifiers.org came along to solve some of these ambiguities with a curated\ndatabase, but it's missing lots of the things in OBO Foundry and OLS, and it\neven disagrees on others. They call the NCBI taxonomy namespace taxonomy and\nsay that identifiers should look like taxonomy:175694. Exhausting!
One more issue is the GOGO problem. Many OBO ontologies use local identifiers\nthat also include the prefix because a given ontology might contain terms\nimported from other ones. However, this means that ontologies that originated\nfrom the OBO world have redundant identifiers, like from GO (e.g.,\nGO:GO:0012501). I know what you're wondering: is Dr. Claw in charge? Maybe.
\nThe reason I went down this rabbit hole is because I want to support people to\ndo better curation. This means I want them to use identifiers instead of ever\nchanging names. For example, it turns out the half life of an HGNC gene symbol\nis very short -\nthousands of them change every year.\nHowever, if I want people to use identifiers instead of names in their\ndatabases, their papers, and other writing, there need to be really good tools\nfor looking up the names that go with each identifier and the cross-references\n(equivalences) to other databases that are talking about the same thing.
\nSo I built PyOBO. It includes tools for\nreading the OBO Foundry and getting all of the OBO resources that are available\n(as well as many manual fixes for incorrect metadata), it uses Daniel\nHimmelstein's Obonet for parsing and\nstoring pre-parsed files for fast loading, and it applies a swath of rule-based\nnormalization that I've\nmanually curated\nby personally reading all of the OBO files, their identifiers, their\ncross-references, relationships, properties, and everything else. When it comes\nto data, there really is no way around getting your hands dirty.
\nI also went ahead and\nwrote parsers and converters\nfor lots of other databases like Entrez, ComplexPortal, InterPro, and others so\nthey could play nice with the rest of the ecosystem. Of course, this is an\nongoing process. There are always more databases to include, and when it comes\nto super-sized ones like PubChem, the paradigms I used might not hold up anymore\n(though I did write parser/converter for it and you're welcome to use it).
\nAfter this long journey of a blog post, I think we're ready to address Rihanna's\nperrenial question: what's my name? Until now, there really didn't exist a\nservice that let you look up the name for an entity by its CURIE. The link I\ngave for the OLS is the closest I have found, and that just doesn't cut it.
\nAfter all of this coding, I wrote a script (just run obo ooh-na-na) that takes\nall of the available sources, normalizes their namespaces, normalizes their\nidentifiers, and dumps them as a big 'ol TSV file. 3 columns - namespace,\nidentifier, and name. No nonsense. Probably legal! Get it at\n.\nI'll make updates periodically as I add more sources, such as if/when I feel\ncomfortable with including the PubChem dump - the\nCID-Title.gz\nfile is about 1.3 gigabytes, which means this will significantly increase the\nsize, but not so much that it's unreasonable.
I can imagine that most people probably won't want to download this file, or\nload it in memory (un-gzipped) every time they want to use it. I wrote a simple\nweb service that wraps this dataset\nincluded in PyOBO.\nIt should be as easy as running with the shell with\npython -m pyobo.apps.resolver then running the following python code:
import requests\n\n# This is an exact match\nsuccessful_request = requests.get('http://localhost:5000/resolve/DOID:14330').json()\n# {\"identifier\": \"14330\", \"name\": \"Parkinson's disease\", \"prefix\": \"doid\", \"query\": \"DOID:14330\", \"success\": True}\n\n# This one remaps the prefix if you get it slightly wrong\nsuccessful_remapped_request = requests.get('http://localhost:5000/resolve/DO:14330').json()\n# {\"identifier\": \"14330\", \"name\": \"Parkinson's disease\", \"prefix\": \"doid\", \"query\": \"DO:14330\", \"success\": True}\n\n# This one can't find the identifier.\nunsuccessful_request = requests.get('http://localhost:5000/resolve/DO:00000').json()\n# {\"identifier\": \"00000\", \"message\": \"Could not look up identifier\", \"prefix\": \"doid\", \"query\": \"DO:00000\", \"success\": False}\n\n# Keep in mind, the point of this service isn't to validate identifiers.\nunsuccessful_crazy_request = requests.get('http://localhost:5000/resolve/DO:thisIsNotRightAtAll').json()\n# {\"identifier\": \"thisIsNotRightAtAll\", \"message\": \"Could not look up identifier\", \"prefix\": \"doid\", \"query\": \"DO:thisIsNotRightAtAll\", \"success\": False}\n\n# No mercy for bad prefixes\nunsuccessful_prefix_lookup = requests.get('http://localhost:5000/resolve/notanamespace:0000').json()\n# {\"message\": \"Could not identify prefix\", \"query\": \"notanamespace:0000\", \"success\": False}\nIt's especially important that the service normalizes curies first, so both\nDOID:14330, doid:14330, and DO:14330 can all be resolved to their name,\nParkinson's disease. Because I did extensive manual curation of namespaces and\ntheir synonyms, NCBITaxon and taxonomy are both acceptable as well. However,\nthis service doesn't load from the aforementioned TSV, but rather takes\nadvantage of PyOBO's internal code for looking up mappings. I can imagine lots\nof ways I might re-write this service to directly take advantage of this dump (I\nalso invite you to do the same, however best suits you) such as loading it into\nEdgeDB and auto-generating a GraphQL endpoint.
The last thing that I'm looking into getting this service hosted so everyone can\nbenefit from it without doing dev-ops in their own organizations. Then I will\ncontinue to obfuscate all usage and documentation with references to pop\nculture. Enjoy!
","doi":"https://doi.org/10.59350/wmj3y-04914","guid":"https://cthoyt.com/2020/04/18/ooh-na-na","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1587168000,"rid":"81da2-mtb29","summary":"We have a big problem in the bioinformatics community with namespaces, identifiers, and names. And nobody's posed the question better than Rihanna herself.","tags":["OBO","Lexica"],"title":"Ooh Na Na, What's My Name?","updated_at":1781539907,"url":"https://cthoyt.com/2020/04/18/ooh-na-na.html","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"On top the issue of resolving identifiers to their\nnames, the bioinformatics community has a\nhard time figuring out when two identifiers from different databases are\nequivalent. You know who else has the same problem? Inspector Javert. Get ready\nfor a Les Miserables-themed post on how to address this long-standing problem.
\nI have to start my tale of woes by disclosing my source material. I loved both\nthe 1985 and 1987 recordings from the respective original London and Broadway\ncasts. But, for the purposes of this post, I will assume that you've seen the\nexcellent 2012 film adaptation of Alain Boublil, Jean-Marc Natel, and Herbert\nKretzmer's musical adaptation of Victor Hugo's novel Les Miserables and tell\nthe story through that perspective. I also want to you to know that I enjoyed\nRussell Crowe's Inspector Javert very much.
\nLes Miserables begins with the Work Song, in which the protagonist,\nprisoner:24601 is confronted by Inspector Javert while doing some\u2026 work. He\ninsists he has a name, Jean Valjean and his identifier in his\nhome village's fictional\ndatabase (that I just retconned) was faverolles:2468. Javert isn't interested\nin his name. It's enough that he has a cross-reference between faverolles:2468\nis equivalent to prisoner:24601. He was only there to inform Jean Valjean that\nhis parole has begun and issues him a passeport jaune (yellow ticket) for the\ncommune of Pontarlier.
I'm sure this passport also had an identifier on it. I'm going to take a bit of\ncreative freedom and say it was pontarlier:25791. It probably also had Jean\nValjean's prisoner number on it so everybody knew he was in the 1800's fictional\nFrench convict database. The fictional 1800's French took maintaining\ncross-references very seriously.
Jean Valjean never made it to Pontarlier. Instead, he broke his parole, forged\nsome new documents, and went to Montreuil-sur-Mer under the new name of Monsieur\nMadeleine. It's probably the case that his identifier for the Montreuil-sur-Mer\ncity database was montreuil-sur-mer:1357, or something like this (more\nretcons!). It must have been a good fake, because even the king of France\nrecognized him (note: this plot point did not appear in the film).
Javert figured out Jean Valjean broke his parole basically immediately and set\nout on his quest to find and capture prisoner:24601 once again. Until this\npoint, Javert has access to the prisoner registry and yellow tickets. He knows\nprisoner:24601 is the same as both faverolles:2468 and pontarlier:25791.
The part that will hit close to home for many bioinformaticians is that when\nJavert goes to Montreuil-sur-Mer, he meets Monsieur Madeleine. He is unaware\nthat it is Jean Valjean. There is no cross-reference between faverolles:2468\nand pontarlier:25791 or montreuil-sur-mer:1357. If there were a\ncross-reference in the fictional French 1800's inspector database, Javert could\nhave arrested Jean Valjean on sight. Instead, Javert had to the hard work of\ncurating cross-references himself and finding out who was the same in the\nmontreuil-sur-mer database as prisoner:24601. Admittedly, he probably would\nhave called this inspecting.
The next part that will hit even closer to home for many bioinformaticians is\nthat after his inspecting, Javert actually identified the wrong guy! This lead\nto one of the my favorite songs in musical theater ever\n(Who Am I?), where Monsieur\nMadeleine (montreuil-sur-mer:1357, also actually Jean Valjean, but Javert\ndidn't yet realize this) admits that he is actually prisoner:24601. In this\nextended metaphor of a blog post, Jean Valjean's confession in \"Who Am I\" is\neffectively the same as a database providing its own cross-references to other\ndatabase. Would be nice if everyone did this, and did it well, huh?
You should know that Javert is a powerful cross-reference reasoning machine. He\nalready knew faverolles:2468 was the same as prisoner:24601. Now he knew\nthat montreuil-sur-mer:1357 was the same as prisoner:24601. This way, he\ncould infer that montreuil-sur-mer:1357 (Monsieur Madeleine) is actually\nfaverolles:2468 (Jean Valjean). One of the nice properties of cross-references\nis that they're transitive through any number of connections. We'll take\nadvantage of this fact later. You'll also have to excuse the fact that\nthroughout this post, I'm operating under the assumption that \"cross-references\"\nand \"equivalences\" are the same thing. That's not always true, and sometimes it\ncan even get you in trouble. For example, provenance can be a cross-reference,\ndisease-gene associations are considererd as cross-references in MONDO (I\nthink), and OBO even gives specific semantics for when you should consider this\nassumption valid. We'll just have to live with it for now.
Javert might have got lucky that Jean Valjean revealed himself once, but the\nshow must go on! Jean Valjean had many more songs to sing and thus had to escape\nfrom Montreuil-sur-Mer to Paris. This meant that Javert has to find another\nmapping to Jean Valjean's new paris identifier. And we already know that the\nFrench 1800's inspector database of cross-references was not being maintained.\nExhausting!
In the bioinformatics community, we have a very similar problem to Inspector\nJavert. There are lots of databases that are talking about the same things, but\nonly a few of them provide mappings between each other. This means that we\neither have to curate our own cross-references, do our best to infer new\ncross-references based on ones we already have, or throw our hands in the air.
\nLuckily, we have a few standardized resources to fall back on. In addition to\nstandardizing the storage of identifier/name pairs, the OBO format standardizes\nthe way cross-references are stored and the OBO Foundry already contains quite a\nfew cross-references imported from the ontologies that it covers.
\nOne of the most difficult entity types to map from database to database are\nphenotypes because of the variety of language used to describe each, the\ndifferences in semantics of how each is defined, and the sheer number of\ndatabases. Unfortunately, some of the most popular like MeSH and to an extent,\nUMLS, NCIT, SNOMED-CT, and ICD (seemingly the culprits are mostly American!?)\nprovide very little accessible information. Some are even paid, so the ony\ncross-references that exist are externally curated ones from other laudable\nsources like HP, DOID, and EFO. In fact, dealing with phenotypes is such a pain,\nthat there is a project called the\nMonarch Initiative that has a huge staff\ntrying to solve exactly this problem and publish the results through the\nMonarch Disease Ontology (MONDO).\nNormally, I would reference\nthis XKCD comic about making new standards when hearing\nabout something like this. But these are dire times, and one of my opinions is\nthat you should always trust curators who love what they do.
\nThere are also lots of cross-references available from databases that don't\nmaintain their nomenclature as an ontology. One example is\nBioGRID,\nwhich assigns proteins internal accession numbers, but almost all of them\ncross-reference out to Entrez Gene (I counted less than 15 that didn't, and 3 of\nthem are COVID-related, so cut them some slack). As an aside, I don't really\nunderstand why BioGRID would go through the effort of maintaining their own\naccession numbers. In the literal handful of cases where they can't reference\nEntrez Gene, I think it would be better to email the maintainers and work with\nthem to make improvements.
\nIt's also worth noting that excellent resources like HGNC, MGI, RGD, SGD,\nEnsembl, UniProt, and others in the genome (and gene product) nomenclature do a\nstellar job at maintaining cross-references. So to all of the curators and\nmaintainers who work there, I would like to sincerely thank you.
\nThere are also community-curated cross-references sources. One of the notable\nones is from Harvard Medical School, that's mapping MeSH identifiers to gene\nidentifiers in the\nGilda GitHub repository.\nI think this is really a good time to point out that MeSH contains a bit of\neverything, is ubiquitous throughout the bioinformatics community, and in my\nopnion is is doing a huge disservice by not providing these kinds of mappings\nitself. Or, alternatively, it is, and both the Harvard guys and I have never\nfound it. It's not impossible, but we're all very motivated, so I think we would\nhave found if it did. If any MeSH maintainers are reading this and want help\nmaking this happen, I would be elated to donate my time to you to help solve\nthis problem.
\nWith all these data source in mind, I built an extensible pipeline in\nPyOBO\nfor extracting cross-references from entries in OBO Foundry and other\ncross-reference sources. Throughout the process, I realized that these sources\nhave an incredible variety in how they name prefixes and how the OBO format\nitself has been (ab)used. I made lots of improvements, wrote extensible code\nthat allowed the specification of new rules through external files (and thus\nless code writing in the future), and did lots more curation. I won't get into\nthe technical part of that here, since you can read the code (if you dare).
\nAfter all that this coding, I wrote a script (just run obo javerts-xrefs) that\ntakes all available cross-references, normalizes their namespaces, normalizes\ntheir identifiers, and dumps them in a big 'ol TSV file. 5 columns - source\nnamespace, source identifier, target namespace, target identifier, and\nprovenance (ontology name or URL). No nonsense. Get it at\n.\nI'll make updates periodically as I add new sources.
Once you have a database of cross-references, you have actually built an\nundirected graph. Equivalences go both ways, and they are transitive. This means\nthat every connected component in an equivalence graph represents a set of\nentities that are mutually equivalent. In other words, if a path exists between\ntwo nodes in an equivalent graph, then they are equivalent.
\nEven better, you don't have to materialize all of the possible inferred\nequivalences when you have an equivalence graph because identifying all of the\nnodes in a connected component can be done in linear time with respect to the\nsize of the connected component, which is usually pretty small, by using a\nbreadth-first or depth-first search.
\nBased off of that, one application of an equivalence graph is to identify all of\nthe nodes that are equivalent to a given node. You can also get a little tricker\nand identify the paths through which the traversal must go if you want to\nestablish an equivalency. You could even go further and weight edges based on\nhow much you trust the source from which they came to identify how much you\nshould believe in a mapping. For example, if you have a percent confidence in\neach mapping being right, then the confidence in the whole pathway would be the\nproduct of the confidences.
\nThe actual problem I set out to solve was given a set of entities, remap all of\nthem based on a prioritized list. For example, I might have a set of entities\nthat contains HGNC genes, Entrez Genes, and OMIM genes. If my favorite\nnomenclature consortium is Entrez, my second favorite is HGNC, and my third\nfavorite is OMIM and I have an equivalence database, I might want to remap all\nof my identifiers. This is very important during the curation of mechanistic\nbiology (such as with BEL), since curators will likely use all sorts of\nidentifiers with no clear guidelines or rules. This means that the same entity\nmight appear twice with different identifiers in the same curated data!
\nGiven a priority list, you can even transform an equivalence graph into a\ndirected graph where each identifier has a single out edge pointing towards the\nidentifier that is the best mapping. Then, each connected component would become\na star graph. There's actually a better data structure for this, since each\nentity points to exactly one thing - a mapping. This is a more efficient data\nstructure for storage, and if your graph is implemented as an adjacency\ndictionary (becuase you're using networkx, right?), then you basically already\nhave this.
I've provided an implementation for all of these in PyOBO. They can be run as a\nweb API with python -m pyobo.apps.mapper. There's a keyword argument to allow\nyou to load the TSV from Inspector Javert's Xref Database directly, or if you're\nfeeling lucky, to regenerate it yourself. Below I will give a few examples of\nhow to use it. Later, I would also like to host this service for anyone to use.
pip install git+https://github.com/pyobo/pyobo.gitpython -m pyobo.apps.mapper -x inspector_javerts_xrefs.tsv.gzimport requests\n\n# Get all entities mapped to MAPT, including through chains of xrefs\nsuccessful_request = requests.get('http://localhost:5000/mappings/hgnc:6893').json()\n\"\"\"\n{\n \"orphanet:123144\": [\n {\n \"provenance\": \"hgnc\",\n \"source\": \"hgnc:6893\",\n \"target\": \"orphanet:123144\"\n }\n ],\n \"pr:P10636\": [\n {\n \"provenance\": \"hgnc\",\n \"source\": \"hgnc:6893\",\n \"target\": \"uniprot:P10636\"\n },\n {\n \"provenance\": \"pr\",\n \"source\": \"uniprot:P10636\",\n \"target\": \"pr:P10636\"\n }\n ],\n ...\n}\n\"\"\"\n\n# Keep in mind this isn't a validation service\nunsuccessful_request = requests.get('http://localhost:5000/mappings/hgnc:0000').json()\n# {\"message\": \"could not find curie\", \"query\": {\"curie\": \"hgnc:0000\"}, \"success\": False}\n\n# Get all paths mapping MAPT in HGNC to Ensembl. Returns a list of paths (which are lists of xrefs)\npath_request = requests.get('http://localhost:5000/mappings/hgnc:6893/ensembl:ENSG00000186868').json()\n\"\"\"\n[\n [\n {\n \"provenance\": \"hgnc\",\n \"source\": \"hgnc:6893\",\n \"target\": \"ensembl:ENSG00000186868\"\n }\n ]\n]\n\"\"\"\n\n# Get the priority identifier for MAPT identified by Ensembl\nprioritize_request = requests.get('http://localhost:5000/prioritize/cosmic:MAPT').json()\n# {\"found\": True, \"query\": \"cosmic:MAPT\", \"result\": \"hgnc:6893\"}\n\n# What happens when a CURIE can't be found for prioritization\nunsuccessful_prioritize_request = requests.get('http://localhost:5000/prioritize/cosmic:NOPE').json()\n# {\"found\": False, \"query\": \"cosmic:NOPE\"}\nI'd like to give a big thanks to my high school music teacher, Ken Tedeschi, for\nhelping me (and basically everyone else) fall in love with Les Mis in high\nschool. Writing about my work was so much more fun in extended metaphor. I would\nalso like to thank Hugh Jackman. You know, for being Hugh Jackman.
\nI have some random afterthoughts that I think might be worth including, that I'm\nadding after originally posting this.
\nYou might be wondering why I didn't get into a discussion about the\nOntology Mapping Service (OXO)\nfrom the EBI. It looks to me like this project has been abandoned. Even if not,\nit's API has most of the same issues that I described in a previous\npost.
\nI'm also aware of BridgeDB, from which I think I\nwill be able to take inspiration to include more xrefs later. However, I think\nthey're limited in scope, and PyOBO is more about standardizing data so nobody\nhas to figure out databases\u2026 again and again and again.
\nOne glaring omission from this work is WikiData mappings. I have a plan to\ninclude curated information in the PyOBO metaregistry that links databases to\ntheir WikiData properties. That will allow me to build an automated framework\nfor downloading these mappings, given the curation of the properties.
","doi":"https://doi.org/10.59350/r3qzt-z0d08","guid":"https://cthoyt.com/2020/04/19/inspector-javerts-xref-database","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1587254400,"rid":"vqbr3-bcb03","summary":"On top the issue of resolving identifiers to their names, the bioinformatics community has a hard time figuring out when two identifiers from different databases are equivalent. You know who else has the same problem? Inspector Javert. Get ready for a Les Miserables-themed post on how to address this long-standing problem.","tags":["Mappings"],"title":"Inspector Javert's Xref Database","updated_at":1781539905,"url":"https://cthoyt.com/2020/04/19/inspector-javerts-xref-database.html","version":"v1"},{"authors":[{"contributor_roles":[],"family":"Tapley Hoyt","given":"Charles","url":"https://orcid.org/0000-0003-4423-4370"}],"blog":{"authors":null,"community_id":"da4ef2af-5fad-46b2-8195-d77db0141ad6","created":1716422400,"current_feed_url":null,"description":"Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.","favicon":"https://rogue-scholar.org/api/communities/da4ef2af-5fad-46b2-8195-d77db0141ad6/logo","feed_format":"application/atom+xml","feed_url":"https://cthoyt.com/feed.xml","filter":null,"generator":"Jekyll","home_page_url":"https://cthoyt.com/","issn":null,"language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.59350","relative_url":null,"secure":true,"slug":"cthoyt","status":"active","subfield":"1312","title":"Biopragmatics","updated":1781539024,"use_api":null},"blog_name":"Biopragmatics","blog_slug":"cthoyt","content_html":"As scientists, we place huge importance on the communication of our results. We\nspend lots of time on editing, revising, and formatting so people can understand\nwhat we did. We also write a lot of code, so why aren't we investing the same\namount of love? Enter, flake8.
\nIt's incredibly important that we write following community standards so when\nother people read our work, they don't have to think about how it's organized.\nFor scientific prose, this usually means the IMRD\n(introduction-methods-results-discussion) format. In Python, my current favorite\nprogramming language for science, this means using a standardized number of\nspaces for indents (4), using triple-double quotes for docstrings in the\nbeginning of each module, class, and function, and lots more.
\nIt's pretty intimidating to figure out style. For english prose, Strunk and\nWhite wrote\nThe Elements of Style.\nFor Python, Guido van Rossum wrote\nPEP-8 and Raymond Hettinger\npresented Beyond PEP-8. Even with\nthese resources, it's still hard to learn which are rules and which\nare more like guidelines.
\nThis post is a short explanation of how I use flake8 to keep a consistent\nstyle in the code in my Python projects. There's a similar command line tool for\nfixing the style in R projects that's already built into most operating\nsystems - rm -rf *, but I won't get more into that here.
It's pretty easy to get up and running with flake8 - just run\npip install flake8 then use it from the shell on a python file like\nflake8 my_file.py or flake8 my_directory/. Then, it outputs a list of\nproblems that need to be fixed on a line-by-line basis in your code.
You can also install plugins with pip like that extend the kinds of things it\nchecks. A few that I install are:
id.flake8 output (explanation how to set up is below)print(). The literal only exception is when using print to get\ntext into a file with print(..., file=...)% or .format() formatting.\nException being for logging.In each of my repositories, I put all of the information on how to install\nflake8 and its plugins then run them in a tox configuration under the\n[testenv:flake8] header so they can easily reproducibly run with\ntox -e flake8. An example of part of one of my tox.ini files (which always\nlives in the root of the repository) is below:
[testenv:flake8]\nskip_install = true\ndeps =\n flake8\n flake8-bandit\n flake8-builtins\n flake8-bugbear\n flake8-colors\n flake8-commas\n flake8-comprehensions\n flake8-docstrings\n flake8-import-order\n flake8-print\n flake8-use-fstring\n pep8-naming\n pydocstyle\ncommands =\n flake8 src/pybel/ tests/ setup.py\ndescription = Run the flake8 tool with several plugins (bandit, docstrings, import order, pep8 naming).\nAnother configuration file you can set up in the root of the repository is\n.flake8. Unfortunately, the Python configuration file reader doesn't allow\nsome of the crazy characters that I want to use for the colors so this can't be\nincluded in setup.cfg or tox.ini like most of your other configuration.
[flake8]\nignore =\n # line break before binary operator\n W503\nexclude =\n .tox,\n .git,\n __pycache__,\n docs/source/conf.py,\n build,\n dist,\n tests/fixtures/*,\n *.pyc,\n *.egg-info,\n .cache,\n .eggs\nmax-line-length = 120\nimport-order-style = pycharm\napplication-import-names =\n pybel\n bel_resources\n tests\nformat = ${cyan}%(path)s${reset}:${yellow_bold}%(row)d${reset}:${green_bold}%(col)d${reset}: ${red_bold}%(code)s${reset} %(text)s\nFirst thing you'll notice is the ignore list. This isn't here to turn flake8\noff because you're feeling lazy. If somebody includes a change in this list in\ntheir PR, you have to explain to them that compliance is not optional, then help\nthem work through the problem that they obviously gave up on solving. It's\nactually there for you, as the project maintainer, to enumerate the flake8\nrules that you don't agree with. For example, I totally disagree with the W503\nline break before operator rule. I want to write long conditionals with and\nstatements on the first line, like this:
if (\n condition_1\n and condition_2\n and condition_3\n):\n print('all true')\nOne of the benefits of this style is you can add more lines with only single\nline diffs. The other is that the reader always sees the operation that goes\nwith each line. Same could be done with arithmatic that could incorporate not\nonly + but also -.
Next is the exclude block. Just copy/paste this each time, since it has lots\nof garbage you don't want flake8 to bother with. One of the checkers in\nflake8 is for function \"cyclomatic\" complexity. You can make the maximum\nnumber higher with max-complexity. Normally, you want this to be enforced, but\nsometimes there's no way around a complex function. For this, you can add a code\ncomment noqa followed by the error code like # noqa:W123. Again, adding tags\nto ignore bad style just to pass flake8 is against the point.
The max-line-length is a very contentious setting. I think 120 is fine. Some\npeople think 78, 79, or 80 is best because of the standard sizes of old computer\nscreens or punch cards\u2026 When I get older and I can't read my computer screen,\nI'll probably make the text bigger and change my mind about this. If you find\nyourself breaking up lines in a totally non-sensical, unstyled way, then you're\nconforming too tightly to the rules. Sorry about the mixed messages!
import-order-style = pycharm\napplication-import-names =\n pybel\n bel_resources\n tests\nI copied this again because this part is really important. You have to tell\nflake8 what rules you use for import order. I use the pycharm rules, which\ngroup python builtin packages, then 3rd party packages, then my packages. The\napplication-import-names is a place to list what are your packages.
Last is the format entry, which gives the nice colorful output. Copy paste\nthis! I borrowed mine from Scott Colby.
After all of that, I set up Travis CI to run tox every time code is pushed to\nthe repository. If you're working in a team, you probably do something like the\nfork/pull request or branch/pull request workflow on GitHub to support doing\ncode review before merging new code. The best part is that there's a big box on\neach pull request that checks if flake8 passed (among other tests), which\nmeans that there were no errors detected.
I encourage my teammates to make pull requests as soon as they start working on\ncode. GitHub even has a \"draft pull request\" mode now. However, before asking\nanyone to review your code, it has to pass flake8. And obviously, no code that\nisn't passing flake8 can be merged.
This is a very painful process to get people used to. I've done it with many\ngroups of people and always got pushback. However, everyone who has gone through\nthe process with me has come out the other side happy that they did it. It's\nimportant that when you start enforcing coding rules on other people that you\nare a resource for them - when somebody is frustrated by a flake8 error code\nthey have never seen, they will likely forget how to use Google. They will\nprobably ask you for help. You have to resist the urge to send\nlmgtfy links to them and be patient. Because eventually,\nthey will do it on their own, and spread the gospel of flake8.
While a good arsenal of flake8 plugins provides a solid foundation, it's not\nall that needs to be done to make your code readable and look good. Just like\nwith reading and speaking, the best way to develop a sense of style is by\nreading lots of code (with the caveat that reading poorly written code\nprobably won't teach you much). Within the rules imposed by flake8, there is\nlots of space for style. If you watch lectures from David Beazley, you'll notice\na very different style from Raymond Hettinger, and also from me.
Now that you've made it to the end of this short guide, I wish you the best of\nluck on developing your own style!
\nAre you working with people who are particularly unsusceptible to Travis CI\nemails or checking the big red box on pull requests? You could try getting them\nset up with pre-commit hooks, which run the style\nchecks locally any time they try and push (even if it's to a branch) and it will\ngive them the message in the console.
\nIs style not your thing at all / you're not ready to let go of your identity as\na Java/Perl developer? Maybe consider Black,\nwhich actually re-writes your code in a deterministic style. I don't live by it,\nbut it's a great tool to run on a code base that's never been loved before going\nback and stylizing it.
","doi":"https://doi.org/10.59350/cfxtw-0ma23","guid":"https://cthoyt.com/2020/04/25/how-to-code-with-me-flake8","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1587772800,"rid":"4y55t-c1k96","summary":"As scientists, we place huge importance on the communication of our results. We spend lots of time on editing, revising, and formatting so people can understand what we did. We also write a lot of code, so why aren't we investing the same amount of love? Enter, flake8.","tags":["Code With Me"],"title":"How to Code with Me - Flake8 Hell","updated_at":1781539903,"url":"https://cthoyt.com/2020/04/25/how-to-code-with-me-flake8.html","version":"v1"}],"out_of":50584,"page":1,"per_page":10,"total-results":50584}