![\"\"](\"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2026/08/Bildschirmfoto-2026-08-20-um-11.00.02.png\")
I ran a TD-DFT calculation on a platinum complex, fed the log file to ELOTE, and got only half an answer. The molecular orbital composition table printed just fine, every orbital with its atomic percentages. But the excited-state analysis, the whole point of the exercise, came back empty. The CSV had a header and nothing underneath. No transitions, no MLCT/LMCT labels, no summary table. Nothing. The calculation was perfectly healthy. So what was going on?
\nIf you've only ever run closed-shell TD-DFT, you're used to seeing excited states written like this:
\n Excited State 1: Singlet-A 2.14 eV 580 nm f=0.0198\n 150 ->151 0.51251\n 150 ->152 -0.30648\n\nOrbitals are labeled by clean integers, a right pointing arrow ->, and the CI coefficient. That's what ELOTE was built to read. But my platinum system was an open-shell calculation and therefore an unrestricted one, with unpaired electrons. Gaussian writes those excited states differently:
Excited State 1: 5.013-A 0.2171 eV 5709 nm f=0.0000 =6.033\n 48B -> 50B 0.67064\n 43B <- 50B -0.17577\n \nIn these calculations the orbital number now specifies if it belongs to the alpha (A) or beta (B) manifold, in other words, it now bears a spin label: 48B, 50B. Second, there's a backwards arrow, <-, mixed in with the forward ones, indicating de-excitations (vide infra)
Both of those broke the parser silently. ELOTE couldn't handle the spin label of the orbital (it expected just a number), so it quietly recorded zero transitions for every single state. Hence the empty tables. No error, no complaint, nothing, which is the most confusing kind of bug. (Sorry about that.)
\nSince orbital 48A isn't the same as 48B, it isn't enough to strip the letter and move on \u2014 ELOTE has to keep the alpha and beta orbitals in separate bookkeeping, label them separately (you'll now see HOMO(B), LUMO(B) and so on), and then decide the metal-versus-ligand character of each transition using the correct spin's orbital. A beta HOMO and an alpha HOMO can have genuinely different atomic compositions, and conflating them would hand you a wrong assignment while looking perfectly confident about it.
With the parser taught to read both manifolds, my platinum spectrum came apart cleanly. The lowest excited state, for instance, turns out to be predominantly a beta HOMO-1 \u2192 LUMO transition with roughly 38% weight, dressed with a handful of smaller ligand-to-metal and intraligand contributions, and everything sums to a tidy 100%. Exactly the kind of table you'd want to drop into a manuscript.
\nIf you're running open-shell systems, you don't have to do anything differently. ELOTE detects the unrestricted output on its own and prints a short note letting you know it's in that mode:
\n\nUnrestricted (open-shell) calculation detected: alpha (A) and beta (B)orbital manifolds are labeled separately.\nThe orbital labels then carry their spin \u2014 HOMO(B), LUMO+2(A) \u2014 so there's never any ambiguity about which manifold a transition belongs to. Closed-shell calculations behave exactly as before; nothing changed for them.
A word of caution that has nothing to do with ELOTE and everything to do with chemistry. My test system had an
The update is live on GitHub, and if you'd found this the hard way like I did, I hope this saved you an afternoon.
\nAs always \u2014 questions and comments welcome below.
\nIn a restricted (closed-shell) calculation, every spatial orbital holds two electrons, one spin-up and one spin-down, and they share the same shape. There is one set of orbitals, and one HOMO, and one LUMO. Simple.
\nIn an unrestricted (open-shell) calculation \u2014 the kind you need whenever you have unpaired electrons \u2014 spin-up (\u03b1) and spin-down (\u03b2) electrons are allowed to occupy different spatial orbitals. So Gaussian keeps two separate sets of orbitals: an alpha manifold and a beta manifold, each with its own numbering, its own HOMO, its own LUMO. The B suffix is simply telling you which manifold a given orbital lives in. 48B is the 48th beta orbital, and it is not the same object as 48A.
The backward arrow, <-, is the other subtlety. It's a de-excitation contribution, a small term that the linear-response formalism includes alongside the main excitations. For the purpose of assigning character, the physically sensible thing is to read it as donor and acceptor swapped, so ELOTE now flips it and treats it consistently.
Cite ELOTE: 10.5281/zenodo.21799957 (Zenodo DOI) \u00b7 Repository: github.com/joaquinbarroso/ELOTE
","doi":"https://doi.org/10.59350/0swjx-kfe02","guid":"http://joaquinbarroso.com/?p=7018","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1785888000,"rid":"z9wae-5fd43","summary":"I ran a TD-DFT calculation on a platinum complex, fed the log file to ELOTE, and got only half an answer. The molecular orbital composition table printed just fine, every orbital with its atomic percentages. But the excited-state analysis, the whole point of the exercise, came back empty.","tags":["Coding","Computational Chemistry","ELOTE","Excited State","Photochemistry"],"title":"ELOTE now works with Open-Shell systems","updated_at":1787221701,"url":"https://joaquinbarroso.com/2026/08/05/elote-now-works-with-open-shell-systems/","version":"v1"}},{"document":{"authors":[{"affiliation":[{"name":"Front Matter"}],"contributor_roles":[],"family":"Fenner","given":"Martin","url":"https://orcid.org/0000-0003-1419-2405"}],"blog":{"authors":[{"name":"Martin Fenner","url":"https://orcid.org/0000-0003-1419-2405"}],"community_id":"15a362ea-8138-42b8-917f-1840a92addf8","created":1672531200,"current_feed_url":null,"description":"The Front Matter Blog covers the intersection of science and technology since 2007.","doi":"https://doi.org/10.53731/front_matter","favicon":"https://rogue-scholar.org/api/communities/15a362ea-8138-42b8-917f-1840a92addf8/logo","feed_format":"application/atom+xml","feed_url":"https://blog.front-matter.de/atom","filter":null,"generator":"Ghost","home_page_url":"https://blog.front-matter.de/","issn":"2749-9952","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.53731","relative_url":null,"secure":true,"slug":"front_matter","status":"active","subfield":"1710","title":"Front Matter","updated":1787218226,"use_api":true},"blog_name":"Front Matter","blog_slug":"front_matter","content_html":"Starting this week the Rogue Scholar science blog archive collects citations of its blog posts using invenio-pidbox, the Python module for the InvenioRDM repository platform that I launched earlier this month, and replacing the integration that used the Crossref cited-by service launched in August 2025.
The reasons for this change are threefold:
invenio-pidbox uses a sqlite database to store scholarly metadata and uses the commonmeta-py Python library and its Rust bindings to query sqlite and integrate with the InvenioRDM repository platform that Rogue Scholar uses. The initial invenio-pidbox versions used the ROR and ORCID regular data dumps (135,710 organizations and 25,048,058 people, respectively) to generate author and organization profiles. With this release I started to also store Crossref and DataCite metadata in sqlite. Initially only the metadata for the blog posts in Rogue Scholar, so only tracking citations by other blog posts archived in Rogue Scholar, but the architecture allows me to scale to all 180 million Crossref DOIs and 120 million DataCite DOIs in their annual data dumps, and the about 2.4 billion references they contain, all in all about 2 TB of data. This sounds like a lot of data, but sqlite can easily handle this, and this is much easier and cheaper than importing all the metadata into Postgres and Opensearch, the database and search index used by InvenioRDM.
invenio-pidbox comes with a celery background worker task to extracts all Rogue Scholar citations out of sqlite and stores them as InvenioRDM metadata so that they can be shown and queried in Rogue Scholar. Blog post citations can be much faster than citations by journal articles, e.g. five days after the publication of the InvenioRDM v14 announcement:
Citations counts in Rogue Scholar are displayed for blogs, other communities, authors and institutions:
![\"\"](\"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2026/08/Bildschirmfoto-2026-08-20-um-11.08.46.png\")
Clicking on the citation count (here 58) opens a query for all cited blog posts. The DataCite blog is also an example for tracking citations of blog posts with DataCite DOIs, something that Rogue Scholar couldn't do until this week.
invenio-pidbox is an invenio module that should work with all InvenioRDM v14 instances, and besides Rogue Scholar they currently all use DataCite DOIs for their content, making this an attractive option for other InvenioRDM instances that have 2 TB disk space to spare \u2013 not S3 object storage as sqlite needs fast disk access.
Institution profiles are an interesting use case. Their main challenge is incomplete metadata, e.g only 137 of currently 683 posts in the Crossref blog have affiliation metadata. Affiliation metadata (using ROR) are more difficult than author metadata (using ORCID), as affiliations have a start and end date, and authors can have multiple affiliations in parallel.
![\"\"](\"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2026/08/Bildschirmfoto-2026-08-20-um-11.20.01.png\")
But with 300 million DOIs for works from Crossref and DataCite stored in sqlite in commonmeta format it becomes much easier to extract that information at scale, not limited by API rate-limits or differences between Crossref and DataCite metadata.
Please reach out via Slack, email, Mastodon, or Bluesky if you have any questions or comments.
Wer schon einmal auf eine Datei mit dem Namen final_neu_wirklichfinal.xlsx gesto\u00dfen ist, kennt das Problem: Daten k\u00f6nnen schnell un\u00fcbersichtlich werden. In der Forschung ist das nicht nur l\u00e4stig, sondern kann dar\u00fcber entscheiden, ob Ergebnisse nachvollziehbar, nachnutzbar und langfristig auffindbar bleiben. Genau hier setzt Forschungsdatenmanagement (FDM) an. Aber wie erkl\u00e4rt man FDM so, dass es f\u00fcr Studierende greifbar wird?
\nMit dieser Frage besch\u00e4ftigte sich eine Gruppe mit acht Studierenden des Bachelorstudiengangs Informationsmanagement an der Hochschule Hannover (HsH) im Sommersemester 2026. Im Rahmen eines Projekts im 6. Semester, betreut durch das Joint Lab Future Libraries & Research Data von HsH und TIB, entwickelten sie Materialien, die FDM aus studentischer Perspektive zug\u00e4nglich machen: einen Selbstlernkurs und modulare Seminareinheiten \u2013 konzipiert von Studierenden f\u00fcr Studierende.
\nAusgangspunkt war die Beobachtung, dass FDM im bisherigen Studienverlauf explizit kaum vorkam. Daher verfolgte das Projekt das Ziel, Materialien zu entwickeln, die niedrigschwellig erkl\u00e4ren, worum es bei FDM geht und warum es schon im Studium n\u00fctzlich ist. Das Projekt war bewusst offen angelegt. Neben der fachlichen Auseinandersetzung mit FDM geh\u00f6rten auch eigenst\u00e4ndige Organisation und Projektmanagement dazu.
\nDabei sollten vorhandene Open Educational Resources (OER) weitergedacht und an eine studentische Zielgruppe angepasst werden. Konkret kn\u00fcpfte die Gruppe dabei an den Kurs \"Nachhaltige Wissenschaft mit Forschungsdatenmanagement \u2013 Eine Einf\u00fchrung f\u00fcr Betreuende von Qualifizierungsarbeiten\" an, der bereits im TIB-Blog vorgestellt wurde.
\nBesonders interessant war der Perspektivwechsel: Die Studierenden entwickelten Materialien f\u00fcr andere Studierende und pr\u00fcften dabei immer wieder, was wirklich verst\u00e4ndlich, brauchbar und motivierend ist. Was braucht man als Einstieg? Welche Begriffe m\u00fcssen erkl\u00e4rt werden? Wo helfen Beispiele mehr als Definitionen? Gleichzeitig wurde FDM Teil des Projektalltags. Einheitliche Dateibenennungen, Dokumentation, Versionierung, regelm\u00e4\u00dfige Backups und README-Dateien waren nicht nur Kursthemen, sondern praktische Werkzeuge in der Zusammenarbeit. So wurde aus einem abstrakten Thema schnell eine konkrete Projekterfahrung.
\nF\u00fcr die Erarbeitung der OER teilte sich die Gruppe in zwei Teams auf, die an den unterschiedlichen Formaten arbeiteten und ihre Materialien jeweils iterativ auf Basis von Feedback aus Lehre und Praxis weiterentwickelten.
\nEin Teilteam entwickelte einen Onlinekurs auf Basis von LiaScript. Ziel war ein kompakter Selbstlernkurs, der FDM-Grundlagen in rund 120 Minuten im Selbststudium vermittelt und Studierenden einen ersten, gut zug\u00e4nglichen \u00dcberblick bietet. Daf\u00fcr arbeitete sich die Gruppe in die Funktionsweise und den Syntax von LiaScript ein. Eine Herausforderung bestand darin, dass kollaboratives Arbeiten in diesem Format nur eingeschr\u00e4nkt m\u00f6glich war und der Zugang \u00fcber GitHub zus\u00e4tzliche technische H\u00fcrden mit sich brachte.
\nInhaltlich ging es darum, bestehende OER mit einer \"Studierendenbrille\" zu \u00fcberarbeiten: Was ist f\u00fcr die Zielgruppe wirklich relevant? Welche Beispiele machen FDM greifbar? Und wie l\u00e4sst sich ein Thema, das schnell trocken wirken kann, in ein \u00fcbersichtliches Selbstlernformat bringen?
\nDas Skript zum Selbstlernkurs ist auf GRO.data ver\u00f6ffentlicht: https://doi.org/10.25625/HLSNR1. Hier geht es direkt zum Selbstlernkurs.
\nDas zweite Teilteam entwickelte drei Seminareinheiten, die Lehrende flexibel in bestehende Veranstaltungen einbauen k\u00f6nnen. Erg\u00e4nzt wurden die Folien durch \u00dcbungen, Arbeitsbl\u00e4tter und Rahmenhinweise. Der modulare Ansatz war dabei zentral: Die Materialien sollen nicht als starre Pr\u00e4sentation funktionieren, sondern als Bausteine, aus denen Lehrende je nach Veranstaltung, Zeitrahmen und Vorwissen der Studierenden ausw\u00e4hlen k\u00f6nnen.
\nAuch hier spielte Feedback eine gro\u00dfe Rolle. Mehrere Versionen wurden \u00fcberarbeitet, angepasst und gesch\u00e4rft. Eine besondere Herausforderung war die Frage, welches Vorwissen k\u00fcnftige Studierende tats\u00e4chlich mitbringen und wie viel Kontext die Materialien deshalb selbst liefern m\u00fcssen.
\nDie Seminarmaterialien sind ebenfalls auf GRO.data ver\u00f6ffentlicht: https://doi.org/10.25625/6105WI.
\nBei der Ver\u00f6ffentlichung der Arbeitsergebnisse sammelte die Gruppe praktische Erfahrungen mit GRO.data, Zenodo und twillo. Das FDM wurde damit ganz konkret: Welche Metadaten werden ben\u00f6tigt? Welche Felder sind verst\u00e4ndlich? Wo fehlen Hinweise? Und wie benutzerfreundlich sind Repositorien aus Sicht von Studierenden?
\nDie Erfahrungen fielen unterschiedlich aus. Zenodo wurde als grunds\u00e4tzlich einfach und verst\u00e4ndlich wahrgenommen, auch wenn an einzelnen Stellen genauere Hilfetexte hilfreich gewesen w\u00e4ren. GRO.data wirkte insgesamt selbsterkl\u00e4rend, stellte die Gruppe aber bei einzelnen Funktionen und Bezeichnungen vor Fragen.
\nGerade diese Erfahrungen machten sichtbar, dass Forschungsdatenmanagement nicht nur aus Prinzipien und Empfehlungen besteht, sondern aus konkreten Entscheidungen, Werkzeugen und manchmal auch technischen H\u00fcrden.
\nDas Projektteam pr\u00e4sentierte die Arbeitsergebnisse auf der BiblioCon 2026 in Berlin in Form eines Posters und einer Pr\u00e4sentation f\u00fcr das Freiraum-Format. Damit wurden die Projektergebnisse nicht nur im Studienkontext genutzt, sondern auch in eine gr\u00f6\u00dfere Fach\u00f6ffentlichkeit getragen.
\n![\"\"](\"https://blog.tib.eu/wp-content/uploads/2026/08/280818_jl_semesterprojekt_bibliocon1-1024x768.jpg\")
F\u00fcr die Studierenden bedeutete das, die eigene Projektarbeit noch einmal anders zu reflektieren: Was ist das Wesentliche? Welche Erfahrungen sind \u00fcbertragbar? Und wie l\u00e4sst sich ein studentisches Projekt so pr\u00e4sentieren, dass es auch f\u00fcr andere Einrichtungen und Studieng\u00e4nge interessant wird? \u00dcber die Plattform der Tagung wurden die Beitr\u00e4ge anschlie\u00dfend ebenfalls publiziert (Freiraum-Folien: nbn-resolving.org/urn:nbn:de:0290-opus4-205328, Poster: nbn-resolving.org/urn:nbn:de:0290-opus4-205129).
\nBei der BiblioCon boten sich weiterhin interessante Ankn\u00fcpfungspunkte zu \u00e4hnlichen Ans\u00e4tzen in Hochschulen, wie etwa zum OER-Kurs \"Data Skills“ der Uni Rostock, der Forschende und Studierende seit Ende 2025 mit einem modular aufgebauten Selbstlernangebot auf ILIAS durch den gesamten Forschungsdatenzyklus f\u00fchrt.
\nAuch das Verbundprojekt \"Einstein Your Data“ der Universit\u00e4ten Vechta und Oldenburg, gef\u00f6rdert im Rahmen der Landesinitiative FDM Niedersachsen, verfolgt ein \u00e4hnliches Ziel: Es st\u00e4rkt die Data Literacy von Studierenden durch forschendes Lernen, bei dem der gesamte Forschungsprozess von der Planung \u00fcber die Datenerhebung bis zur Archivierung selbst durchlaufen wird. Ein besonderer Fokus liegt dabei auf den Umgang mit sensiblen Daten. Solche Projekte zeigen: Der Bedarf an praxisnaher Vermittlung von Datenkompetenz ist an vielen Standorten pr\u00e4sent und der Austausch \u00fcber Formate und Erfahrungen lohnt sich.
\nR\u00fcckblickend beschreibt die Projektgruppe das Semester als fachlich und organisatorisch lehrreich. Besonders pr\u00e4gend war die Erfahrung, dass FDM nicht erst bei gro\u00dfen Forschungsprojekten beginnt. Neben Herausforderungen, etwa im Umgang mit technischen Schwierigkeiten beim genutzten Cloud-Dienst, war vor allem die Projektarbeit selbst pr\u00e4gend: Wie im FDM selbst erwiesen sich auch im Projektteam eine sorgf\u00e4ltige Dokumentation und klare Kommunikation als zentrale Erfolgsfaktoren.
\nDamit liefert das Projekt mehr als nur die Materialien f\u00fcr k\u00fcnftige Lehrveranstaltungen: Es zeigt, wie FDM-Kompetenzen im Studium praktisch erfahrbar werden k\u00f6nnen \u2013 nicht als zus\u00e4tzliche Pflicht, sondern als Bestandteil guter wissenschaftlicher und kollaborativer Arbeit.
\nAlle Materialien stehen offen lizenziert zur Nachnutzung bereit und laden dazu ein, Forschungsdatenmanagement fr\u00fch im Studium aufzugreifen und an die eigene Lehre anzupassen.
\nJoint Lab Future Libraries & Research Data \u2013 futurelibraries@tib.eu
\nFranziska Altemeier, TIB \u2013 Leibniz-Informationszentrum Technik und Naturwissenschaften und Universit\u00e4tsbibliothek: franziska.altemeier@tib.eu
\nProf. Dr. Ina Bl\u00fcmel, TIB \u2013 Leibniz-Informationszentrum Technik und Naturwissenschaften und Universit\u00e4tsbibliothek & Hochschule Hannover, ina.bluemel@tib.eu\u00a0
\nDr. Nina D\u00fcvel, TIB \u2013 Leibniz-Informationszentrum Technik und Naturwissenschaften und Universit\u00e4tsbibliothek & Hochschule Hannover: nina.duevel@tib.eu
\nI have finally opened a Posts section on my website! Every post should now automatically get a DOI.
This is something I have wanted to do for a long time, largely inspired by the tireless and consistent example set by Egon Willighagen (Willighagen 2024b, 2024a, 2025).
\nIt was today's post of (Fenner 2025) that finally motivated me to look into it again. That led me down a productive rabbit hole to set up Rogue Scholar: first landing on (Csefalvay 2023)'s excellent guide, and then (Fruehwald 2025)'s clear write-up, both of which made the process of integrating Rogue Scholar into a Quarto-based site surprisingly smooth.
\nAll the changes are documented in the following commit:
\nhttps://github.com/Adafede/adafede.github.io/commit/bc2dfe6f
\nIf you care about attribution, long-term archiving, DOIs and metadata, I highly recommend looking into Rogue Scholar.
\nEdit (1): I realized that integrating CiTO could be a significant enhancement. With some effort (and thanks again to Egon), I managed to implement a working solution for the HTML and PDF outputs, see (Willighagen 2023). However, the solution for the XML feed still feels suboptimal.
\nEdit (2): After some help from Egon and Martin, I could improve my feed with correct CiTO annotations and their cool custom json feed, see: https://adafede.github.io/posts.json!
\n@online{rutz2025,\n author = {{Adriano Rutz}},\n title = {Open {Science} {Upgrade:} {Adding} {Blog} {Posts} to My\n {Website} and {Linking} to {Rogue} {Scholar}},\n date = {2025-08-04},\n url = {https://adafede.github.io/posts/2025-08-04_rogue_scholar.html},\n doi = {10.59350/yckwd-9vm79},\n langid = {en}\n}\nFive years ago, LOTUS (Rutz et al. 2022) started as a small attempt to cultivate the flow of chemical knowledge, in the same way we study how metabolites flow through living systems, rather than to build yet another database.
\nThe idea was simple. Natural products data should be open, structured, reusable, and belong to everyone.
\nLike many community-driven efforts, LOTUS had bursts of activity, long pauses, and years of invisible maintenance. From the outside, silence can look like disappearance. From the inside, it usually means people are still doing the work; slowly, carefully, and often without announcements.
\nThe early focus of LOTUS was necessarily inward: assembling data, cleaning records, releasing versions. We built a website, curated entries, and archived releases.
\nBut we slowly realized something uncomfortable: data stored in a repository, even a good one, does not automatically live.
\nArchiving on Zenodo (https://zenodo.org/communities/the-lotus-initiative) was the right thing to do, but archived data is mostly silent data. It waits to be discovered, and versioning remains labor-intensive.
\nWhat we really needed were entry points where people already were.
\nWikipedia , Wikidata , and Scholia building on top of it are not dissemination platforms in the classical sense. They are circulatory systems. They persist because communities maintain them.
\nThe Scholia Chemistry preprint (Willighagen et al. 2025) co-authored with Egon , Denise , Daniel , and Finn belongs to this continuity. It does not introduce a new platform. It offers a lens for communities to see what they already collectively know.
\n\n\nKnowledge needs channels, not just reservoirs.
\n
One important step was enabling Wikipedia articles and infoboxes to directly access found in taxon (P703) relationships from Wikidata in an efficient way.
\nThis sounds like a small technical detail. It is not.
\nThe idea had circulated quietly for years, in hallway conversations, chats, and conferences. I also mentioned it during the WikiCite 2025 conference last August, but it took time before conditions were right. Infrastructure work rarely happens on schedule. It almost never happens on stage. It happens in version histories, talk pages, and tiny edits that fix one Lua bug, enabling thousands of articles to improve forever.
\nOnce data is in Wikidata, it can flow into thousands of chemical articles, in dozens of languages, without duplication or translation overhead. It becomes visible to non-experts, students, and readers who will never see a database interface. This is how open data becomes public knowledge.
\nThese modules are now available on multiple Wikipedias:
\nIf you speak another language, feel free to copy them and increase their use.
\nInitially, I tried to reuse existing modules, and quickly learned why module reuse across Wikipedias is famously difficult. Each wiki evolves its own ecosystem of dependencies, conventions, and technical debt. So the modules were written fully contained, independent of language-specific infrastructure. Only lines that need to be changed are the language-specific translations at the top of the module, trying to follow 18n.
\nThese modules are not a new website or a new interface. They are simply better plumbing.
\n![](\"https://adafede.github.io/images/screenshots/screenshot_p703_module.png\"/)
By default, only 5 organisms are shown, keeping the text clean and readable. But the magic is in how the module handles the rest: each taxon links to its Wikipedia article, and if no article exists in the current language, the module gracefully redirects to an equivalent page in another wiki. A beautiful example of this is war for Quassia africana.
\n\nNot everyone wants to learn SPARQL.
\n
I have heard this sentiment countless times. Regardless of personal preferences, if we want the data to truly live, it must reach as many people as possible.
\nSmall tools like the LOTUS Wikidata Explorer help lower that barrier. It is imperfect. It is still growing. But it already allows chemists, curators, and students to access and export data in formats they can actually use.
\nThe principle is simple: knowledge only flows when it reaches people. The data must be seen, explored, and reused. Only then does it fulfill its purpose.
\n![](\"https://adafede.github.io/images/screenshots/screenshot_lotus_wikidata_explorer.png\"/)
On a more technical note, the LOTUS Wikidata Explorer leverages the powerful IDSM endpoint (Galgonek and Vondr\u00e1\u0161ek 2021), which allows for chemical similarity searches thanks to Sachem (Kratochv\u00edl et al. 2018). Its speed for large-scale queries could never have been reached without QLever (Bast and Buchhold 2017), and chemical depictions come from CDK Depict.
\nIt returns structured metadata for traceability and reproducibility, together with hashes that uniquely identify the query and its results. It can be queried programmatically via simple API calls, for example ?taxon=Gentianaceae or ?smiles=c1ccccc1&formula_filter=true&f_state=required. It works directly in the browser without requiring heavy dependencies, almost everything works out of the box.
Alternatively, users can take advantage of a local version, for example to extract a small, personal knowledge graph in TTL format, or to export all or selected LOTUS data in more chemistry-friendly formats such as JSON or CSV, ready for analysis, visualization, or integration into other workflows.
\nFor example, to export a complete or filtered snapshot locally:
\nuvx \\\n --from \\\n git+https://github.com/adafede/marimo \\\n lotus_wikidata_explorer \\\n export \\\n --taxon \"*\" \\ # to get all taxa, else \"Gentianaceae\", for example\n --format csv \\ # also supports json, ttl\n --output 20260119_lotus.csv.gz \\\n --compress \\\n --verboseAnd if you are curious where halogenated compounds appear most often, you can simply ask:
\nxan dedup 20260119_lotus.csv.gz \\\n --select compound_inchikey,molecular_formula,taxon_name | \\\n xan select compound_inchikey,molecular_formula,taxon_name | \\\n xan filter \"contains(molecular_formula, 'Br') or\n contains(molecular_formula, 'Cl') or\n contains(molecular_formula, 'F') or\n contains(molecular_formula, 'I')\" | \\\n xan filter '!contains(molecular_formula, \"Fe\")' | \\\n xan freq -s taxon_name | \\\n xan hist -l value -RHistogram for taxon_name (bars: 11, sum: 11,604, max: 10,060):\n\nStreptomyces | 374 3.22%|\u25a0\u25a0\u25a0\u25a0 |\nLaurencia dendroidea | 314 2.71%|\u25a0\u25a0\u25a0\u25a0 |\nLaurencia obtusa | 184 1.59%|\u25a0\u25a0 |\nPseudoceratina purpurea | 112 0.97%|\u25a0\u25a0 |\nAplysia dactylomela | 99 0.85%|\u25a0\u25a0 |\nNostoc | 99 0.85%|\u25a0\u25a0 |\nLaurencia nipponica | 95 0.82%|\u25a0 |\nPortieria hornemannii | 94 0.81%|\u25a0 |\nLyngbya majuscula | 88 0.76%|\u25a0 |\nChaetomium globosum | 85 0.73%|\u25a0 |\n<rest> |10,060 86.69%|\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0|\n\nAt some point, flow only runs if it is maintained.
\n
As I do not post as often as I probably should, here are some other pointers to related work and discussions from the past months, for those who want to follow the flow a bit further:
\nRecently, I also contributed to improving how chemical reactions are modeled in Wikidata.
\nPreviously, many reactions were modeled as instances of \"chemical reaction\", which violated disjointness, see (Do\u01e7an and Patel-Schneider 2025).
\nThe introduction of type of chemical reaction now allows reactions to be classified more precisely, while preserving their hierarchy using subclass of.
\nOut of curiosity, I also made a small contribution to chemistry-related functions in Wikifunctions: Z30950. It validates CAS Registry Numbers. It does one thing, and it does it reliably.
\nIt is tiny. But it is a seed.
\nProjects like Wikifunctions and Abstract Wikipedia point to the next phase of open knowledge: knowledge that is not only stored, but executed, reused, and recombined.
\nA global, open metabolomics knowledge graph is slowly taking shape, one where chemical structures, organisms, reactions, and evidence can finally be traced together.
\nLOTUS is no longer an initiative. It is one contributor among many in that graph.
\nIf you edit Wikipedia, curate Wikidata, maintain a SPARQL endpoint, write a template, review a module, or fix a tiny detail no one will notice, thank you. This work only matters because you are here.
\n@online{rutz2026,\n author = {{Adriano Rutz}},\n title = {Cultivating {Knowledge} {Flow} in {Open} {Chemistry}},\n date = {2026-01-20},\n url = {https://adafede.github.io/posts/2026-01-20_chem_flow.html},\n doi = {10.59350/sk00y-3gh44},\n langid = {en}\n}\nThe Rogue Scholar archive of scholarly blogs has grown to 34 blogs with about 420 blog posts. In order to implement DOI registration for these blog posts, I needed two things:
Initial work on DOI registration for blog posts focussed on exposing the relevant metadata on the blog landing page, using schema.org and/or HTML meta tags. While this approach worked well for this and similar blogs, it was too complicated and didn't scale well for the large and diverse number of blogs the Rogue Scholar aims to cover.
Therefore I implemented a different workflow taking advantage of the fact that all blogs come with RSS feeds that include content and metadata. More work was needed because there are different formats for these feeds (multiple flavors of RSS, as well as Atom, and the newer JSON Feed). Luckily, libraries in multiple programming languages exist to simplify the parsing of the various RSS Feed formats (I use the Javascript library feed-extractor).
The main challenge with metadata for blog posts \u2013 and with DOI metadata more general \u2013 is author names. They might not be natural names (for example mfenner instead of Martin Fenner), might be names for organizations and not people, the blogging platform might not support multiple authors, and some work is required to include the ORCID author identifier (or ROR institutional. identifier). The Atom format supports an author URL, which can hold the ORCID ID (or ROR ID), and Wordpress can be enhanced with the popular Co-Authors Plus plugin to support multiple authors.
The other challenge with DOI registration is keeping track of the content that has already been registered, and for this I launched a database, with one record for each post. I need the database also to enable full-text search across all blog posts, something I will implement in the coming weeks.
Will all the required pieces coming together, I was finally able to start DOI registrations yesterday. You will easily detect blog posts with a DOI on the Rogue Scholar website (there is a DOI icon next to the title, and the underlying link to the blog post is a DOI):
![\"\"](\"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2023/06/Bildschirmfoto-2023-06-05-um-14.58.49.png\")
The process of DOI registration for all included blog posts should be concluded by the end of the month. There is more work needed to resolve issues with some author names, and DOI registration can be further automated (I am currently using GitHub Actions and a cronjob).
What also needs more work is getting the DOIs displayed on the blogs (the DOIs resolve to the blog post and not the Rogue Scholar archive). This is probably straightforward when using a static site generator, but requires more work when a database is involved (e.g. Wordpress). For Ghost blogs like this one, I found the canonical_url field to be a good place to store the DOI.
","doi":"https://doi.org/10.53731/m9fs5-nap05","guid":"https://doi.org/10.53731/m9fs5-nap05","image":"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2023/06/Bildschirmfoto-2023-06-05-um-14.58.49-1.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1685923200,"rid":"aas3g-wcg94","summary":"The Rogue Scholar archive of scholarly blogs has grown to 34 blogs with about 420 blog posts. In order to implement DOI registration for these blog posts, I needed two things: * Content and metadata, ideally without requiring blogs to implement anything special.","tags":["Metadata","Rogue Scholar"],"title":"Starting to register DOIs for all blog posts included in the Rogue Scholar","updated_at":1787210289,"url":"https://blog.front-matter.de/posts/starting-to-register-dois-for-all-blog-posts-included-in-the-rogue-scholar/","version":"v1"}},{"document":{"authors":[{"affiliation":[{"name":"Front Matter"}],"contributor_roles":[],"family":"Fenner","given":"Martin","url":"https://orcid.org/0000-0003-1419-2405"}],"blog":{"authors":[{"name":"Martin Fenner","url":"https://orcid.org/0000-0003-1419-2405"}],"community_id":"15a362ea-8138-42b8-917f-1840a92addf8","created":1672531200,"current_feed_url":null,"description":"The Front Matter Blog covers the intersection of science and technology since 2007.","doi":"https://doi.org/10.53731/front_matter","favicon":"https://rogue-scholar.org/api/communities/15a362ea-8138-42b8-917f-1840a92addf8/logo","feed_format":"application/atom+xml","feed_url":"https://blog.front-matter.de/atom","filter":null,"generator":"Ghost","home_page_url":"https://blog.front-matter.de/","issn":"2749-9952","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.53731","relative_url":null,"secure":true,"slug":"front_matter","status":"active","subfield":"1710","title":"Front Matter","updated":1787218226,"use_api":true},"blog_name":"Front Matter","blog_slug":"front_matter","content_html":"The dedicated API for the Rogue Scholar science blog archive launched two weeks ago. The initial release supported fetching metadata and content from Rogue Scholar. Today this API was updated with important new functionality: parsing of science blog posts and storing the metadata and content in the Rogue Scholar. This continues the transition from a Javascript serverless API that is part of the frontend application to a dedicated Python API that can better support long-running and/or complex processes such as constantly (currently every 10 min) fetching new blog posts from 60+ participating blogs and parsing the metadata and content.
The updated API is again available as Open Source software via PyPi, GitHub, and Zenodo. Installing the API on your computer requires data from the database (Postgres) and search index (Typesense), which both also use Open Source software but need authentication for access. The API is documented via the Swagger UI, and is well-suited to be used in Jupyter notebooks that take advantage of the metadata and content hosted on Rogue Scholar, as of today 9,512 science blog posts from 65 participating blogs.
One interesting idea is to write overlay blog posts, and Heinz Pampel did exactly that today when he published a blog post commemorating Open Access Week 2023 (which started Monday with the motto Community over Commercialization), referencing several blog posts archived in Rogue Scholar.
This blog post is a follow-up to a post in September (Fenner 2022a), where I announced that I had started working on an archive for scholarly blog posts based on the InvenioRDM open-source repository software. In the last two months, I focussed on two activities \u2013 besides lots of physical therapy and other training following a stroke earlier this year (Fenner 2022b): helping to make it easier (and safer) to run InvenioRDM in Docker container infrastructure, and working on converting the bolognese metadata conversion Ruby gem (Fenner 2017) to Python (work in progress on GitHub) to enhance InvenioRDM functionality.
Building an archive of scholarly blog posts faces the same fundamental challenges as repositories for other types of scholarly content, whether data, software, preprints, or journal articles. You have to collect metadata and content, and that approach only scales with standardization and open licenses.
Luckily we already know a lot about required and optional but desired scholarly metadata, and they are fundamentally not different for scholarly blog posts. This means we can take similar approaches as we have for example taken for research data:
![\"\"](\"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2022/12/guidelines-3.png\")
Persistent identifiers for blog posts can be DOIs, as this blog is doing since earlier this year (Fenner 2022). The main advantage of using DOIs is registering standard metadata stored independently of the blogging platform, in case the platform changes or disappears (as has happened several times in the 15 years this blog exists). While there are several blogs using DOIs for their posts, they often fail in guideline #4: the persistent identifier must be embedded in the landing page in machine-readable format. This is important so that reference managers can capture the DOI and retrieve the associated metadata.
When the metadata are embedded directly in the blog post, schema.org markup in JSON-LD format (guideline #7) is much more convenient than HTML meta tags (guideline #8), but for the time being reference managers only work with the latter. The blogging platform used for this blog (https://ghost.org/) has schema.org metadata built in, and there was only a small amount of work needed to expose all metadata needed (or desired) for DOI registration:
An issue I have seen with schema.org metadata is that sometimes they are added by a script running in the browser instead of coming from the server, and this makes metadata harvesting unreliable. Multiple versions and different levels of granularity \u2013 a major challenge when working with data and software \u2013 luckily is not a major issue with scholarly blogs, in this regard, they behave similarly to preprints and journal articles.
There are several possible approaches to building infrastructure for scholarly blog posts, and they all have well-known real-world examples:
Based on the above, what makes sense for a scholarly blog post archive?
Fenner, M. (2022a). Starting Work on the Front Matter Archive [Blog]. Front Matter. https://doi.org/10.53731/9z6rz5d-djbay0y
Fenner, M. (2022b). I spent the last five months in the hospital [Blog]. Front Matter. https://doi.org/10.53731/bkkzj8g-gd14mb6
Fenner, M. (2017). Bolognese: A Ruby library for conversion of DOI Metadata. DataCite. https://doi.org/10.5438/N138-Z3MK
Fenner, M., Crosas, M., Grethe, J. S., Kennedy, D., Hermjakob, H., Rocca-Serra, P., Durand, G., Berjon, R., Karcher, S., Martone, M., & Clark, T. (2019). A data citation roadmap for scholarly data repositories. Scientific Data, 6(1), Article. https://doi.org/10.1038/s41597-019-0031-8
Fenner, M. (2022c). DOI Registrations for all Ghost Blogs [Blog]. Front Matter. https://doi.org/10.53731/fezg09h-hgn1gzm
Fenner, M. (2009). Interview with Geoffrey Bilder [Blog]. Front Matter. https://doi.org/10.53731/r294649-6f79289-8cw1h
","doi":"https://doi.org/10.53731/br9f5xa-a556w2t","guid":"https://doi.org/10.53731/br9f5xa-a556w2t","image":"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2022/12/guidelines-4.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1670803200,"reference":[{"id":"https://doi.org/10.53731/9z6rz5d-djbay0y","unstructured":"Fenner, M. (2022, September 28). Starting Work on the Front Matter Archive. Front Matter."},{"id":"https://doi.org/10.53731/bkkzj8g-gd14mb6","unstructured":"Fenner, M. (2022, July 28). I spent the last five months in the hospital. Front Matter."},{"id":"https://doi.org/10.5438/n138-z3mk","unstructured":"Fenner, M. (2017). Bolognese: a Ruby library for conversion of DOI Metadata [Computer software]. DataCite."},{"id":"https://doi.org/10.1038/s41597-019-0031-8","unstructured":"Fenner, M., Crosas, M., Grethe, J. S., Kennedy, D., Hermjakob, H., Rocca-Serra, P., Durand, G., Berjon, R., Karcher, S., Martone, M.& Clark, T. (2019). A data citation roadmap for scholarly data repositories. Scientific Data, 6(1)."},{"id":"https://doi.org/10.53731/fezg09h-hgn1gzm","unstructured":"Fenner, M. (2022, August 25). DOI Registrations for all Ghost Blogs. Front Matter."},{"id":"https://doi.org/10.53731/r294649-6f79289-8cw1h","unstructured":"Fenner, M. (2009, February 17). Author Identifiers: Interview with Geoffrey Bilder. Front Matter."}],"rid":"0mwpp-hqg82","summary":"This blog post is a follow-up to a post in September (Fenner 2022a), where I announced that I had started working on an archive for scholarly blog posts based on the InvenioRDM open-source repository software.","tags":["Rogue Scholar","Open Infrastructure"],"title":"Building an archive for scholarly blog posts","updated_at":1787210287,"url":"https://blog.front-matter.de/posts/building-an-archive-for-scholarly-blog-posts/","version":"v1"}},{"document":{"authors":[{"affiliation":[{"id":"https://ror.org/008zgvp64","name":"Public Library of Science"}],"contributor_roles":[],"family":"Fenner","given":"Martin","url":"https://orcid.org/0000-0003-1419-2405"}],"blog":{"authors":[{"name":"Martin Fenner","url":"https://orcid.org/0000-0003-1419-2405"}],"community_id":"15a362ea-8138-42b8-917f-1840a92addf8","created":1672531200,"current_feed_url":null,"description":"The Front Matter Blog covers the intersection of science and technology since 2007.","doi":"https://doi.org/10.53731/front_matter","favicon":"https://rogue-scholar.org/api/communities/15a362ea-8138-42b8-917f-1840a92addf8/logo","feed_format":"application/atom+xml","feed_url":"https://blog.front-matter.de/atom","filter":null,"generator":"Ghost","home_page_url":"https://blog.front-matter.de/","issn":"2749-9952","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.53731","relative_url":null,"secure":true,"slug":"front_matter","status":"active","subfield":"1710","title":"Front Matter","updated":1787218226,"use_api":true},"blog_name":"Front Matter","blog_slug":"front_matter","content_html":"The standard local file formats for bibliographic data are probably bibtex and RIS. They have been around for a long time, and are supported by all reference managers and many other tools and services. Unfortunately these formats are far from perfect:
@article{fenner2012a,\n title = {One-click science marketing},\n volume = {11},\n url = {http://dx.doi.org/10.1038/nmat3283},\n doi = {10.1038/nmat3283},\n number = {4},\n journal = {Nature Materials},\n publisher = {Nature Publishing Group},\n author = {Fenner, Martin},\n year = {2012},\n month = {mar},\n pages = {261-263}\n}One obvious solution would be to store bibliographic data in XML or JSON. These formats have very good support in all programming languages, and they are the formats used by APIs on the web. There have been some efforts to standardize these formats for bibliographic data, e.g. BibJSON, MODS, BibTeX XML or Endnote XML.
<bibtex:entry id='fenner2012a'>\n <bibtex:article>\n <bibtex:title>One-click science marketing</bibtex:title>\n <bibtex:volume>11</bibtex:volume>\n <bibtex:url>http://dx.doi.org/10.1038/nmat3283</bibtex:url>\n <bibtex:doi>10.1038/nmat3283</bibtex:doi>\n <bibtex:number>4</bibtex:number>\n <bibtex:journal>Nature Materials</bibtex:journal>\n <bibtex:publisher>Nature Publishing Group</bibtex:publisher>\n <bibtex:person>\n <bibtex:first>Martin</bibtex:first>\n <bibtex:last>Fenner</bibtex:last>\n <bibtex:person><bibtex:author/>\n <bibtex:year>2012</bibtex:year>\n <bibtex:month>mar</bibtex:month>\n <bibtex:pages>261-263</bibtex:pages>\n </bibtex:article>\n</bibtex:entry>My problem with these formats is that they are made for computers talking to each other and not humans. I personally think that a file with bibliographic data should be human-readable, similar to why I like markdown for writing scientific documents.
When you have too many standards and are not happy with any of them, you of course create a new standard.
My suggestion for a new bibliographic file format is twofold: a) use YAML for data serialization and b) use CSL as data format. YAML is a data format popular with Ruby Developers and is described on the YAML website as
YAML is a human friendly data serialization standard for all programming languages.
Something that not may people seem to know is that YAML is a superset of JSON and that every JSON file is also a valid YAML file. The main difference is the better human readability of YAML.
Citation Style Language is described on the CSL website as
CSL is an open XML-based language to describe the formatting of citations and bibliographies.
Although some commercial applications still use proprietary citation styles, CSL has become the de facto standard, and is used by the reference managers Zotero, Mendeley, Papers, and others. This blog uses CSL via Pandoc and the citeproc-hs library. CSL processors need bibliographic data in a standard format. The popular Citeproc-js Javascript CSL processor by Frank Bennett for example uses JSON, but we might as well use YAML:
- title: One-click science marketing\n volume: '11'\n URL: http://dx.doi.org/10.1038/nmat3283\n DOI: 10.1038/nmat3283\n issue: '4'\n container-title: Nature Materials\n publisher: Nature Publishing Group\n author:\n - family: Fenner\n given: Martin\n orcid: 0000-0003-1419-2405\n page: 261-263\n id: fenner2012a\n type: article-journal\n issued:\n date-parts:\n - 2012\n - 3I hope you agree that this format is not only structured and can be understood by computers, but is also very readable by humans. You may have noticed that I have inserted my ORCID, something that is very difficult to do with bibtex where all authors are stored in one text string (see above).
Careful readers of this blog will of course remember that I have written about using YAML to store metadata about a blog post. We could now add bibliographic information to these metadata, either in the YAML frontmatter (if it is a Jekyll blog), or in a separate file. It should be straightforward to adapt the existing CSL processors to understand YAML since YAML and JSON are so similar. To get started with some Citeproc YAML, use the new (and still experimental) ORCID Feed Webservice with your ORCID and specify the yml format, e.g. http://feed.labs.orcid-eu.org/0000-0003-1419-2405.yml for my publications.
Tobias Kind has submitted a very thoughtful comment (in reply to Approaches to compound documents \u2013 ORE, PDF, DOCX) which deserves printing and commenting.
\n\nTK: Hello Peter,
\nthanks for your thoughts. The more I read the more complex and frustrating it gets. I was just reading your comments about Adobe Acrobat; I would assume that everybody in the chemistry world has an Acrobat Full license. But I recognize that's not the case. Furthermore there are people who have problems opening a ZIP file, so one can not assume that everybody is operating at the same level of tools.\n
PMR: I personally do not have an Acrobat full licenCe. That's not religious \u2013 I just don't have one.\u00a0 Maybe the University has a site licence \u2013 I don't know. FWIW I send manuscripts as *.doc. I realise this is also a proprietary and yes, it has to be paid for. It's just that I happen to have it painlessly. In contrast it is possible to get Open Source/free solutions for ZIP (though only after the infamous patent ran out).
\n\nTK: That's the problem with the long tail. According to the power law it is probably safe to assume that the majority of chemists doesn't even care if there is chemical semantics lurking out of a document. Not to offend the majority of chemists, but at the end of the day it's only the number of publications on the CV that counts (well quality of course).\n
PMR: agreed
\n\nTK: Tim,
\nRSC with Project Prospect and Nature with journals that annotate structures and submit them to PubChem are probably top notch regarding semantics. And as I said before, yes PDF can include metadata with XMP, http://www.adobe.com/products/xmp/ but as long as there are no easy (free) tools out there its hard to push semantics from the PDF side. Acrobat Reader 8 did not know XMP and yes one can attach XML using the Full Acrobat. But the mentioned ExifTool is not a commodity tool for most chemists.\n
PMR: agreed. I am a pragmatist in that I can reasonably persuade chemists to include various bits of information into Word2007, but not into LaTeX or Acrobat. If everyone used semantic Acrobat instead of Word I'd probably be suggesting that. I'm pleased to see that Writer is better than Open Office, but it doesn't (I think) solve the semantic packaging problem seamlessly.
\n\nTK:\u00a0 But then again the whole semantics train currently depends on the journal itself or the editorial board, or single people or innovative groups at the publisher side. And there is certainly the tools side, so for WEB 2.0 in chemistry only a broad range of software tools can act as an enabler for chemical semantics.\n
PMR: Completely agreed. This is why the International Union of Crystallography deserves praise \u2013 it designs and requires semantic data publication for many of its journals.
\n\nTK:Peter,
\nI would not go so far as \"PDF corrupts and restricts thought\". Chemists can not make third parties responsible for the current mess in missing annotations and data exchange. Most of the better chemistry and life sciences journals allow supporting info, so what speaks against attaching the source HTML, DOC as supplement. Yes its redundant, but as long as publishers do not convert supporting data into bitmap PDF it is not a problem.\n
PMR: Like Tufte I allow myself some hyperbole. I would certainly say that \"in a digital age where many new forms of information and publication are possible, a universally used format whose primary purpose is to allow printing of documents onto paper is an active restriction on the imagination\".
\n\nPMR: As an example if you go to ACS Journal of Proteome Research, you can find some of the evil PDFs, and even the evil flat 2D PDF attachments including molecular spectra or information. But a few publications also include supplement RAW data (as XLS. MDB or ZIP) and even PDB codes. So I assume if the authors and reviewers insist on publishing meta data in the supplement in a specific format the journal would agree. Well, then there is that unholy ACS supplement data copyright. But there are also ways to submit data on personal websites. For instance you could find the ACS journal supplement data for \"T.IMPAFIFEHIIK.R\" also on google: \"Powered by Yates Bioinformatics Team; This is ongoing project with preliminary results\", ok copyrighted by the Yates group itself
\n
PMR: If you look at ACS J.Org.Chem you will see that almost every paper has a large supplement. This is almost always in PDF. It's clearly taken a lot of work to create. The information was, originally, semantic and the publication process has encouraged the community to turn it into PDF. The spectra were JCAMPs (or could be JCAMPs), The molecules were CDX or Mol, The reactions were RXN, etc. All have been steamrollered into flat PDFs.
\nPMR: The exception are the CIFs, designed, advocated, and managed by the IUCr. They have shone as an example to the rest of the chemical world.
\n\nTK: For example some of our public US taxpayer funded metabolomics data sets are fully available via our SetupX LIMS and study design database:
\nhttp://fiehnlab.ucdavis.edu:8080/m1/
\nFor those public studies people can download all the raw data and all the annotated and result data and even the underlying software. Not all research data is open access and publicly available and yes we are also guilty of publishing flat PDFs without any semantics, but we allow people to reproduce some of our experiments and download RAW and processed data and all needed software and that can only be topped by Open NoteBook Science, the purest form of scientific reporting.\n
PMR: This again is the influence of the bioscience community. It makes me envious.
\nTK: Tobias Kind
\nfiehnlab.ucdavis.edu
It's technically trivial \u2013 yes trivial \u2013 to publish molecules and spectra. If a journal said \"no need to write 200 pages of supplemental info in PDF, just publish the *.cdx, *.mol. \".jdx.\" That's all. But where is the editorial push for this? Will any chemical editors (technical, management, academic) step up and say \"this journal will require authors to deposit semantic chemistry in \u2026 months/years\". That's all it takes. There wouldn't even be much resistance \u2013 probably rejoicing.
\nThe good news is that we have an Open Source infrastructure that can convert all of these legacy formats into semantic chemistry (Chemical Markup Language, CML) essentially automatically. We've done it for crystallography in the chemistry depertament here and the issues are not technical but things like embargoes.
\nYou don't even need to know about CML.
\nI'll be explaining in future posts how it is now conceptually simple to publish chemical data in semantic form. I'd like to work with, not against, publishers. And, with some like IUCr and RSC we do.
I ran a TD-DFT calculation on a platinum complex, fed the log file to ELOTE, and got only half an answer. The molecular orbital composition table printed just fine, every orbital with its atomic percentages. But the excited-state analysis, the whole point of the exercise, came back empty. The CSV had a header and nothing underneath. No transitions, no MLCT/LMCT labels, no summary table. Nothing. The calculation was perfectly healthy. So what was going on?
\nIf you've only ever run closed-shell TD-DFT, you're used to seeing excited states written like this:
\n Excited State 1: Singlet-A 2.14 eV 580 nm f=0.0198\n 150 ->151 0.51251\n 150 ->152 -0.30648\n\nOrbitals are labeled by clean integers, a right pointing arrow ->, and the CI coefficient. That's what ELOTE was built to read. But my platinum system was an open-shell calculation and therefore an unrestricted one, with unpaired electrons. Gaussian writes those excited states differently:
Excited State 1: 5.013-A 0.2171 eV 5709 nm f=0.0000 =6.033\n 48B -> 50B 0.67064\n 43B <- 50B -0.17577\n \nIn these calculations the orbital number now specifies if it belongs to the alpha (A) or beta (B) manifold, in other words, it now bears a spin label: 48B, 50B. Second, there's a backwards arrow, <-, mixed in with the forward ones, indicating de-excitations (vide infra)
Both of those broke the parser silently. ELOTE couldn't handle the spin label of the orbital (it expected just a number), so it quietly recorded zero transitions for every single state. Hence the empty tables. No error, no complaint, nothing, which is the most confusing kind of bug. (Sorry about that.)
\nSince orbital 48A isn't the same as 48B, it isn't enough to strip the letter and move on \u2014 ELOTE has to keep the alpha and beta orbitals in separate bookkeeping, label them separately (you'll now see HOMO(B), LUMO(B) and so on), and then decide the metal-versus-ligand character of each transition using the correct spin's orbital. A beta HOMO and an alpha HOMO can have genuinely different atomic compositions, and conflating them would hand you a wrong assignment while looking perfectly confident about it.
With the parser taught to read both manifolds, my platinum spectrum came apart cleanly. The lowest excited state, for instance, turns out to be predominantly a beta HOMO-1 \u2192 LUMO transition with roughly 38% weight, dressed with a handful of smaller ligand-to-metal and intraligand contributions, and everything sums to a tidy 100%. Exactly the kind of table you'd want to drop into a manuscript.
\nIf you're running open-shell systems, you don't have to do anything differently. ELOTE detects the unrestricted output on its own and prints a short note letting you know it's in that mode:
\n\nUnrestricted (open-shell) calculation detected: alpha (A) and beta (B)orbital manifolds are labeled separately.\nThe orbital labels then carry their spin \u2014 HOMO(B), LUMO+2(A) \u2014 so there's never any ambiguity about which manifold a transition belongs to. Closed-shell calculations behave exactly as before; nothing changed for them.
A word of caution that has nothing to do with ELOTE and everything to do with chemistry. My test system had an
The update is live on GitHub, and if you'd found this the hard way like I did, I hope this saved you an afternoon.
\nAs always \u2014 questions and comments welcome below.
\nIn a restricted (closed-shell) calculation, every spatial orbital holds two electrons, one spin-up and one spin-down, and they share the same shape. There is one set of orbitals, and one HOMO, and one LUMO. Simple.
\nIn an unrestricted (open-shell) calculation \u2014 the kind you need whenever you have unpaired electrons \u2014 spin-up (\u03b1) and spin-down (\u03b2) electrons are allowed to occupy different spatial orbitals. So Gaussian keeps two separate sets of orbitals: an alpha manifold and a beta manifold, each with its own numbering, its own HOMO, its own LUMO. The B suffix is simply telling you which manifold a given orbital lives in. 48B is the 48th beta orbital, and it is not the same object as 48A.
The backward arrow, <-, is the other subtlety. It's a de-excitation contribution, a small term that the linear-response formalism includes alongside the main excitations. For the purpose of assigning character, the physically sensible thing is to read it as donor and acceptor swapped, so ELOTE now flips it and treats it consistently.
Cite ELOTE: 10.5281/zenodo.21799957 (Zenodo DOI) \u00b7 Repository: github.com/joaquinbarroso/ELOTE
","doi":"https://doi.org/10.59350/0swjx-kfe02","guid":"http://joaquinbarroso.com/?p=7018","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1785888000,"rid":"z9wae-5fd43","summary":"I ran a TD-DFT calculation on a platinum complex, fed the log file to ELOTE, and got only half an answer. The molecular orbital composition table printed just fine, every orbital with its atomic percentages. But the excited-state analysis, the whole point of the exercise, came back empty.","tags":["Coding","Computational Chemistry","ELOTE","Excited State","Photochemistry"],"title":"ELOTE now works with Open-Shell systems","updated_at":1787221701,"url":"https://joaquinbarroso.com/2026/08/05/elote-now-works-with-open-shell-systems/","version":"v1"},{"authors":[{"affiliation":[{"name":"Front Matter"}],"contributor_roles":[],"family":"Fenner","given":"Martin","url":"https://orcid.org/0000-0003-1419-2405"}],"blog":{"authors":[{"name":"Martin Fenner","url":"https://orcid.org/0000-0003-1419-2405"}],"community_id":"15a362ea-8138-42b8-917f-1840a92addf8","created":1672531200,"current_feed_url":null,"description":"The Front Matter Blog covers the intersection of science and technology since 2007.","doi":"https://doi.org/10.53731/front_matter","favicon":"https://rogue-scholar.org/api/communities/15a362ea-8138-42b8-917f-1840a92addf8/logo","feed_format":"application/atom+xml","feed_url":"https://blog.front-matter.de/atom","filter":null,"generator":"Ghost","home_page_url":"https://blog.front-matter.de/","issn":"2749-9952","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.53731","relative_url":null,"secure":true,"slug":"front_matter","status":"active","subfield":"1710","title":"Front Matter","updated":1787218226,"use_api":true},"blog_name":"Front Matter","blog_slug":"front_matter","content_html":"Starting this week the Rogue Scholar science blog archive collects citations of its blog posts using invenio-pidbox, the Python module for the InvenioRDM repository platform that I launched earlier this month, and replacing the integration that used the Crossref cited-by service launched in August 2025.
The reasons for this change are threefold:
invenio-pidbox uses a sqlite database to store scholarly metadata and uses the commonmeta-py Python library and its Rust bindings to query sqlite and integrate with the InvenioRDM repository platform that Rogue Scholar uses. The initial invenio-pidbox versions used the ROR and ORCID regular data dumps (135,710 organizations and 25,048,058 people, respectively) to generate author and organization profiles. With this release I started to also store Crossref and DataCite metadata in sqlite. Initially only the metadata for the blog posts in Rogue Scholar, so only tracking citations by other blog posts archived in Rogue Scholar, but the architecture allows me to scale to all 180 million Crossref DOIs and 120 million DataCite DOIs in their annual data dumps, and the about 2.4 billion references they contain, all in all about 2 TB of data. This sounds like a lot of data, but sqlite can easily handle this, and this is much easier and cheaper than importing all the metadata into Postgres and Opensearch, the database and search index used by InvenioRDM.
invenio-pidbox comes with a celery background worker task to extracts all Rogue Scholar citations out of sqlite and stores them as InvenioRDM metadata so that they can be shown and queried in Rogue Scholar. Blog post citations can be much faster than citations by journal articles, e.g. five days after the publication of the InvenioRDM v14 announcement:
Citations counts in Rogue Scholar are displayed for blogs, other communities, authors and institutions:
Clicking on the citation count (here 58) opens a query for all cited blog posts. The DataCite blog is also an example for tracking citations of blog posts with DataCite DOIs, something that Rogue Scholar couldn't do until this week.
invenio-pidbox is an invenio module that should work with all InvenioRDM v14 instances, and besides Rogue Scholar they currently all use DataCite DOIs for their content, making this an attractive option for other InvenioRDM instances that have 2 TB disk space to spare \u2013 not S3 object storage as sqlite needs fast disk access.
Institution profiles are an interesting use case. Their main challenge is incomplete metadata, e.g only 137 of currently 683 posts in the Crossref blog have affiliation metadata. Affiliation metadata (using ROR) are more difficult than author metadata (using ORCID), as affiliations have a start and end date, and authors can have multiple affiliations in parallel.
But with 300 million DOIs for works from Crossref and DataCite stored in sqlite in commonmeta format it becomes much easier to extract that information at scale, not limited by API rate-limits or differences between Crossref and DataCite metadata.
Please reach out via Slack, email, Mastodon, or Bluesky if you have any questions or comments.
Wer schon einmal auf eine Datei mit dem Namen final_neu_wirklichfinal.xlsx gesto\u00dfen ist, kennt das Problem: Daten k\u00f6nnen schnell un\u00fcbersichtlich werden. In der Forschung ist das nicht nur l\u00e4stig, sondern kann dar\u00fcber entscheiden, ob Ergebnisse nachvollziehbar, nachnutzbar und langfristig auffindbar bleiben. Genau hier setzt Forschungsdatenmanagement (FDM) an. Aber wie erkl\u00e4rt man FDM so, dass es f\u00fcr Studierende greifbar wird?
\nMit dieser Frage besch\u00e4ftigte sich eine Gruppe mit acht Studierenden des Bachelorstudiengangs Informationsmanagement an der Hochschule Hannover (HsH) im Sommersemester 2026. Im Rahmen eines Projekts im 6. Semester, betreut durch das Joint Lab Future Libraries & Research Data von HsH und TIB, entwickelten sie Materialien, die FDM aus studentischer Perspektive zug\u00e4nglich machen: einen Selbstlernkurs und modulare Seminareinheiten \u2013 konzipiert von Studierenden f\u00fcr Studierende.
\nAusgangspunkt war die Beobachtung, dass FDM im bisherigen Studienverlauf explizit kaum vorkam. Daher verfolgte das Projekt das Ziel, Materialien zu entwickeln, die niedrigschwellig erkl\u00e4ren, worum es bei FDM geht und warum es schon im Studium n\u00fctzlich ist. Das Projekt war bewusst offen angelegt. Neben der fachlichen Auseinandersetzung mit FDM geh\u00f6rten auch eigenst\u00e4ndige Organisation und Projektmanagement dazu.
\nDabei sollten vorhandene Open Educational Resources (OER) weitergedacht und an eine studentische Zielgruppe angepasst werden. Konkret kn\u00fcpfte die Gruppe dabei an den Kurs \"Nachhaltige Wissenschaft mit Forschungsdatenmanagement \u2013 Eine Einf\u00fchrung f\u00fcr Betreuende von Qualifizierungsarbeiten\" an, der bereits im TIB-Blog vorgestellt wurde.
\nBesonders interessant war der Perspektivwechsel: Die Studierenden entwickelten Materialien f\u00fcr andere Studierende und pr\u00fcften dabei immer wieder, was wirklich verst\u00e4ndlich, brauchbar und motivierend ist. Was braucht man als Einstieg? Welche Begriffe m\u00fcssen erkl\u00e4rt werden? Wo helfen Beispiele mehr als Definitionen? Gleichzeitig wurde FDM Teil des Projektalltags. Einheitliche Dateibenennungen, Dokumentation, Versionierung, regelm\u00e4\u00dfige Backups und README-Dateien waren nicht nur Kursthemen, sondern praktische Werkzeuge in der Zusammenarbeit. So wurde aus einem abstrakten Thema schnell eine konkrete Projekterfahrung.
\nF\u00fcr die Erarbeitung der OER teilte sich die Gruppe in zwei Teams auf, die an den unterschiedlichen Formaten arbeiteten und ihre Materialien jeweils iterativ auf Basis von Feedback aus Lehre und Praxis weiterentwickelten.
\nEin Teilteam entwickelte einen Onlinekurs auf Basis von LiaScript. Ziel war ein kompakter Selbstlernkurs, der FDM-Grundlagen in rund 120 Minuten im Selbststudium vermittelt und Studierenden einen ersten, gut zug\u00e4nglichen \u00dcberblick bietet. Daf\u00fcr arbeitete sich die Gruppe in die Funktionsweise und den Syntax von LiaScript ein. Eine Herausforderung bestand darin, dass kollaboratives Arbeiten in diesem Format nur eingeschr\u00e4nkt m\u00f6glich war und der Zugang \u00fcber GitHub zus\u00e4tzliche technische H\u00fcrden mit sich brachte.
\nInhaltlich ging es darum, bestehende OER mit einer \"Studierendenbrille\" zu \u00fcberarbeiten: Was ist f\u00fcr die Zielgruppe wirklich relevant? Welche Beispiele machen FDM greifbar? Und wie l\u00e4sst sich ein Thema, das schnell trocken wirken kann, in ein \u00fcbersichtliches Selbstlernformat bringen?
\nDas Skript zum Selbstlernkurs ist auf GRO.data ver\u00f6ffentlicht: https://doi.org/10.25625/HLSNR1. Hier geht es direkt zum Selbstlernkurs.
\nDas zweite Teilteam entwickelte drei Seminareinheiten, die Lehrende flexibel in bestehende Veranstaltungen einbauen k\u00f6nnen. Erg\u00e4nzt wurden die Folien durch \u00dcbungen, Arbeitsbl\u00e4tter und Rahmenhinweise. Der modulare Ansatz war dabei zentral: Die Materialien sollen nicht als starre Pr\u00e4sentation funktionieren, sondern als Bausteine, aus denen Lehrende je nach Veranstaltung, Zeitrahmen und Vorwissen der Studierenden ausw\u00e4hlen k\u00f6nnen.
\nAuch hier spielte Feedback eine gro\u00dfe Rolle. Mehrere Versionen wurden \u00fcberarbeitet, angepasst und gesch\u00e4rft. Eine besondere Herausforderung war die Frage, welches Vorwissen k\u00fcnftige Studierende tats\u00e4chlich mitbringen und wie viel Kontext die Materialien deshalb selbst liefern m\u00fcssen.
\nDie Seminarmaterialien sind ebenfalls auf GRO.data ver\u00f6ffentlicht: https://doi.org/10.25625/6105WI.
\nBei der Ver\u00f6ffentlichung der Arbeitsergebnisse sammelte die Gruppe praktische Erfahrungen mit GRO.data, Zenodo und twillo. Das FDM wurde damit ganz konkret: Welche Metadaten werden ben\u00f6tigt? Welche Felder sind verst\u00e4ndlich? Wo fehlen Hinweise? Und wie benutzerfreundlich sind Repositorien aus Sicht von Studierenden?
\nDie Erfahrungen fielen unterschiedlich aus. Zenodo wurde als grunds\u00e4tzlich einfach und verst\u00e4ndlich wahrgenommen, auch wenn an einzelnen Stellen genauere Hilfetexte hilfreich gewesen w\u00e4ren. GRO.data wirkte insgesamt selbsterkl\u00e4rend, stellte die Gruppe aber bei einzelnen Funktionen und Bezeichnungen vor Fragen.
\nGerade diese Erfahrungen machten sichtbar, dass Forschungsdatenmanagement nicht nur aus Prinzipien und Empfehlungen besteht, sondern aus konkreten Entscheidungen, Werkzeugen und manchmal auch technischen H\u00fcrden.
\nDas Projektteam pr\u00e4sentierte die Arbeitsergebnisse auf der BiblioCon 2026 in Berlin in Form eines Posters und einer Pr\u00e4sentation f\u00fcr das Freiraum-Format. Damit wurden die Projektergebnisse nicht nur im Studienkontext genutzt, sondern auch in eine gr\u00f6\u00dfere Fach\u00f6ffentlichkeit getragen.
\nF\u00fcr die Studierenden bedeutete das, die eigene Projektarbeit noch einmal anders zu reflektieren: Was ist das Wesentliche? Welche Erfahrungen sind \u00fcbertragbar? Und wie l\u00e4sst sich ein studentisches Projekt so pr\u00e4sentieren, dass es auch f\u00fcr andere Einrichtungen und Studieng\u00e4nge interessant wird? \u00dcber die Plattform der Tagung wurden die Beitr\u00e4ge anschlie\u00dfend ebenfalls publiziert (Freiraum-Folien: nbn-resolving.org/urn:nbn:de:0290-opus4-205328, Poster: nbn-resolving.org/urn:nbn:de:0290-opus4-205129).
\nBei der BiblioCon boten sich weiterhin interessante Ankn\u00fcpfungspunkte zu \u00e4hnlichen Ans\u00e4tzen in Hochschulen, wie etwa zum OER-Kurs \"Data Skills“ der Uni Rostock, der Forschende und Studierende seit Ende 2025 mit einem modular aufgebauten Selbstlernangebot auf ILIAS durch den gesamten Forschungsdatenzyklus f\u00fchrt.
\nAuch das Verbundprojekt \"Einstein Your Data“ der Universit\u00e4ten Vechta und Oldenburg, gef\u00f6rdert im Rahmen der Landesinitiative FDM Niedersachsen, verfolgt ein \u00e4hnliches Ziel: Es st\u00e4rkt die Data Literacy von Studierenden durch forschendes Lernen, bei dem der gesamte Forschungsprozess von der Planung \u00fcber die Datenerhebung bis zur Archivierung selbst durchlaufen wird. Ein besonderer Fokus liegt dabei auf den Umgang mit sensiblen Daten. Solche Projekte zeigen: Der Bedarf an praxisnaher Vermittlung von Datenkompetenz ist an vielen Standorten pr\u00e4sent und der Austausch \u00fcber Formate und Erfahrungen lohnt sich.
\nR\u00fcckblickend beschreibt die Projektgruppe das Semester als fachlich und organisatorisch lehrreich. Besonders pr\u00e4gend war die Erfahrung, dass FDM nicht erst bei gro\u00dfen Forschungsprojekten beginnt. Neben Herausforderungen, etwa im Umgang mit technischen Schwierigkeiten beim genutzten Cloud-Dienst, war vor allem die Projektarbeit selbst pr\u00e4gend: Wie im FDM selbst erwiesen sich auch im Projektteam eine sorgf\u00e4ltige Dokumentation und klare Kommunikation als zentrale Erfolgsfaktoren.
\nDamit liefert das Projekt mehr als nur die Materialien f\u00fcr k\u00fcnftige Lehrveranstaltungen: Es zeigt, wie FDM-Kompetenzen im Studium praktisch erfahrbar werden k\u00f6nnen \u2013 nicht als zus\u00e4tzliche Pflicht, sondern als Bestandteil guter wissenschaftlicher und kollaborativer Arbeit.
\nAlle Materialien stehen offen lizenziert zur Nachnutzung bereit und laden dazu ein, Forschungsdatenmanagement fr\u00fch im Studium aufzugreifen und an die eigene Lehre anzupassen.
\nJoint Lab Future Libraries & Research Data \u2013 futurelibraries@tib.eu
\nFranziska Altemeier, TIB \u2013 Leibniz-Informationszentrum Technik und Naturwissenschaften und Universit\u00e4tsbibliothek: franziska.altemeier@tib.eu
\nProf. Dr. Ina Bl\u00fcmel, TIB \u2013 Leibniz-Informationszentrum Technik und Naturwissenschaften und Universit\u00e4tsbibliothek & Hochschule Hannover, ina.bluemel@tib.eu\u00a0
\nDr. Nina D\u00fcvel, TIB \u2013 Leibniz-Informationszentrum Technik und Naturwissenschaften und Universit\u00e4tsbibliothek & Hochschule Hannover: nina.duevel@tib.eu
\nI have finally opened a Posts section on my website! Every post should now automatically get a DOI.
This is something I have wanted to do for a long time, largely inspired by the tireless and consistent example set by Egon Willighagen (Willighagen 2024b, 2024a, 2025).
\nIt was today's post of (Fenner 2025) that finally motivated me to look into it again. That led me down a productive rabbit hole to set up Rogue Scholar: first landing on (Csefalvay 2023)'s excellent guide, and then (Fruehwald 2025)'s clear write-up, both of which made the process of integrating Rogue Scholar into a Quarto-based site surprisingly smooth.
\nAll the changes are documented in the following commit:
\nhttps://github.com/Adafede/adafede.github.io/commit/bc2dfe6f
\nIf you care about attribution, long-term archiving, DOIs and metadata, I highly recommend looking into Rogue Scholar.
\nEdit (1): I realized that integrating CiTO could be a significant enhancement. With some effort (and thanks again to Egon), I managed to implement a working solution for the HTML and PDF outputs, see (Willighagen 2023). However, the solution for the XML feed still feels suboptimal.
\nEdit (2): After some help from Egon and Martin, I could improve my feed with correct CiTO annotations and their cool custom json feed, see: https://adafede.github.io/posts.json!
\n@online{rutz2025,\n author = {{Adriano Rutz}},\n title = {Open {Science} {Upgrade:} {Adding} {Blog} {Posts} to My\n {Website} and {Linking} to {Rogue} {Scholar}},\n date = {2025-08-04},\n url = {https://adafede.github.io/posts/2025-08-04_rogue_scholar.html},\n doi = {10.59350/yckwd-9vm79},\n langid = {en}\n}\nFive years ago, LOTUS (Rutz et al. 2022) started as a small attempt to cultivate the flow of chemical knowledge, in the same way we study how metabolites flow through living systems, rather than to build yet another database.
\nThe idea was simple. Natural products data should be open, structured, reusable, and belong to everyone.
\nLike many community-driven efforts, LOTUS had bursts of activity, long pauses, and years of invisible maintenance. From the outside, silence can look like disappearance. From the inside, it usually means people are still doing the work; slowly, carefully, and often without announcements.
\nThe early focus of LOTUS was necessarily inward: assembling data, cleaning records, releasing versions. We built a website, curated entries, and archived releases.
\nBut we slowly realized something uncomfortable: data stored in a repository, even a good one, does not automatically live.
\nArchiving on Zenodo (https://zenodo.org/communities/the-lotus-initiative) was the right thing to do, but archived data is mostly silent data. It waits to be discovered, and versioning remains labor-intensive.
\nWhat we really needed were entry points where people already were.
\nWikipedia , Wikidata , and Scholia building on top of it are not dissemination platforms in the classical sense. They are circulatory systems. They persist because communities maintain them.
\nThe Scholia Chemistry preprint (Willighagen et al. 2025) co-authored with Egon , Denise , Daniel , and Finn belongs to this continuity. It does not introduce a new platform. It offers a lens for communities to see what they already collectively know.
\n\n\nKnowledge needs channels, not just reservoirs.
\n
One important step was enabling Wikipedia articles and infoboxes to directly access found in taxon (P703) relationships from Wikidata in an efficient way.
\nThis sounds like a small technical detail. It is not.
\nThe idea had circulated quietly for years, in hallway conversations, chats, and conferences. I also mentioned it during the WikiCite 2025 conference last August, but it took time before conditions were right. Infrastructure work rarely happens on schedule. It almost never happens on stage. It happens in version histories, talk pages, and tiny edits that fix one Lua bug, enabling thousands of articles to improve forever.
\nOnce data is in Wikidata, it can flow into thousands of chemical articles, in dozens of languages, without duplication or translation overhead. It becomes visible to non-experts, students, and readers who will never see a database interface. This is how open data becomes public knowledge.
\nThese modules are now available on multiple Wikipedias:
\nIf you speak another language, feel free to copy them and increase their use.
\nInitially, I tried to reuse existing modules, and quickly learned why module reuse across Wikipedias is famously difficult. Each wiki evolves its own ecosystem of dependencies, conventions, and technical debt. So the modules were written fully contained, independent of language-specific infrastructure. Only lines that need to be changed are the language-specific translations at the top of the module, trying to follow 18n.
\nThese modules are not a new website or a new interface. They are simply better plumbing.
\n
By default, only 5 organisms are shown, keeping the text clean and readable. But the magic is in how the module handles the rest: each taxon links to its Wikipedia article, and if no article exists in the current language, the module gracefully redirects to an equivalent page in another wiki. A beautiful example of this is war for Quassia africana.
\n\nNot everyone wants to learn SPARQL.
\n
I have heard this sentiment countless times. Regardless of personal preferences, if we want the data to truly live, it must reach as many people as possible.
\nSmall tools like the LOTUS Wikidata Explorer help lower that barrier. It is imperfect. It is still growing. But it already allows chemists, curators, and students to access and export data in formats they can actually use.
\nThe principle is simple: knowledge only flows when it reaches people. The data must be seen, explored, and reused. Only then does it fulfill its purpose.
\n
On a more technical note, the LOTUS Wikidata Explorer leverages the powerful IDSM endpoint (Galgonek and Vondr\u00e1\u0161ek 2021), which allows for chemical similarity searches thanks to Sachem (Kratochv\u00edl et al. 2018). Its speed for large-scale queries could never have been reached without QLever (Bast and Buchhold 2017), and chemical depictions come from CDK Depict.
\nIt returns structured metadata for traceability and reproducibility, together with hashes that uniquely identify the query and its results. It can be queried programmatically via simple API calls, for example ?taxon=Gentianaceae or ?smiles=c1ccccc1&formula_filter=true&f_state=required. It works directly in the browser without requiring heavy dependencies, almost everything works out of the box.
Alternatively, users can take advantage of a local version, for example to extract a small, personal knowledge graph in TTL format, or to export all or selected LOTUS data in more chemistry-friendly formats such as JSON or CSV, ready for analysis, visualization, or integration into other workflows.
\nFor example, to export a complete or filtered snapshot locally:
\nuvx \\\n --from \\\n git+https://github.com/adafede/marimo \\\n lotus_wikidata_explorer \\\n export \\\n --taxon \"*\" \\ # to get all taxa, else \"Gentianaceae\", for example\n --format csv \\ # also supports json, ttl\n --output 20260119_lotus.csv.gz \\\n --compress \\\n --verboseAnd if you are curious where halogenated compounds appear most often, you can simply ask:
\nxan dedup 20260119_lotus.csv.gz \\\n --select compound_inchikey,molecular_formula,taxon_name | \\\n xan select compound_inchikey,molecular_formula,taxon_name | \\\n xan filter \"contains(molecular_formula, 'Br') or\n contains(molecular_formula, 'Cl') or\n contains(molecular_formula, 'F') or\n contains(molecular_formula, 'I')\" | \\\n xan filter '!contains(molecular_formula, \"Fe\")' | \\\n xan freq -s taxon_name | \\\n xan hist -l value -RHistogram for taxon_name (bars: 11, sum: 11,604, max: 10,060):\n\nStreptomyces | 374 3.22%|\u25a0\u25a0\u25a0\u25a0 |\nLaurencia dendroidea | 314 2.71%|\u25a0\u25a0\u25a0\u25a0 |\nLaurencia obtusa | 184 1.59%|\u25a0\u25a0 |\nPseudoceratina purpurea | 112 0.97%|\u25a0\u25a0 |\nAplysia dactylomela | 99 0.85%|\u25a0\u25a0 |\nNostoc | 99 0.85%|\u25a0\u25a0 |\nLaurencia nipponica | 95 0.82%|\u25a0 |\nPortieria hornemannii | 94 0.81%|\u25a0 |\nLyngbya majuscula | 88 0.76%|\u25a0 |\nChaetomium globosum | 85 0.73%|\u25a0 |\n<rest> |10,060 86.69%|\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0|\n\nAt some point, flow only runs if it is maintained.
\n
As I do not post as often as I probably should, here are some other pointers to related work and discussions from the past months, for those who want to follow the flow a bit further:
\nRecently, I also contributed to improving how chemical reactions are modeled in Wikidata.
\nPreviously, many reactions were modeled as instances of \"chemical reaction\", which violated disjointness, see (Do\u01e7an and Patel-Schneider 2025).
\nThe introduction of type of chemical reaction now allows reactions to be classified more precisely, while preserving their hierarchy using subclass of.
\nOut of curiosity, I also made a small contribution to chemistry-related functions in Wikifunctions: Z30950. It validates CAS Registry Numbers. It does one thing, and it does it reliably.
\nIt is tiny. But it is a seed.
\nProjects like Wikifunctions and Abstract Wikipedia point to the next phase of open knowledge: knowledge that is not only stored, but executed, reused, and recombined.
\nA global, open metabolomics knowledge graph is slowly taking shape, one where chemical structures, organisms, reactions, and evidence can finally be traced together.
\nLOTUS is no longer an initiative. It is one contributor among many in that graph.
\nIf you edit Wikipedia, curate Wikidata, maintain a SPARQL endpoint, write a template, review a module, or fix a tiny detail no one will notice, thank you. This work only matters because you are here.
\n@online{rutz2026,\n author = {{Adriano Rutz}},\n title = {Cultivating {Knowledge} {Flow} in {Open} {Chemistry}},\n date = {2026-01-20},\n url = {https://adafede.github.io/posts/2026-01-20_chem_flow.html},\n doi = {10.59350/sk00y-3gh44},\n langid = {en}\n}\nThe Rogue Scholar archive of scholarly blogs has grown to 34 blogs with about 420 blog posts. In order to implement DOI registration for these blog posts, I needed two things:
Initial work on DOI registration for blog posts focussed on exposing the relevant metadata on the blog landing page, using schema.org and/or HTML meta tags. While this approach worked well for this and similar blogs, it was too complicated and didn't scale well for the large and diverse number of blogs the Rogue Scholar aims to cover.
Therefore I implemented a different workflow taking advantage of the fact that all blogs come with RSS feeds that include content and metadata. More work was needed because there are different formats for these feeds (multiple flavors of RSS, as well as Atom, and the newer JSON Feed). Luckily, libraries in multiple programming languages exist to simplify the parsing of the various RSS Feed formats (I use the Javascript library feed-extractor).
The main challenge with metadata for blog posts \u2013 and with DOI metadata more general \u2013 is author names. They might not be natural names (for example mfenner instead of Martin Fenner), might be names for organizations and not people, the blogging platform might not support multiple authors, and some work is required to include the ORCID author identifier (or ROR institutional. identifier). The Atom format supports an author URL, which can hold the ORCID ID (or ROR ID), and Wordpress can be enhanced with the popular Co-Authors Plus plugin to support multiple authors.
The other challenge with DOI registration is keeping track of the content that has already been registered, and for this I launched a database, with one record for each post. I need the database also to enable full-text search across all blog posts, something I will implement in the coming weeks.
Will all the required pieces coming together, I was finally able to start DOI registrations yesterday. You will easily detect blog posts with a DOI on the Rogue Scholar website (there is a DOI icon next to the title, and the underlying link to the blog post is a DOI):
The process of DOI registration for all included blog posts should be concluded by the end of the month. There is more work needed to resolve issues with some author names, and DOI registration can be further automated (I am currently using GitHub Actions and a cronjob).
What also needs more work is getting the DOIs displayed on the blogs (the DOIs resolve to the blog post and not the Rogue Scholar archive). This is probably straightforward when using a static site generator, but requires more work when a database is involved (e.g. Wordpress). For Ghost blogs like this one, I found the canonical_url field to be a good place to store the DOI.
","doi":"https://doi.org/10.53731/m9fs5-nap05","guid":"https://doi.org/10.53731/m9fs5-nap05","image":"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2023/06/Bildschirmfoto-2023-06-05-um-14.58.49-1.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1685923200,"rid":"aas3g-wcg94","summary":"The Rogue Scholar archive of scholarly blogs has grown to 34 blogs with about 420 blog posts. In order to implement DOI registration for these blog posts, I needed two things: * Content and metadata, ideally without requiring blogs to implement anything special.","tags":["Metadata","Rogue Scholar"],"title":"Starting to register DOIs for all blog posts included in the Rogue Scholar","updated_at":1787210289,"url":"https://blog.front-matter.de/posts/starting-to-register-dois-for-all-blog-posts-included-in-the-rogue-scholar/","version":"v1"},{"authors":[{"affiliation":[{"name":"Front Matter"}],"contributor_roles":[],"family":"Fenner","given":"Martin","url":"https://orcid.org/0000-0003-1419-2405"}],"blog":{"authors":[{"name":"Martin Fenner","url":"https://orcid.org/0000-0003-1419-2405"}],"community_id":"15a362ea-8138-42b8-917f-1840a92addf8","created":1672531200,"current_feed_url":null,"description":"The Front Matter Blog covers the intersection of science and technology since 2007.","doi":"https://doi.org/10.53731/front_matter","favicon":"https://rogue-scholar.org/api/communities/15a362ea-8138-42b8-917f-1840a92addf8/logo","feed_format":"application/atom+xml","feed_url":"https://blog.front-matter.de/atom","filter":null,"generator":"Ghost","home_page_url":"https://blog.front-matter.de/","issn":"2749-9952","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.53731","relative_url":null,"secure":true,"slug":"front_matter","status":"active","subfield":"1710","title":"Front Matter","updated":1787218226,"use_api":true},"blog_name":"Front Matter","blog_slug":"front_matter","content_html":"The dedicated API for the Rogue Scholar science blog archive launched two weeks ago. The initial release supported fetching metadata and content from Rogue Scholar. Today this API was updated with important new functionality: parsing of science blog posts and storing the metadata and content in the Rogue Scholar. This continues the transition from a Javascript serverless API that is part of the frontend application to a dedicated Python API that can better support long-running and/or complex processes such as constantly (currently every 10 min) fetching new blog posts from 60+ participating blogs and parsing the metadata and content.
The updated API is again available as Open Source software via PyPi, GitHub, and Zenodo. Installing the API on your computer requires data from the database (Postgres) and search index (Typesense), which both also use Open Source software but need authentication for access. The API is documented via the Swagger UI, and is well-suited to be used in Jupyter notebooks that take advantage of the metadata and content hosted on Rogue Scholar, as of today 9,512 science blog posts from 65 participating blogs.
One interesting idea is to write overlay blog posts, and Heinz Pampel did exactly that today when he published a blog post commemorating Open Access Week 2023 (which started Monday with the motto Community over Commercialization), referencing several blog posts archived in Rogue Scholar.
This blog post is a follow-up to a post in September (Fenner 2022a), where I announced that I had started working on an archive for scholarly blog posts based on the InvenioRDM open-source repository software. In the last two months, I focussed on two activities \u2013 besides lots of physical therapy and other training following a stroke earlier this year (Fenner 2022b): helping to make it easier (and safer) to run InvenioRDM in Docker container infrastructure, and working on converting the bolognese metadata conversion Ruby gem (Fenner 2017) to Python (work in progress on GitHub) to enhance InvenioRDM functionality.
Building an archive of scholarly blog posts faces the same fundamental challenges as repositories for other types of scholarly content, whether data, software, preprints, or journal articles. You have to collect metadata and content, and that approach only scales with standardization and open licenses.
Luckily we already know a lot about required and optional but desired scholarly metadata, and they are fundamentally not different for scholarly blog posts. This means we can take similar approaches as we have for example taken for research data:
Persistent identifiers for blog posts can be DOIs, as this blog is doing since earlier this year (Fenner 2022). The main advantage of using DOIs is registering standard metadata stored independently of the blogging platform, in case the platform changes or disappears (as has happened several times in the 15 years this blog exists). While there are several blogs using DOIs for their posts, they often fail in guideline #4: the persistent identifier must be embedded in the landing page in machine-readable format. This is important so that reference managers can capture the DOI and retrieve the associated metadata.
When the metadata are embedded directly in the blog post, schema.org markup in JSON-LD format (guideline #7) is much more convenient than HTML meta tags (guideline #8), but for the time being reference managers only work with the latter. The blogging platform used for this blog (https://ghost.org/) has schema.org metadata built in, and there was only a small amount of work needed to expose all metadata needed (or desired) for DOI registration:
An issue I have seen with schema.org metadata is that sometimes they are added by a script running in the browser instead of coming from the server, and this makes metadata harvesting unreliable. Multiple versions and different levels of granularity \u2013 a major challenge when working with data and software \u2013 luckily is not a major issue with scholarly blogs, in this regard, they behave similarly to preprints and journal articles.
There are several possible approaches to building infrastructure for scholarly blog posts, and they all have well-known real-world examples:
Based on the above, what makes sense for a scholarly blog post archive?
Fenner, M. (2022a). Starting Work on the Front Matter Archive [Blog]. Front Matter. https://doi.org/10.53731/9z6rz5d-djbay0y
Fenner, M. (2022b). I spent the last five months in the hospital [Blog]. Front Matter. https://doi.org/10.53731/bkkzj8g-gd14mb6
Fenner, M. (2017). Bolognese: A Ruby library for conversion of DOI Metadata. DataCite. https://doi.org/10.5438/N138-Z3MK
Fenner, M., Crosas, M., Grethe, J. S., Kennedy, D., Hermjakob, H., Rocca-Serra, P., Durand, G., Berjon, R., Karcher, S., Martone, M., & Clark, T. (2019). A data citation roadmap for scholarly data repositories. Scientific Data, 6(1), Article. https://doi.org/10.1038/s41597-019-0031-8
Fenner, M. (2022c). DOI Registrations for all Ghost Blogs [Blog]. Front Matter. https://doi.org/10.53731/fezg09h-hgn1gzm
Fenner, M. (2009). Interview with Geoffrey Bilder [Blog]. Front Matter. https://doi.org/10.53731/r294649-6f79289-8cw1h
","doi":"https://doi.org/10.53731/br9f5xa-a556w2t","guid":"https://doi.org/10.53731/br9f5xa-a556w2t","image":"https://storage.ghost.io/c/c5/33/c533c955-b5f3-4ff1-ae2d-6b52a212e602/content/images/2022/12/guidelines-4.png","language":"en","license":"https://creativecommons.org/licenses/by/4.0/legalcode","published_at":1670803200,"reference":[{"id":"https://doi.org/10.53731/9z6rz5d-djbay0y","unstructured":"Fenner, M. (2022, September 28). Starting Work on the Front Matter Archive. Front Matter."},{"id":"https://doi.org/10.53731/bkkzj8g-gd14mb6","unstructured":"Fenner, M. (2022, July 28). I spent the last five months in the hospital. Front Matter."},{"id":"https://doi.org/10.5438/n138-z3mk","unstructured":"Fenner, M. (2017). Bolognese: a Ruby library for conversion of DOI Metadata [Computer software]. DataCite."},{"id":"https://doi.org/10.1038/s41597-019-0031-8","unstructured":"Fenner, M., Crosas, M., Grethe, J. S., Kennedy, D., Hermjakob, H., Rocca-Serra, P., Durand, G., Berjon, R., Karcher, S., Martone, M.& Clark, T. (2019). A data citation roadmap for scholarly data repositories. Scientific Data, 6(1)."},{"id":"https://doi.org/10.53731/fezg09h-hgn1gzm","unstructured":"Fenner, M. (2022, August 25). DOI Registrations for all Ghost Blogs. Front Matter."},{"id":"https://doi.org/10.53731/r294649-6f79289-8cw1h","unstructured":"Fenner, M. (2009, February 17). Author Identifiers: Interview with Geoffrey Bilder. Front Matter."}],"rid":"0mwpp-hqg82","summary":"This blog post is a follow-up to a post in September (Fenner 2022a), where I announced that I had started working on an archive for scholarly blog posts based on the InvenioRDM open-source repository software.","tags":["Rogue Scholar","Open Infrastructure"],"title":"Building an archive for scholarly blog posts","updated_at":1787210287,"url":"https://blog.front-matter.de/posts/building-an-archive-for-scholarly-blog-posts/","version":"v1"},{"authors":[{"affiliation":[{"id":"https://ror.org/008zgvp64","name":"Public Library of Science"}],"contributor_roles":[],"family":"Fenner","given":"Martin","url":"https://orcid.org/0000-0003-1419-2405"}],"blog":{"authors":[{"name":"Martin Fenner","url":"https://orcid.org/0000-0003-1419-2405"}],"community_id":"15a362ea-8138-42b8-917f-1840a92addf8","created":1672531200,"current_feed_url":null,"description":"The Front Matter Blog covers the intersection of science and technology since 2007.","doi":"https://doi.org/10.53731/front_matter","favicon":"https://rogue-scholar.org/api/communities/15a362ea-8138-42b8-917f-1840a92addf8/logo","feed_format":"application/atom+xml","feed_url":"https://blog.front-matter.de/atom","filter":null,"generator":"Ghost","home_page_url":"https://blog.front-matter.de/","issn":"2749-9952","language":"eng","license":"https://creativecommons.org/licenses/by/4.0/legalcode","prefix":"10.53731","relative_url":null,"secure":true,"slug":"front_matter","status":"active","subfield":"1710","title":"Front Matter","updated":1787218226,"use_api":true},"blog_name":"Front Matter","blog_slug":"front_matter","content_html":"The standard local file formats for bibliographic data are probably bibtex and RIS. They have been around for a long time, and are supported by all reference managers and many other tools and services. Unfortunately these formats are far from perfect:
@article{fenner2012a,\n title = {One-click science marketing},\n volume = {11},\n url = {http://dx.doi.org/10.1038/nmat3283},\n doi = {10.1038/nmat3283},\n number = {4},\n journal = {Nature Materials},\n publisher = {Nature Publishing Group},\n author = {Fenner, Martin},\n year = {2012},\n month = {mar},\n pages = {261-263}\n}One obvious solution would be to store bibliographic data in XML or JSON. These formats have very good support in all programming languages, and they are the formats used by APIs on the web. There have been some efforts to standardize these formats for bibliographic data, e.g. BibJSON, MODS, BibTeX XML or Endnote XML.
<bibtex:entry id='fenner2012a'>\n <bibtex:article>\n <bibtex:title>One-click science marketing</bibtex:title>\n <bibtex:volume>11</bibtex:volume>\n <bibtex:url>http://dx.doi.org/10.1038/nmat3283</bibtex:url>\n <bibtex:doi>10.1038/nmat3283</bibtex:doi>\n <bibtex:number>4</bibtex:number>\n <bibtex:journal>Nature Materials</bibtex:journal>\n <bibtex:publisher>Nature Publishing Group</bibtex:publisher>\n <bibtex:person>\n <bibtex:first>Martin</bibtex:first>\n <bibtex:last>Fenner</bibtex:last>\n <bibtex:person><bibtex:author/>\n <bibtex:year>2012</bibtex:year>\n <bibtex:month>mar</bibtex:month>\n <bibtex:pages>261-263</bibtex:pages>\n </bibtex:article>\n</bibtex:entry>My problem with these formats is that they are made for computers talking to each other and not humans. I personally think that a file with bibliographic data should be human-readable, similar to why I like markdown for writing scientific documents.
When you have too many standards and are not happy with any of them, you of course create a new standard.
My suggestion for a new bibliographic file format is twofold: a) use YAML for data serialization and b) use CSL as data format. YAML is a data format popular with Ruby Developers and is described on the YAML website as
YAML is a human friendly data serialization standard for all programming languages.
Something that not may people seem to know is that YAML is a superset of JSON and that every JSON file is also a valid YAML file. The main difference is the better human readability of YAML.
Citation Style Language is described on the CSL website as
CSL is an open XML-based language to describe the formatting of citations and bibliographies.
Although some commercial applications still use proprietary citation styles, CSL has become the de facto standard, and is used by the reference managers Zotero, Mendeley, Papers, and others. This blog uses CSL via Pandoc and the citeproc-hs library. CSL processors need bibliographic data in a standard format. The popular Citeproc-js Javascript CSL processor by Frank Bennett for example uses JSON, but we might as well use YAML:
- title: One-click science marketing\n volume: '11'\n URL: http://dx.doi.org/10.1038/nmat3283\n DOI: 10.1038/nmat3283\n issue: '4'\n container-title: Nature Materials\n publisher: Nature Publishing Group\n author:\n - family: Fenner\n given: Martin\n orcid: 0000-0003-1419-2405\n page: 261-263\n id: fenner2012a\n type: article-journal\n issued:\n date-parts:\n - 2012\n - 3I hope you agree that this format is not only structured and can be understood by computers, but is also very readable by humans. You may have noticed that I have inserted my ORCID, something that is very difficult to do with bibtex where all authors are stored in one text string (see above).
Careful readers of this blog will of course remember that I have written about using YAML to store metadata about a blog post. We could now add bibliographic information to these metadata, either in the YAML frontmatter (if it is a Jekyll blog), or in a separate file. It should be straightforward to adapt the existing CSL processors to understand YAML since YAML and JSON are so similar. To get started with some Citeproc YAML, use the new (and still experimental) ORCID Feed Webservice with your ORCID and specify the yml format, e.g. http://feed.labs.orcid-eu.org/0000-0003-1419-2405.yml for my publications.
Tobias Kind has submitted a very thoughtful comment (in reply to Approaches to compound documents \u2013 ORE, PDF, DOCX) which deserves printing and commenting.
\n\nTK: Hello Peter,
\nthanks for your thoughts. The more I read the more complex and frustrating it gets. I was just reading your comments about Adobe Acrobat; I would assume that everybody in the chemistry world has an Acrobat Full license. But I recognize that's not the case. Furthermore there are people who have problems opening a ZIP file, so one can not assume that everybody is operating at the same level of tools.\n
PMR: I personally do not have an Acrobat full licenCe. That's not religious \u2013 I just don't have one.\u00a0 Maybe the University has a site licence \u2013 I don't know. FWIW I send manuscripts as *.doc. I realise this is also a proprietary and yes, it has to be paid for. It's just that I happen to have it painlessly. In contrast it is possible to get Open Source/free solutions for ZIP (though only after the infamous patent ran out).
\n\nTK: That's the problem with the long tail. According to the power law it is probably safe to assume that the majority of chemists doesn't even care if there is chemical semantics lurking out of a document. Not to offend the majority of chemists, but at the end of the day it's only the number of publications on the CV that counts (well quality of course).\n
PMR: agreed
\n\nTK: Tim,
\nRSC with Project Prospect and Nature with journals that annotate structures and submit them to PubChem are probably top notch regarding semantics. And as I said before, yes PDF can include metadata with XMP, http://www.adobe.com/products/xmp/ but as long as there are no easy (free) tools out there its hard to push semantics from the PDF side. Acrobat Reader 8 did not know XMP and yes one can attach XML using the Full Acrobat. But the mentioned ExifTool is not a commodity tool for most chemists.\n
PMR: agreed. I am a pragmatist in that I can reasonably persuade chemists to include various bits of information into Word2007, but not into LaTeX or Acrobat. If everyone used semantic Acrobat instead of Word I'd probably be suggesting that. I'm pleased to see that Writer is better than Open Office, but it doesn't (I think) solve the semantic packaging problem seamlessly.
\n\nTK:\u00a0 But then again the whole semantics train currently depends on the journal itself or the editorial board, or single people or innovative groups at the publisher side. And there is certainly the tools side, so for WEB 2.0 in chemistry only a broad range of software tools can act as an enabler for chemical semantics.\n
PMR: Completely agreed. This is why the International Union of Crystallography deserves praise \u2013 it designs and requires semantic data publication for many of its journals.
\n\nTK:Peter,
\nI would not go so far as \"PDF corrupts and restricts thought\". Chemists can not make third parties responsible for the current mess in missing annotations and data exchange. Most of the better chemistry and life sciences journals allow supporting info, so what speaks against attaching the source HTML, DOC as supplement. Yes its redundant, but as long as publishers do not convert supporting data into bitmap PDF it is not a problem.\n
PMR: Like Tufte I allow myself some hyperbole. I would certainly say that \"in a digital age where many new forms of information and publication are possible, a universally used format whose primary purpose is to allow printing of documents onto paper is an active restriction on the imagination\".
\n\nPMR: As an example if you go to ACS Journal of Proteome Research, you can find some of the evil PDFs, and even the evil flat 2D PDF attachments including molecular spectra or information. But a few publications also include supplement RAW data (as XLS. MDB or ZIP) and even PDB codes. So I assume if the authors and reviewers insist on publishing meta data in the supplement in a specific format the journal would agree. Well, then there is that unholy ACS supplement data copyright. But there are also ways to submit data on personal websites. For instance you could find the ACS journal supplement data for \"T.IMPAFIFEHIIK.R\" also on google: \"Powered by Yates Bioinformatics Team; This is ongoing project with preliminary results\", ok copyrighted by the Yates group itself
PMR: If you look at ACS J.Org.Chem you will see that almost every paper has a large supplement. This is almost always in PDF. It's clearly taken a lot of work to create. The information was, originally, semantic and the publication process has encouraged the community to turn it into PDF. The spectra were JCAMPs (or could be JCAMPs), The molecules were CDX or Mol, The reactions were RXN, etc. All have been steamrollered into flat PDFs.
\nPMR: The exception are the CIFs, designed, advocated, and managed by the IUCr. They have shone as an example to the rest of the chemical world.
\n\nTK: For example some of our public US taxpayer funded metabolomics data sets are fully available via our SetupX LIMS and study design database:
\nhttp://fiehnlab.ucdavis.edu:8080/m1/
\nFor those public studies people can download all the raw data and all the annotated and result data and even the underlying software. Not all research data is open access and publicly available and yes we are also guilty of publishing flat PDFs without any semantics, but we allow people to reproduce some of our experiments and download RAW and processed data and all needed software and that can only be topped by Open NoteBook Science, the purest form of scientific reporting.\n
PMR: This again is the influence of the bioscience community. It makes me envious.
\nTK: Tobias Kind
\nfiehnlab.ucdavis.edu
It's technically trivial \u2013 yes trivial \u2013 to publish molecules and spectra. If a journal said \"no need to write 200 pages of supplemental info in PDF, just publish the *.cdx, *.mol. \".jdx.\" That's all. But where is the editorial push for this? Will any chemical editors (technical, management, academic) step up and say \"this journal will require authors to deposit semantic chemistry in \u2026 months/years\". That's all it takes. There wouldn't even be much resistance \u2013 probably rejoicing.
\nThe good news is that we have an Open Source infrastructure that can convert all of these legacy formats into semantic chemistry (Chemical Markup Language, CML) essentially automatically. We've done it for crystallography in the chemistry depertament here and the issues are not technical but things like embargoes.
\nYou don't even need to know about CML.
\nI'll be explaining in future posts how it is now conceptually simple to publish chemical data in semantic form. I'd like to work with, not against, publishers. And, with some like IUCr and RSC we do.