Synopsys
This study describes a flaw in article metadata, where a reference contains a functional DOI that resolves to a different paper than the one cited. Unlike simple typos, these "internally contradictory" errors are often asserted by publishers, revealing a systemic failure in current metadata verification systems.
Introduction
Metadata integrity is the backbone of modern science, ensuring proper attribution of citations and accuracy of the scholarly record. The integrity of metadata of published articles is often taken for granted in a largely trust-based and progressively more automated publishing system. Potential errors can lead to misattribution of citation counts (Kriváchy, 2025), corruption of the bibliographic record, and compromise the traceability of references. Lack of appropriate oversight, especially connected with the sudden adaptation of generative AI into academic workflows, is likely to trigger problems the community will have to face sooner or later. While the knowledge about the “hallucinated references” created by large-language models is relatively widespread, in this work, a new and common type of error affecting article metadata is described.
Research Questions
The objective was to find and flag articles with possibly hallucinated references automatically using the metadata of the article, based on inconsistencies of the data provided within references. Moreover, the scope of the problem was investigated, and the extent to which some of the major publishers submit internally contradictory metadata to central repositories was estimated.
Results
A new type of error is reported here, consisting of a mismatch between the DOI and multiple other data fields provided within the reference field (for example, first author, article title, or date of publication). Unlike a typical typo or use of unsupported characters in a suffix of the DOI that results in it being unresolvable (Boente et al., 2021), these errors occur with a resolvable identifier, resolving to a different article. Results of the automated screening were manually confirmed in hundreds of cases obtained from several major publishers, cross-referencing the metadata obtained from resolving the DOI with the rest of the data present in the references. In the majority of cases, the errors within the metadata are found as well in the content of the published article. Notably, most of the affected references are claimed to be asserted by the publisher, showing possible inadequacy of the current verification mechanisms. Moreover, although the issue is present across multiple publishers, the systematic variability of metadata completeness submitted by different publishers - where some publishers omit key descriptive fields - makes the detection of errors difficult and sometimes impossible when using the metadata. The variability in ‘density’ of metadata creates challenges for automated methods of detection of errors. Many of the cases were documented on PubPeer to ensure transparency, and an openly-available database of publications affected by this problem is being curated.
Conclusion
This work demonstrates gaps in current metadata verification systems and risks they pose to research integrity, calling for urgent accountability of publishers for the data they deposit. Establishment and enforcement of criteria of metadata completeness and accuracy are paramount to the integrity of science and the quantitative metrics used in research evaluation.
Keywords
Metadata integrity, DOI mismatch, Crossref, research evaluation, PubPeer, institutional accountability, hallucinations, Gen-AI.
References:
Boente, R., Massari, A., Santini, C., & Tural, D. (2021, May 3). Cleaning different types of DOI errors found in cited references on Crossref using automated methods. OpenAIRE - Explore. https://explore.openaire.eu/search/publication?pid=10.5281%2Fzenodo.4734512
Kriváchy, T. (2025, November 3). Incorrect Citation Association for Articles in Online-Only Springer Nature Journals. arXiv.Org. https://arxiv.org/abs/2511.01675v2
PubPeer Foundation. (2026). PubPeer: The online journal club. https://pubpeer.com