The .bibu file format and latex plugin, for uncertain bibliographies

[quinn-dougherty]

Estimational programming in metascience

In the status quo, authors register their beliefs about the literature with something called citations, which in latex markup looks like this:

Aeson versions preceding 2.0.1.0 are vulnerable to an attack when used to ingest untrusted input \cite{@AesonDos}.

The input to the \cite operator queries an associated biblio.bib file in the source directory. .bib entries look like the following.

@online{AesonDos,
author="Kerckhove, Tom Sydney",
title="JSON Vulnerability in Haskell's Aeson library",
url="https://cs-syd.eu/posts/2021-09-11-json-vulnerability",
year="2021",
}

If citation A makes multiple claims, then authors will \cite{@A} for each claim they're interested in.

The citation mechanism is not where the user qualifies their belief in the claim supported by the resource. Instead, the user relies on natural language to imply qualifications. Additionally, beliefs about the probability of a replication succeeding or a confidence interval around any point estimates in the resource are strictly not supported, awkwardly non-intuitive to force into the paper.

Entries in a .bibu file

The uncertain bibliography file format is a minor adjustment to the traditional .bib file format. An entry in a .bibu file would look like the following

@online{AesonDos
author="Kerckhove, Tom Sydney",
title="JSON Vulnerability in Haskell's Aeson library",
url="https://cs-syd.eu/posts/2021-09-11-json-vulnerability",
year="2021",
estimates.claim1.description = "<natural language description of claim, with page number>",
estimates.claim1.squiggle = "<squiggle expression, or empty string for binary propositions>",
estimates.claim1.credence = "<number in [0,1]>",
estimates.claim1.replicationProbability = "<number in [0,1]>"
}

In other words, for every claim that a user is drawing from a paper, they write a credence that they have in the claim, a replication probability which is their subjective forecast of if it got replicated how it would go, and for quantitative claims a squiggle expression.

Then, the latex compiler/renderer could provide inline sparkline distributions after the citation, and online resources could use the information in a ton of different ways.