[New metadata element]: mapping_set_confidence

[cthoyt]

Element id (e.g. creator_id, mapping_tool_version):

mapping_set_confidence

Value data type (e.g. URI, URL, text, xsd:boolean):

xsd:float

Description

An optional mapping-set level confidence, assigned by the creator/producer of the set, based on whatever metric they deem fitting. This is different than an individual mapping confidence, such that it can be conveyed what's the overall confidence from the mapping provider in the correctness of the assertions.

Some examples:

• If an SSSOM file contains only human curations, then this could be the average of all confidences. Individual mappings can then have more granular confidence values, given by the curator of each row.
• If an SSSOM file contains lexical generations, then this annotation can correspond to a benchmarked confidence (i.e., accuracy) of the mapping tool. Individual mappings can have more granular confidence values, given by the tool for each generation

Complete example to a SSSOM file with this element
(This example can be given as a markdown table or a linked SSSOM file, feel free to edit the markdown table below)

# curie_map:
#   HP: http://purl.obolibrary.org/obo/FBbt_
#   MP: http://purl.obolibrary.org/obo/UBERON_
#   owl: http://www.w3.org/2002/07/owl#
#   rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#
#   rdfs: http://www.w3.org/2000/01/rdf-schema#
#   semapv: https://w3id.org/semapv/vocab/
#   skos: http://www.w3.org/2004/02/skos/core#
#   sssom: https://w3id.org/sssom/
# license: https://w3id.org/sssom/license/unspecified
# mapping_set_id: https://w3id.org/sssom/mappings/ac9e1878-73f4-4767-8402-a6c40e1b0835
# mapping_set_confidence: 0.8

subject_id	predicate_id	object_id	mapping_justification	element_id_example
HP:0009124	skos:exactMatch	MP:0000003	semapv:LexicalMatching	YOUR EXAMPLE VALUE
HP:0008551	skos:exactMatch	MP:0000018	semapv:LexicalMatching	YOUR EXAMPLE VALUE

[matentzn]

I think this is a good addition to the general confidence model I am trying to document in #338. However, personally, I would not want to tie the value to a specific formula (like average mapping confidence).

I think if I say:

a, exact, b, 0.5
c, exact, d, 0.2

We can absolutely say "I trust the mapping set 100%" as the confidence value is sort of "inherently part of the mapping".

I would have preferred this property to mean: "I trust this mapping set X % overall", which encompasses all the information it provides. This way you can express, for example: "I used this tool, and I trust it X%" in this value.

I do see the value of an aggregated confidence score as well, but I would be hesitant to add two mapping set level confidence values to the sssom data model..