Missing object imports in duplicate RPSL keys in the same source

[mxsasha]

IRRDv4 is built on the assumption that the primary key is unique in each source. However, this is not true, and causes IRRD to miss objects.

The scale of this is unknown, but this has been the case since before 4.0.0, so likely not too widespread. It also has not come up in the many v3/v4 consistency checks.

One case is AS-DONNATRANS, which is both a mntner and as-set in RIPE. The import of one probably overwrites the other. Regardless of whether this is a good way to name objects, or should or should not be valid, it is a thing people do, so should be supported.

Probably the right fix is for IRRd to treat object class, RPSL pk, and source as unique together.

[job]

Your idea how to fix it makes sense to me

[mxsasha]

There's probably still an issue with deletions:

irrd/irrd/storage/database_handler.py

Lines 381 to 382 in e8ecaa3

	stmt = table.delete(
	sa.and_(table.c.rpsl_pk == rpsl_pk, table.c.source == source),