[feature] Allow eXist-db to start but log an error if the database contains a bad package

[djbpitt]

Is your feature request related to a problem? Please describe.

I installed a package in eXist-db and then deleted expath-pkg.xml from the local filesystem, and the deletion was synched to the running database because I had VSCode synchronization enabled. The deletion was deliberate (I was working on a script that can generate expath-pkg.xml), but I hadn't anticipated that if the deletion wound up getting synched to the database it would make it impossible to restart eXist-db, which seems to be the case. The missing file seems to prevent a launch:

Caused by: org.expath.pkg.repo.Storage$NotExistException: File '/Users/djb/Library/Application Support/org.exist/expathrepo/init/expath-pkg.xml' does not exist
	at org.expath.pkg.repo.FileSystemStorage$FileSystemResolver.resolveWithin(FileSystemStorage.java:300) ~[pkg-java-1.6.1.jar:1.6.1]

Describe the solution you'd like
If an installed package is damaged I would like eXist-db to log an error but to permit the database to start.

Describe alternatives you've considered
I am experimenting now (with @adamretter 's help on the eXist-db Slack) in adjusting the contents of expathrepo manually.

Additional context
The problem is present in 6.0.1 and 6.1.0. I'm running on MacOS Ventura 13.1 with Java JDK 1.17.

[line-o]

refs #4686

[line-o]

FYI: I am currently refactoring this part of exist-db but it will include breaking changes.

[djbpitt]

@line-o Is this a appropriate time to ask what is likely to break as a side-effect of the refactoring? If not, I'll just watch for and read the PR when it's ready.

[line-o]

Only error codes, messages and exception classes.

[line-o]

From the error message it seems like you deleted the file in the filesystem and not the database. I do not assume deleting the file in the database will be reflected in the filesystem. This needs further testing.

@djbpitt Is it possible that you removed the file from the filesystem in addition to removing it from the database?

[djbpitt]

@line-o I think I probably (sorry for the vagueness; it was both confusing and a few weeks ago) deleted it only from the file system explicitly, but I was using the automatic VSCode sync, which I think meant that a deletion in the file system was automatically mirrored by a deletion inside the database. Is that hypothesis consistent with the effect?

[line-o]

existdb keeps a copy of installed applications metadata in a separate location on the file system whenever it is deployed.
The location given in the error message hints at that location .../expathrepo/init/....
This is why I would like to know if it is possible you removed it there (instead of the database).
If not, it indicates that existdb synchronizes the contents of its copy with what is stored in the database collection.

[djbpitt]

@line-o I don't think I knowingly interacted with that directory because I didn't know about its existence before @adamretter sent me to look there to fix the problem, and I can't imagine how I might have touched it accidentally. I can't rule it out, though. I had been working on an app configuration, and as part of that work I would install and uninstall, through the package manager, repeatedly, so I can imagine that I may unwittingly have broken something during that experimentation.

I wonder whether there might not be two intersecting but fundamentally independent issues: 1) is there a problem in eXist-db (as opposed to clumsy user error) that allowed an unstable condition to arise? and 2) should eXist-db have been prevented from launching one the instability--whatever the cause--had arisen? The feature request I opened was about #2: even if it should not normally happen, does eXist-db have to respond to it by refusing to launch? I think you're now asking about #1, and I'm curious about that, too, and I understand that it may involve misbehavior on the eXist-db end or something I perpetrated inadvertently. I didn't knowingly interact with the directory you mention, but I can't be confident that I didn't mess it up without realizing it.