Sqlitedbstore

[ghost]

Add SQLite implementation of an RDFLib Store to provide native Python persistence. Existing API is respected.

(ftr: motivated by ncar's assertion)

• Checked that there aren't other open pull requests for the same change.
• Added tests for any changes that have a runtime impact.
• Checked that all tests and type checking passes.
• For changes that have a potential impact on users of this project:
  • Will update relevant documentation to avoid inaccuracies if PR is accepted.
  • Will add additional documentation if PR is accepted.
  • Will add an example in ./examples for new features if PR is accepted.
  • Will update changelog (CHANGELOG.md) if PR is accepted.
• Granted push permissions to the PR branch,
  so maintainers can fix minor issues and keep the PR up to date.

[coveralls]

Coverage: 90.731% (-0.1%) from 90.85% when pulling 8e3f604 on gjhiggins:sqlitedbstore into fec7f0a on RDFLib:main.

[aucampia]

@mwatts15 @adamhadani it would be good to get your feedback on this since you maintained https://github.com/RDFLib/rdflib-sqlalchemy and the functionality here overlaps slightly with that.

I would be open to integrating https://github.com/RDFLib/rdflib-sqlalchemy into RDFLib, provided it is done with an optional extra.

But even if we do that, it may still be good to have something that works with a more or less stock python's sqlite3.

CC: @RDFLib/core-reviewers

[ghost]

@mwatts15 @adamhadani it would be good to get your feedback on this since you maintained https://github.com/RDFLib/rdflib-sqlalchemy and the functionality here overlaps slightly with that.

Only in the general terms of providing a back-end persistence for RDFLib graphs.

Other than implementing the Store API, there's little in common between rdflib-sqlalchemy's approach and this, based as it is on a key-value model for SQLite that then uses lightly-adapted version of Drew Pertulla's Tokyo Cabinet adaptation of the SleepyCat key-value RDFLib Store implementation (described in more detail in the aged-but-still-pertinent RDFExtras docs).

In contrast, rdflib-sqlalchemy is a SQL-based approach that uses AbstractSQLStore, a subsequent distillation of Chimezie's original 2006 FOPLRelationalModel (again described in the RDFExtras docs)

I would be open to integrating https://github.com/RDFLib/rdflib-sqlalchemy into RDFLib, provided it is done with an optional extra.

Ah, rdflib-sqlalchemy is pinned to the 1.X versions of SQLAlchemy (SQLAlchemy>=1.1.4,<2.0.0) and would need a rewrite to be able to work with SQLAchemy 2.0.X versions - which I imagine would be required before integrating into RDFLib, even as an optional extra.

Adam Ever-Hadani is now Senior VP, Ai & Product at Globality and hasn't been involved in rdflib-sqlalchemy since 2016. Mark is also in the process of disengaging: “Currently, rdflib-sqlalchemy is in maintenance mode. That means the current maintainer (@mwatts15) will do what he can to keep the package working for existing use-cases, but new features will not be added and newer versions of SQLAlchemy will not be supported”

But even if we do that, it may still be good to have something that works with a more or less stock python's sqlite3.

A Python-native RDFLib Store has been on the wishlist for several years, this PR is informed and motivated by the subsequent discussion for the extant PR that has a key-value implementation based on shelve but is pending improvements in the speed of key-handling.

[ghost]

On reflection, I realise that I have failed to recognise the degree of maintenance burden that RDFLib currently imposes and this PR could instead be fairly trivially published separately as a standard RDFLIb plugin.

So I'm minded to close this PR and republish it elsewhere - comments?

[aucampia]

On reflection, I realise that I have failed to recognise the degree of maintenance burden that RDFLib currently imposes and this PR could instead be fairly trivially published separately as a standard RDFLIb plugin.

So I'm minded to close this PR and republish it elsewhere - comments?

I don't think you should, I will review this as soon as I have time (probably next week) and I don't think this is unreasonable to include, I think it is quite essential behaviour.

[aucampia]

It makes at least as much sense as BerkeleyDB, and much easier to use for users.

[ghost]

I will review this as soon as I have time (probably next week)

Okay and, importantly --- as and when it is convenient for you --- a native-Python RDFLib Store has been on the wishlist for years, there is 0 urgency here, this can sit for a while.

(And apologies for the lack of prior discussion)

[aucampia]

PRs to V6 is closed until further notice. See this for more details:

• RDFLib V6 and V7 #2395

[aucampia]

PRs to V6 is closed until further notice. See this for more details:

• RDFLib V6 and V7 #2395

We will be open for PRs again once this is resolved:

• docs: add guidelines for breaking changes #2402

[aucampia]

@gjhiggins you should not close this, if you are unsure better to keep it in draft, I think we need this in, I just have not had time to look at it yet.

[ghost]

The closure is just a byproduct of some local admin/re-org