Remove zodb from sage

[williamstein]

zodb only use the pickle protocol version 1. It is also not used in sage apart from one package that has been migrated away from it in #12205.

There is a zodb mailing list discussion related to the pickle protocol here: http://www.mail-archive.com/zodb-dev@zope.org/msg04628.html

In particular, pickle protocol 1 is hardcoded. But SageObjects/Cython objects must use protocol 2 in many cases:

sage: import cPickle
sage: F = factor(12)
sage: cPickle.dumps(F, protocol=0)
TypeError: can't pickle SageObject objects
sage: cPickle.dumps(F, protocol=1)
TypeError: can't pickle SageObject objects
sage: cPickle.dumps(F, protocol=2)
'\x80\x02csage.structure.factorization\nFactorization\nq\x01)\x81q\x02}q\x03(U\x12_Factorization__crq\x04\x89U\x14_Factorization__unitq\x05csage.rings.integer\nmake_integer\nq\x06U\x011\x85Rq\x07U\x18_Factorization__universeq\x08csage.rings.integer_ring\nIntegerRing\nq\t)Rq\nU\x11_Factorization__xq\x0b]q\x0c(h\x06U\x012\x85Rq\rK\x02\x86q\x0eh\x06U\x013\x85Rq\x0fK\x01\x86q\x10eub.'

See also the related ticket #10352.

The suggested option for this ticket is to remove zodb from sage.

---

• Apply attachment: trac10353-db-removal.patch to the sage library
• Apply attachment: trac10353-sage_root.patch to the sage root repository
• Remove the zodb package.

Depends on #12205
Depends on #13717
Depends on #13963

CC: @kini

Component: misc

Author: François Bissey

Reviewer: Jeroen Demeyer

Merged: sage-5.7.beta1

Issue created by migration from https://trac.sagemath.org/ticket/10353

[cschwan]

comment:1

Please do not feel offended of my naive question, but what do you think of completely removing ZODB? As far as I understand (by looking at the failing doctests when removing ZODB), only Cremona's database and the Conway polynomials database use it. Ticket #11587 migrates the Cremona's DB to SQLite, so would it not be possible to the same with the Polynomial DB? Are there any optional SPKGs which do depend on ZODB?

[williamstein]

comment:2

I added ZODB to Sage (long ago), and I am definitely +1 to removing it from Sage, if we can figure out how. I think it is a technology whose value (for Sage!) has come and gone. One can do better with SQLite these days.

[kiwifb]

comment:3

OK it is really a pain on sage-on-gentoo side and will become worse in a few months (removal from the main tree, zope maintenance is apparently hard that it will be put in its own repo for people who are interested in it).

So I am putting my hand up to remove zodb from sage. Anything else apart from conway polynomial database needs to be cleaned and converted to sqlite?

[kiwifb]

comment:5

It looks like stein-watkins-ecdb also use zodb.

[kiwifb]

comment:6

Hum it looks like sage/databases/stein-watkins-ecdb.py imports sage.databases.db but it is not actually using any of it as far as I can see. So it will be a simple patch to get rid of db.py and compressed_storage.py.

[jpflori]

comment:7

If I'm not wrong, sagenb uses some Python packages which are currently included in zodb spkg (I think zope.interface or stg like that, see #10352).

[kiwifb]

comment:8

If I remember the discussion correctly the dependency on zope.interface will be included in a future version of sagenb. My quick check on the current version of sagenb show no import from zope.
It was suggested that it should be included in the sagenb bundle, I personally agree with the idea. In any case that would mean replacing the current zodb spkg with a new one cotaining only zope.interface if we want to keep it in main line.

[kiwifb]

Attachment: trac10353-db-removal.patch.gz

this patch remove reference to zodb from the sage library

[kiwifb]

comment:9

I just attached a patch to remove zodb references from the sage library. I haven't done the patch for sage-root yet.

[kiwifb]

Description changed:

--- 
+++ 
@@ -1,16 +1,4 @@
-
-```
-wstein@sage:~$ /usr/local/bin/sage
-----------------------------------------------------------------------
-| Sage Version 4.6, Release Date: 2010-10-30                         |
-| Type notebook() for the GUI, and license() for information.        |
-----------------------------------------------------------------------
-sage: d = sage.databases.db.Database('sage_zodb', read_only=False)
-sage: d[12] = factor(12)
-sage: d.commit()
-...
-TypeError: can't pickle SageObject objects
-```
+zodb only use the pickle protocol version 1. It is also not used in sage apart from one package that has been migrated away from it in #12205.
 
 There is a zodb mailing list discussion related to the pickle protocol here:  http://www.mail-archive.com/zodb-dev@zope.org/msg04628.html
 
@@ -27,16 +15,11 @@
 '\x80\x02csage.structure.factorization\nFactorization\nq\x01)\x81q\x02}q\x03(U\x12_Factorization__crq\x04\x89U\x14_Factorization__unitq\x05csage.rings.integer\nmake_integer\nq\x06U\x011\x85Rq\x07U\x18_Factorization__universeq\x08csage.rings.integer_ring\nIntegerRing\nq\t)Rq\nU\x11_Factorization__xq\x0b]q\x0c(h\x06U\x012\x85Rq\rK\x02\x86q\x0eh\x06U\x013\x85Rq\x0fK\x01\x86q\x10eub.'
 ```
 
-There is some code in Sage that uses ZODB (e.g, Cremona database), but I think it uses only pure python objects, so the above isn't a problem. 
-
 See also the related ticket #10352.
 
-Some ideas for how to fix this:
+The suggested option for this ticket is to remove zodb from sage.
 
-- Modify ZODB itself, e.g., by changing: ZODB.serialiize.ObjectWriter as suggested here: http://www.mail-archive.com/zodb-dev@zope.org/msg04766.html
- 
-- Figure out how to monkeypatch ODB.serialiize.ObjectWriter at runtime, then document this hack
+---
+* Apply [attachment: trac10353-db-removal.patch](https://github.com/sagemath/sage-prod/files/10651525/trac10353-db-removal.patch.gz) to the sage library
+* Apply [attachment: trac10353-sage_root.patch](https://github.com/sagemath/sage-prod/files/10651526/trac10353-sage_root.patch.gz) to the sage root repository
 
-- "Fix" SageObject and/or Cython, so Sage objects can serialize using Pickle protocol 1 instead of requiring 2.  I don't personally understand exactly why SageObject can't be pickled using protocol 1. 
-
-

[kiwifb]

Author: François Bissey