AttributeError: 'Transaction' object has no attribute 'extension_bytes' [was: Tests in TravisCI are broken] #35

icemac · 2020-06-26T06:31:43Z

All jobs besides the OLDZODB one are broken, e. g. https://travis-ci.org/github/zopefoundation/zodbupdate/jobs/701272511

There are multiple test failures which all look like this:

Traceback (most recent call last):
264  File "/opt/python/3.8.0/lib/python3.8/unittest/case.py", line 60, in testPartExecutor
265    yield
266  File "/opt/python/3.8.0/lib/python3.8/unittest/case.py", line 676, in run
267    self._callTestMethod(testMethod)
268  File "/opt/python/3.8.0/lib/python3.8/unittest/case.py", line 633, in _callTestMethod
269    method()
270  File "/home/travis/build/zopefoundation/zodbupdate/src/zodbupdate/tests.py", line 871, in test_convert_attribute_to_bytes
271    self.update(convert_py3=True, default_decoders={
272  File "/home/travis/build/zopefoundation/zodbupdate/src/zodbupdate/tests.py", line 168, in update
273    updater()
274  File "/home/travis/build/zopefoundation/zodbupdate/src/zodbupdate/update.py", line 78, in __call__
275    t = self.__new_transaction()
276  File "/home/travis/build/zopefoundation/zodbupdate/src/zodbupdate/update.py", line 59, in __new_transaction
277    self.storage.tpc_begin(t)
278  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/ZODB/BaseStorage.py", line 193, in tpc_begin
279    ext = transaction.extension_bytes
280AttributeError: 'Transaction' object has no attribute 'extension_bytes'

Do we use a too new or too old version of transaction?
The tests break locally in the same way.
There is even no extension_bytes in the transaction package.
Am I missing something here?

The text was updated successfully, but these errors were encountered:

jamadden · 2020-06-26T15:04:02Z

Storage instances in ZODB expect their tpc_* methods to get passed an instance of ZODB.Connection.TransactionMetaData (See IStorage.tpc_begin for one.)

In the past, one could get away with passing a transaction.Transaction object instead, because the interfaces were similar, but as-of zopefoundation/ZODB#207, released in ZODB 5.6.0, that's no longer the case. The __new_transaction method probably needs to change to accommodate this.

zodbupdate/src/zodbupdate/update.py

Lines 57 to 61 in cf93479

    
           def __new_transaction(self): 
        
               t = transaction.Transaction() 
        
               self.storage.tpc_begin(t) 
        
               t.note(six.u('Updated factory references using `zodbupdate`.')) 
        
               return t

1letter · 2020-07-16T08:24:52Z

With ZODB-5.6.0: Today, i have this error while i run a migration:
The zodbverify show me zero errors, but the zodbupdate throw an error:

An error occured
Traceback (most recent call last):
  File "/opt/mig-to-p5/py3/eggs/zodbupdate-1.4-py3.7.egg/zodbupdate/main.py", line 221, in main
    updater()
  File "/opt/mig-to-p5/py3/eggs/zodbupdate-1.4-py3.7.egg/zodbupdate/update.py", line 76, in __call__
    t = self.__new_transaction()
  File "/opt/mig-to-p5/py3/eggs/zodbupdate-1.4-py3.7.egg/zodbupdate/update.py", line 57, in __new_transaction
    self.storage.tpc_begin(t)
  File "/opt/mig-to-p5/py3/eggs/ZODB-5.6.0-py3.7.egg/ZODB/BaseStorage.py", line 193, in tpc_begin
    ext = transaction.extension_bytes
AttributeError: 'Transaction' object has no attribute 'extension_bytes'
Stopped processing, due to: 'Transaction' object has no attribute 'extension_bytes'
Traceback (most recent call last):
  File "/opt/mig-to-p5/py3/eggs/zodbupdate-1.4-py3.7.egg/zodbupdate/main.py", line 221, in main
    updater()
  File "/opt/mig-to-p5/py3/eggs/zodbupdate-1.4-py3.7.egg/zodbupdate/update.py", line 76, in __call__
    t = self.__new_transaction()
  File "/opt/mig-to-p5/py3/eggs/zodbupdate-1.4-py3.7.egg/zodbupdate/update.py", line 57, in __new_transaction
    self.storage.tpc_begin(t)
  File "/opt/mig-to-p5/py3/eggs/ZODB-5.6.0-py3.7.egg/ZODB/BaseStorage.py", line 193, in tpc_begin
    ext = transaction.extension_bytes
AttributeError: 'Transaction' object has no attribute 'extension_bytes'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./py3/bin/zodbupdate", line 252, in <module>
    sys.exit(zodbupdate.main.main())
  File "/opt/mig-to-p5/py3/eggs/zodbupdate-1.4-py3.7.egg/zodbupdate/main.py", line 225, in main
    raise AssertionError()
AssertionError

jamadden · 2020-07-16T09:54:46Z

Added the bug label because it seems end users are affected too.

@1letter the workaround would be to use ZODB 5.5 to do the update.

1letter · 2020-07-16T10:02:49Z

I patch ZODB locally, but i don't know what are sideeffect happens. but the migration runs.

# BaseStorage.py

def tpc_begin(self, transaction, tid=None, status=' '):
        if self._is_read_only:
            raise POSException.ReadOnlyError()

        with self._lock:
            if self._transaction is transaction:
                raise POSException.StorageTransactionError(
                    "Duplicate tpc_begin calls for same transaction")

        self._commit_lock.acquire()

        with self._lock:
            self._transaction = transaction
            self._clear_temp()

            user = transaction.user
            desc = transaction.description
            # ext = transaction.extension_bytes
            
            # my patch begin
            try:
                ext = transaction.extension_bytes
            except:
                ext = b''
            # my patch end

            self._ude = user, desc, ext

            if tid is None:
                now = time.time()
                t = TimeStamp(*(time.gmtime(now)[:5] + (now % 60,)))
                self._ts = t = t.laterThan(self._ts)
                self._tid = t.raw()
            else:
                self._ts = TimeStamp(tid)
                self._tid = tid

            del self._resolved[:]
            self._tstatus = status
            self._begin(self._tid, user, desc, ext)

jamadden · 2020-07-16T10:29:22Z

Please don't use that patch if you have any data you care about in transaction extension data. That includes things like the username and note/description associated with the transaction. The note is commonly used to hold the URL of a web request, for example.

1letter · 2020-07-16T13:50:39Z

Yes, with ZODB 5.5 can i migrate. @jamadden 👍 Thanks for advice.

mauritsvanrees · 2020-07-24T13:06:26Z

I have changed the title to contain the error message, instead of only that tests are broken.

I am seeing the same. Python 2, Plone 5.2.2-pending. An earlier version of this pending release used Zope 4.4.4, with ZODB 5.5.1. There it works. But latest pending release has Zope 4.5 with ZODB 5.6.0 and there I have the error.

$ bin/zodbupdate --pack -f var/filestorage/Data.fs
Loaded 3 rename rules from zest.zodbupdate.renames:renames
Loaded 1 rename rules from OFS:renames
An error occured
Traceback (most recent call last):
  File "/Users/maurits/shared-eggs/cp27m/zodbupdate-1.4-py2.7.egg/zodbupdate/main.py", line 221, in main
    updater()
  File "/Users/maurits/shared-eggs/cp27m/zodbupdate-1.4-py2.7.egg/zodbupdate/update.py", line 76, in __call__
    t = self.__new_transaction()
  File "/Users/maurits/shared-eggs/cp27m/zodbupdate-1.4-py2.7.egg/zodbupdate/update.py", line 57, in __new_transaction
    self.storage.tpc_begin(t)
  File "/Users/maurits/shared-eggs/cp27m/ZODB-5.6.0-py2.7.egg/ZODB/BaseStorage.py", line 193, in tpc_begin
    ext = transaction.extension_bytes
AttributeError: 'Transaction' object has no attribute 'extension_bytes'
Stopped processing, due to: 'Transaction' object has no attribute 'extension_bytes'
Traceback (most recent call last):
  File "bin/zodbupdate", line 317, in <module>
    sys.exit(zodbupdate.main.main())
  File "/Users/maurits/shared-eggs/cp27m/zodbupdate-1.4-py2.7.egg/zodbupdate/main.py", line 225, in main
    raise AssertionError()
AssertionError

dataflake · 2020-07-24T13:10:29Z

@mauritsvanrees I'm touching this issue with zc.zrs as well. @jamadden posted this code snippet earlier:

zodbupdate/src/zodbupdate/update.py

Lines 57 to 61 in cf93479

    
           def __new_transaction(self): 
        
               t = transaction.Transaction() 
        
               self.storage.tpc_begin(t) 
        
               t.note(six.u('Updated factory references using `zodbupdate`.')) 
        
               return t

You may be able to fix this by substituting Transaction with TransactionMetaData from ZODB.Connection.

mauritsvanrees · 2020-07-24T16:09:01Z

I don't know enough about this to know if the above change from Transaction to TransactionMetaData is good. But I confirm it fixes the error.

dataflake · 2020-07-24T16:15:41Z

Unfortunately I don't have enough internal ZODB knowledge, I cannot judge if that's a 100% safe solution.

Fixes #35 This applies the fix from #35 (comment)

jamadden added the bug label Jul 16, 2020

mauritsvanrees changed the title ~~Tests in TravisCI are broken~~ AttributeError: 'Transaction' object has no attribute 'extension_bytes' [was: Tests in TravisCI are broken] Jul 24, 2020

mauritsvanrees added a commit that referenced this issue Jul 24, 2020

Fixed incompatibility with ZODB 5.6.

ef639de

Fixes #35 This applies the fix from #35 (comment)

mauritsvanrees mentioned this issue Jul 24, 2020

Fixed incompatibility with ZODB 5.6. #37

Merged

mauritsvanrees closed this as completed in #37 Jul 27, 2020

jamadden mentioned this issue Aug 5, 2020

Tests broken with ZODB 5.6 zopefoundation/zc.zodbdgc#14

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AttributeError: 'Transaction' object has no attribute 'extension_bytes' [was: Tests in TravisCI are broken] #35

AttributeError: 'Transaction' object has no attribute 'extension_bytes' [was: Tests in TravisCI are broken] #35

icemac commented Jun 26, 2020

jamadden commented Jun 26, 2020

1letter commented Jul 16, 2020 •

edited

Loading

jamadden commented Jul 16, 2020

1letter commented Jul 16, 2020 •

edited

Loading

jamadden commented Jul 16, 2020

1letter commented Jul 16, 2020

mauritsvanrees commented Jul 24, 2020

dataflake commented Jul 24, 2020

mauritsvanrees commented Jul 24, 2020

dataflake commented Jul 24, 2020

AttributeError: 'Transaction' object has no attribute 'extension_bytes' [was: Tests in TravisCI are broken] #35

AttributeError: 'Transaction' object has no attribute 'extension_bytes' [was: Tests in TravisCI are broken] #35

Comments

icemac commented Jun 26, 2020

jamadden commented Jun 26, 2020

1letter commented Jul 16, 2020 • edited Loading

jamadden commented Jul 16, 2020

1letter commented Jul 16, 2020 • edited Loading

jamadden commented Jul 16, 2020

1letter commented Jul 16, 2020

mauritsvanrees commented Jul 24, 2020

dataflake commented Jul 24, 2020

mauritsvanrees commented Jul 24, 2020

dataflake commented Jul 24, 2020

1letter commented Jul 16, 2020 •

edited

Loading

1letter commented Jul 16, 2020 •

edited

Loading