cli: fixes replace method

[ioannistsanaktsidis]

• Closes fixtures: files and insert-or-replace mode #2039.

Signed-off-by: Ioannis Tsanaktsidis [email protected]

[ioannistsanaktsidis]

@tiborsimko please check and merge if everything is ok. I think the problem is fixed now .

[tiborsimko]

@ioannistsanaktsidis I think this would not update the files, i.e. the scenario described in the issue would not be covered by this PR. See also my comment #2039 (comment)

[ioannistsanaktsidis]

@tiborsimko I updated with @pamfilos the PR. You can have a look now. I think it covers all our cases.

[tiborsimko]

Does not seem to work fully for me, here's one use case:

# clean instance:
./scripts/clean-instance.sh
./scripts/populate-instance.sh --skip-records
# 1) upload a record without files:
cernopendata fixtures records -f ./cernopendata/modules/fixtures/data/records/cms-tools-vm-image.json --mode insert --skip-files
firefox http://0.0.0.0:5000/record/249
# ... file names not visible, which is good
curl http://0.0.0.0:5000/record/249/files/cms-user-data.txt
# ... gives "Page not found", which is good
# 2) reupload the same record with files:
cernopendata fixtures records -f ./cernopendata/modules/fixtures/data/records/cms-tools-vm-image.json --mode replace
firefox http://0.0.0.0:5000/record/249
# ... file names are now visible, which is good
curl http://0.0.0.0:5000/record/249/files/cms-user-data.txt
# ... but still returns "Page not found", which is bad
# 3) reupload the same record once more:
cernopendata fixtures records -f ./cernopendata/modules/fixtures/data/records/cms-tools-vm-image.json --mode replace
# ... gives exception sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (psycopg2.IntegrityError) duplicate key value violates unique constraint "uq_files_files_uri"
DETAIL:  Key (uri)=(root://eospublic.cern.ch//eos/opendata/cms/environment/2010/CMS-OpenData-1.0.0-rc7.ova) already exists.
 [SQL: 'INSERT INTO files_files (created, updated, id, uri, storage_class, size, checksum, readable, writable, last_check_at, last_check) VALUES (%(created)s, %(updated)s, %(id)s, %(uri)s, %(storage_class)s, %(size)s, %(checksum)s, %(readable)s, %(writable)s, %(last_check_at)s, %(last_check)s)'] [parameters: {'updated': datetime.datetime(2017, 12, 14, 1, 56, 51, 400905), 'created': datetime.datetime(2017, 12, 14, 1, 56, 51, 400898), 'checksum': u'sha1:3a5d1ca5041a83a0db1c835b6e7af18d46e2ceb0', 'last_check_at': None, 'readable': True, 'uri': u'root://eospublic.cern.ch//eos/opendata/cms/environment/2010/CMS-OpenData-1.0.0-rc7.ova', 'last_check': True, 'writable': False, 'storage_class': 'S', 'id': UUID('c4b1b2a4-3fdd-4b6e-9e18-4480118c2fbc'), 'size': 9159569}]

[tiborsimko]

Works well for the common use cases. I'll create a someday ticket for one corner case observation.