This package contains tools for performing batch imports to the Vertica database.
pyvertica.connection- Module which contains logic for connection to a Vertica DB.
pyvertica.batch- High speed loader for Vertica.
pyvertica.importer- Base-class for writing Vertica batch importers.
pyvertica.migrate- Module managing export from one Vertica cluster to another.
vertica_batch_import- Command-line interface for the
pyvertica.batchmodule. vertica_migrate- Command-line interface for the
pyvertica.migratemodule.
pyvertica can be installed by executing pip install pyvertica.
- CHANGE Add argument pointing to a credential file
for
scripts/vertica_batch_import(to not expose credential in bash history).
- UPDATE
TaskThreadrequred version to 1.3 or greater.
- FEATURE
VerticaBatchwill not open a new thread or fifo each time a new batch is started ifmulti_batchis set toTrue. This will reduce the overhead each time the batch is committed, provided the user wants to callcommitmore than once on a single batch.
- ADD:
insert_listsmethod toVerticaBatch. This method takes multiple lists, converts them to rows, and invokes a single insert rather than one insert per row.
- ADD:
connectionparatmeter toVerticaBatchto allow usage of an existing connection.
- Use latest pyodbc version.
- ADD:
rolllbackmethod toVerticaBatchobject. - ADD:
REJECTMAXoption. - CHANGE:
VerticaBatch.get_errorsnow returns the number of errors instead of aboolindicating if there are errors. - CHANGE:
REJECTEDFILEoption is now set to__debug__.
- CHANGE: Make it possible to pass host, and credentials to the
vertica_migratetool by using an config file (to not expose credentials on the command-line).
- CHANGE: Make it more easy to pass more arguments to
get_connectionthroughBaseImporterandVerticaBatch.Note that this is a backwards incompatible change as the arguments forBaseImporter,VerticaBatchandget_connectionhave changed.
- FEATURE: Add a migration module
pyvertica.migrate``and script ``vertica_migrate, to move data between clusters. - CHANGE: Make the reconnect optional in
VerticaBatch.get_connection
- CHANGE: Make the execution of
ANALYZE_CONSTRAINTSoptional (executed when getting the errors).
- FIX: Handle exceptions raised inside the
QueryThreadso that the semaphore always gets released.
- FIX: Line-endings in file-object returned by
get_errors. All is now\n.
- CHANGE: The Batch-history table is now configurable in
BaseImporter. - CHANGE:
get_connectionselects a random node from the cluster and returns a connection to that node. - FIX: Cleanup fifo + temporary directory.
- CHANGE: Change the way in how we detect if the FIFO object is consumed
by the QueryThread. By opening the FIFO object (for writing) in
'w'mode after the QueryThread was created, it will block until the COPY SQL statement is started.
- Public release!