Releases: google/dwh-migration-tools
Releases · google/dwh-migration-tools
v1.0.62
What's Changed
- More precise .gitignore rules by @misolt in #575
- [b/368572924] Fix: missing data of public synonym objects by @misolt in #558
Full Changelog: v1.0.61...v1.0.62
v1.0.61
What's Changed
- [b/359499526] Dump raw Ranger API entries to output jsonl files. by @aleofreddi in #518
- [b/364818924] Remove all 'owner' filters from stats SQLs by @misolt in #553
- Provide a more descriptive error message when driver JAR does not exist by @misolt in #555
- Extract Cloudera metaconnector in-dump paths to SPI. by @damian-compilerworks in #554
- [b/355196176] Implement fail fast strategy for Teradata logs tables by @vladislav-sidorovich in #557
- [b/370428253] Rename 'Dbid' to 'DbId' by @misolt in #563
- Replace boolean flag with enum by @vladislav-sidorovich in #562
- [b/369075732] Add Teradata tables validation for additional command line options by @vladislav-sidorovich in #561
- Bump org.yaml:snakeyaml from 2.2 to 2.3 by @dependabot in #551
- update gradle lockfiles to fix snakeyaml by @vladislav-sidorovich in #564
- Update compliance information with license links by @misolt in #570
- Move license overrides to a util class by @misolt in #572
New Contributors
- @vladislav-sidorovich made their first contribution in #557
Full Changelog: v1.0.60...v1.0.61
v1.0.60
What's Changed
- Remove dwh-migration-client by @dawidxc in #521
- [b/361574176] Extract additional software packages by @dawidxc in #523
- [b/361566488] Extract user local libraries by @dawidxc in #525
- [b/353201454] Extract HDFS report by @sten1ee in #524
- Group single-line scripts in a single text file by @dawidxc in #526
- [b/356467392] Remove MIGRATION_METADATA flag by @dawidxc in #530
- [b/361568886] Extract hive properties by @dawidxc in #527
- [b/361572606] Fix the command for extracting spark version by @dawidxc in #528
- [b/353664125] Extract statuses of services by @dawidxc in #529
- [b/353201454] Ignore slow nodes extraction failure by @dawidxc in #535
- [b/362196485] Add db-links, db-link-sources SQLs by @misolt in #539
- [b/362205662] Extend Oracle source-conn-latest sql with additional columns by @kpiotrowski in #537
- [b/360840839] Extract Hadoop native libraries by @dawidxc in #532
- [b/361577312] Extract hardware, storage and HDFS details by @dawidxc in #533
- [b/357517011] Extract filenames from local filesystem by @dawidxc in #531
- [b/352490642] Add cloudera metaconnector by @dawidxc in #538
- Rename hadoop connector to cloudera-metadata by @dawidxc in #542
- Rename hdfs-permissions connector to hdfs by @dawidxc in #543
- [b/352490642] Add HDFS to Cloudera metaconnector by @dawidxc in #544
- [b/352490642] Add child connectors to metadata yaml by @dawidxc in #545
- [b/361371764] Add Java version requirement to README by @dawidxc in #547
- Make HDFS tasks optional by @dawidxc in #548
- Cleanup loggers in HDFS connector by @dawidxc in #549
- [b/364035944] Prevent a NullPointerException where a result's toString has returned null by @misolt in #552
- ParallelTaskGroup fixes by @misolt in #541
Full Changelog: v1.0.59...v1.0.60
v1.0.59
What's Changed
- [b/350521386] Add correct Oracle column name for ALL_SEQUENCES.SEQUENCE_OWNER by @ikhavkin-google in #508
- [b/359866106] Add SQLs to discover Oracle exadata and system configuration by @kpiotrowski in #520
- [b/361012673] Extract historical metrics from Oracle cdb_hist_seg_stat view by @kpiotrowski in #522
- [b/342096412] Detect Java version in the main script by @dawidxc in #434
Full Changelog: v1.0.58...v1.0.59
v1.0.58
What's Changed
- [b/353960111] Extract additional software versions by @dawidxc in #512
- [b/353527405] Add gradle version lock files by @Hackerpilot in #503
- [b/357128377] Update Java dependencies via ./gradlew versionCatalogApplyUpdates by @ajwelch4 in #504
Full Changelog: v1.0.57...v1.0.58
v1.0.57
What's Changed
- Extract metastore connection details by @sten1ee in #499
- [b/352468287] Add non-CDB sql-stats-awr by @misolt in #491
- [b/352468287] Add non-CDB versions of source-conn (latest) and hist-cmd-types-awr by @misolt in #502
- Add a summary success message for each stats SQL group by @misolt in #501
- [b/352468287] Add non-CDB version of db-features SQL by @misolt in #505
- [b/350521386] Improve Oracle sequence metadata collection by @ikhavkin-google in #500
- Move remaining Oracle Task classes to the right package by @misolt in #506
- Remove unused app-schemas-summary by @misolt in #507
- Add non-CDB versions of db-instances, dtl-index-type SQLs by @misolt in #509
- Add non-CDB versions for remaining optional SQLs extracting native data by @misolt in #511
- [b/354721825] Add Apache Ranger connector. by @aleofreddi in #494
- [b/358310586] Fix: 'url' flag does not work with Oracle connectors by @misolt in #513
- [b/355392538] Delete script tmp directory after tests by @dawidxc in #514
- Cleanup of ConnectorArguments by @misolt in #515
- Improve tests of getHost and getHostOrDefault in ConnectorArguments by @misolt in #516
New Contributors
- @ikhavkin-google made their first contribution in #500
- @aleofreddi made their first contribution in #494
Full Changelog: v1.0.56...v1.0.57
v1.0.56
What's Changed
- [b/344812305] Add more columns to the generated csv file by @sten1ee in #443
- [b/344808098] Add file size field to csv file generated by hdfs connector by @sten1ee in #467
- [b/352469141] Make oracle-stats duration customizable by @misolt in #469
- [b/344804598] Produce metadata files for hadoop connector by @dawidxc in #468
- Remove an ignored test in RecordProgressMonitor by @misolt in #458
- [b/352493863] divide oracle-stats tasks into required and optional by @misolt in #471
- [b/352469802] Update the Statspack SQLs to match changes in AWR queries by @misolt in #470
- Summary cleanup by @dawidxc in #473
- Update Statspack SQLs to match AWR by @misolt in #475
- Move all existing SQLs to a 'cdb' subdirectory by @misolt in #476
- Wrap IOException for oracle-stats SQL resources by @misolt in #477
- [b/344812305] Extract top-level folders information by @sten1ee in #474
- Oracle task package by @misolt in #478
- [b/354785499] Add a missing column header 'FileSize' in hdfs-permissions.csv by @sten1ee in #481
- [b/354119674] Refactor HdfsPermissionsExtractionTask to extend AbstractTask by @sten1ee in #479
- [b/353905712] Add metadata files for hdfs-permissions connector by @sten1ee in #484
- [Python client] Update dependencies, remove support for Python 3.7 and add support for Python 3.11 and 3.12 by @ajwelch4 in #480
- Add tools for running single-tenant SQLs conditionally by @misolt in #482
- [b/352468287] Add non-CDB versions of both db-objects SQLs by @misolt in #486
- Change log level for hadoop to info by @dawidxc in #487
- [b/352468287] Add non-CDB versions of db-info and data-types by @misolt in #485
- [b/354783600] Count files and directories separately by @sten1ee in #489
- Add more columns to AWR histcmdtypes by @misolt in #488
- Group together the attributes related to SQL scripts subdirectories by @misolt in #483
- [b/352468287] Add non-CDB table-types-dtl and used-space-details SQLs by @misolt in #490
- [b/354119249] Add HadoopHandle with HDFS FileSystem by @sten1ee in #492
- [b/355650388] Add FileType column in hdfs-permissions.csv file by @sten1ee in #493
- Make OracleConnectorScopeTest parameterized by @dawidxc in #496
- Fix output dir format in metadata yaml by @dawidxc in #495
- [b/309765009] Change support reference to Google by @dawidxc in #497
Full Changelog: v1.0.55...v1.0.56
v1.0.55
v1.0.54
What's Changed
- [b/341097448] Use temporary directory for extracting scripts by @dawidxc in #427
- [b/341297017] Add table-types-dtl.sql by @misolt in #433
- [b/341297017] Add m-view-types.sql by @misolt in #438
- Exclude AutoValue files from coverage in dumper/app by @misolt in #440
- [b/341297017] Add dtl-source-code.sql by @misolt in #441
- [b/341297017] Add data-types.sql by @misolt in #442
- Add tests for MessageTask by @misolt in #437
- Add tests for AbstractOracleConnector by @misolt in #439
- [b/341297017] Add source-conn supporting newer Oracle versions by @misolt in #445
- [b/341297017] Integrate sql-stats (AWR) from DMA by @misolt in #447
- [b/341297017] Integrate sql-stats (STATSPACK) from DMA by @misolt in #450
- [b/341297017] Integrate hist-cmd-types (AWR) from DMA by @misolt in #449
- Fixes for hist-cmd-types (STATSPACK) SQL by @misolt in #451
- Add the SEGMENT_NAME column by @misolt in #454
- Add a test class for StatsJdbcTask by @misolt in #446
- Add the OBJECT_NAME column to both db-objects SQLs by @misolt in #457
- [b/341297017] Integrate exttab.sql from DMA scripts by @misolt in #459
- Fix: missing column names in sql-stats-statspack SQL by @misolt in #460
- Throw exception when Oracle connection parameters are ambiguous by @misolt in #453
- [b/350668065] Make column names consistent for sql-stats CSVs by @misolt in #461
- Fix: Parameter names don't match their values by @misolt in #448
- [b/350668065] Update oracle-stats SQLs with feedback from ingestion by @misolt in #463
- [b/343058608] Change the format of master-keys file from csv to jsonl by @dawidxc in #455
- [b/343058965] Change delegation tokens file from csv to jsonl by @dawidxc in #456
- [b/343063007] Split partitions into files per table by @dawidxc in #464
- Add dtl-index-type, update other SQLs with a snapshot time range by @misolt in #465
Full Changelog: v1.0.53...v1.0.54
v1.0.53
What's Changed
- [b/341297017] Integrate the PDBs version of app_schemas.sql from the DMA tool by @misolt in #387
- [b/341297017] Integrate the summary version of app_schemas.sql from the DMA tool by @misolt in #388
- [b/341931592] Fix tests to always use the utc timezone by @misolt in #390
- [code health] Remove testConnectorSynthetic by @misolt in #384
- [b/275708841] Add output name tests for teradata-logs by @misolt in #391
- [b/275708841] Fix duplicate "-logs" suffix in some output archive names by @misolt in #393
- [b/342534698] fix: missing summary after oracle-stats completes by @misolt in #396
- [b/341297017] Integrate more queries getting general pdb information by @misolt in #395
- Cleanup of redundant generics for String by @dawidxc in #394
- [b/343037983] Extract catalogs from HMS by @dawidxc in #398
- remove
ExtendableResultSetExtractor
by @misolt in #406 - [b/343047614] Extract databases from HMS by @dawidxc in #399
- [b/343047614] Hide migration metadata extraction behind a flag by @dawidxc in #400
- [b/343058608] Extract master keys from HMS by @dawidxc in #401
- [b/343058965] Extract delegation tokens from HMS by @dawidxc in #402
- Add metadata and log connector for greenplum by @zzzzwang in #392
- [b/343187503] Change output paths to avoid subdirectories by @misolt in #407
- Rename name to targetPath in StatsJdbcTask by @dawidxc in #408
- [b/343862780] add a compilerworks-format text file in oracle-logs and -stats output by @misolt in #410
- Remove the special handling of count=-1 in newCsvRSE by @misolt in #409
- Add an optional Jacoco report task in Gradle by @misolt in #413
- [b/343064637] Extract functions from HMS by @dawidxc in #403
- [b/343068084] Extract resource plans from HMS by @dawidxc in #404
- [b/343069385] Extract tables from HMS by @dawidxc in #405
- Add RunWith JUnit4 annotation by @dawidxc in #419
- [b/343069385] Extract primary keys from HMS by @dawidxc in #414
- [b/343069385] Extract foreign keys from HMS by @dawidxc in #415
- [b/343069385] Extract constraints from HMS by @dawidxc in #416
- [b/343069385] Extract table statistics from HMS by @dawidxc in #417
- [b/343063007] Extract partitions from HMS by @dawidxc in #420
- Address concurrency issues in TaskSetState by @misolt in #422
- Use Jackson to generate the JSON objects and arrays. by @dawidxc in #421
- [b/344804598] Add hadoop connector by @dawidxc in #424
- [b/339392173] implement HdfsPermissionExtractionConnector by @sten1ee in #426
- [b/344804598] Add test for HadoopMetadataConnector by @dawidxc in #425
- [b/346977975] Add missing project and license urls for the new dependencies by @kpiotrowski in #428
- [b/345177751] Revert the deprecation of some metadata file tasks by @misolt in #431
- [b/346977975] Add missing dependency project information by @kpiotrowski in #430
- [b/341297017] Add used-space-details, db-objects SQLs by @misolt in #432
- Add a basic check for oracle-stats SQLs by @misolt in #436
New Contributors
Full Changelog: v1.0.52...v1.0.53