v1.6.1

What's Changed

• test: use T.TempDir to create temporary test directory by @Juneezee in #101
• Add cocureency ingestion per threads setting by @atlas-comstock in #100
• Update badgerdb and ristretto v0.2.0 to fix bugs on arm by @atlas-comstock in #98
• Add config block cache size for badgerDB by @atlas-comstock in #102
• fix issues with pr #86 by @jeffreylean in #106
• Refactor/streaming v2 by @jeffreylean in #107
• add tables field to ingest grpc contract by @jeffreylean in #108
• Upgrade github CI go version to 1.17 by @atlas-comstock in #111
• Update GitHub action go version by @atlas-comstock in #112
• Fix graceful shutdown by @atlas-comstock in #110
• implement instrumentation in bigquery by @jeffreylean in #117
• lint the code base on https://staticcheck.io/docs/checks/ by @atlas-comstock in #119
• fix streaming inside ingestWithTable method by @jeffreylean in #121
• add grpc call message size configuration by @jeffreylean in #123
• Feature/nats ingress by @jeffreylean in #122
• Spark connector by @tardunge in #115
• fix nats conf tag casing by @jeffreylean in #124
• Bump github.com/aws/aws-sdk-go from 1.30.25 to 1.33.0 by @dependabot in #125
• Hotfix/slow consumer handling by @jeffreylean in #127
• ingest add metrics ingested rows by @atlas-comstock in #129
• Feat/nats multi table support by @jeffreylean in #133

New Contributors

• @Juneezee made their first contribution in #101
• @jeffreylean made their first contribution in #106
• @dependabot made their first contribution in #125

Full Changelog: v1.6.0...v1.6.1

v1.6.0

What's Changed

• Refactor Sink Writer to accept blocks inplace of bytes by @tardunge in #86

Full Changelog: v1.5.1...v1.6.0

v1.5.1

What's Changed

• Fix issue #88: Use env to set Azure auth if existed by @atlas-comstock in #91

Full Changelog: v1.5.0...v1.6.0

v1.5.0

What's Changed

• adding go plugin as a new method in Computed by @atlas-comstock in #87

New Contributors

• @atlas-comstock made their first contribution in #87

Full Changelog: v1.4.0...v1.5.0

v1.4.0

Changelog

• Added support for managed identities on Azure and improved monitoring #85

v1.3.1

This release fixes the protobuf errors that occur with the new gRPC.

Changelog

• Gogo protobuf is incompatible with grpc-go API/v2.
• Replaced proto-gen-gogo-slick with proto-gen-go-grpc and
  proto-gen-go-vtproto plugins.
• Modified python code generation script.

v1.3.0

This release upgrades to badger/v3 which has a lot of memory-related changes under the hood. See https://github.com/dgraph-io/badger/blob/master/CHANGELOG.md

Changelog

• (Breaking) Rebasing on badger/v3 which makes data incompatible #78

v1.2.9

This releases introduces fixes of parquet ingestion and expands the number of keys that can be added as partition keys.

Changelog

• Expand the range of data types that can be used for hashBy column #75
• Fix parquet ingestion for missing columns #77

v1.2.8

This release includes azure weighted storage when using multi-account (#70) and parquet encoder (egress) #67

Changelog

• Added support for writing to Azure with weighted choice
• Added support for parquet encoding when writing data out of Talaria during compaction
• Added an entry point to allow loading env variable and potentially and generic init procedure within the container. I believe this is useful because if you are using something like Vault to store your secrets, the secrets are mounted as files within the container. With this entry point it will be able to load those secrets.
• Updated Docker build pattern, I noticed there was the attempt to have a "build" pattern however it was commented out. I believe having the Dockerfile in this way is better because people can test easily locally to see if the image builds correctly.

v1.2.7

This release contains major throughput improvements to Azure writer and Talaria now writes to all sinks in parallel. See #66 for more details. Special thanks to @qiaowei-g and @ocassetti for this PR.

Changelog

• Add Azure Multi Storage Account Writer, use Azure AD credential for authentication.
• Change multi.writer to call writers in parallel.
• Add Datadog metric for writer latency and error stats.