Releases: atlanhq/atlan-java
v0.9.0
π New features
- Asset filters to pipeline bulk query and update operations directly and efficiently
- Search for terms by their categories and categories by their parent category
- Search and retrieve logged administrative events, such as user logins
- Models for:
- Azure Event Hub
- Functions
- StarredBy details
- DbtTag
- Adds a unique
X-Atlan-Request-Id
header to every call that can be used for tracing requests all the way through Atlan's back-end operations
βοΈ Breaking changes
- Replaces the
retrieveFull()
,retrieveMinimal()
,retrieveByGuid()
andretrieveByQualifiedName()
methods with a single overloadedget()
method:- From any asset you can now retrieve an instance using
get()
, for example:Table.get()
. The SDK will automatically detect whether you are retrieving the asset by GUID or qualifiedName and act accordingly. To retrieve the minimal version (without relationships) simply pass an optional argument offalse
to not include the relationships. - For dynamic typing, you can use
Asset.get()
in place ofAsset.retrieveFull()
andAsset.retrieveMinimal()
.
- From any asset you can now retrieve an instance using
- The
builder()
method of assets was only ever intended for internal use. As such, it has now been renamed_internal()
to make this more apparent. Unless explicitly mentioned in developer.atlan.com, you should always usecreator()
,updater()
, ortrimToRequired()
rather than directly accessing_internal()
to construct an asset. - The SDK now supports running against multiple tenants in a single JVM. To enable this, we've:
- Added a mandatory
AtlanClient
parameter when creating anAssetBatch
. (You can passAtlan.getDefaultClient()
to retain the existing behavior.) - Removed the ability to directly access the
Serde.mapper
ObjectMapper β instead now use the.readValue()
and.writeValueAsString()
operations on anAtlanClient
. (For example:Atlan.getDefaultClient().readValue()
.) - Related to above,
toJson()
on any asset now requires a client β if you were using this directly, you can now call.toJson(Atlan.getDefaultClient())
to retain the existing behavior. - Removed the ability to directly access the various
...Endpoint
classes. These are now only available through anAtlanClient
, for example:Atlan.getDefaultClient().assets.save(List.of(asset1, asset2), false)
rather thanBulkEntityEndpoint.upsert(List.of(asset1, asset2), false)
. - Removed static caches β caches are now client-specific:
- Anywhere you previously did something like
RoleCache.getIdForName()
you will now need to useAtlan.getDefaultClient().getRoleCache().getIdForName()
. - Note that this pattern applies to all caches (
RoleCache
,CustomMetadataCache
,AtlanTagCache
,EnumCache
,UserCache
andGroupCache
).
- Anywhere you previously did something like
- Added a mandatory
- The static
getBaseUrlSafe()
method has been removed, as thegetBaseUrl()
method no longer throws a checked exception and can therefore be used directly instead. - The
upsert()
operation has been deprecated, replaced by asave()
operation that takes the same arguments and behaves in the same way.
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Fixes an
IndexOutOfBoundsException
that could occur when trying to retrieve a non-existent API token by its name. - Fixes an issue with the default page size used for asset filters, so they should now work even if no page size is explicitly set.
- Marks all API-operable objects as serializable.
- Replaces the
toString()
method that produced JSON output with an object-nativetoString()
. While this may feel less readable, it removes any dependency on tenant-specific information for the serialization process, and should avoid hiding any transformation or translations that would occur through such a serialization. - Adds a missing
PURGED
status that can exist in some circumstances on deleted instances of metadata. - No longer fails if custom metadata that no longer exists is encountered, but instead uses a value of
(DELETED)
and logs. - Fixes blocking check on asset deletion.
π₯ QOL improvements
- Asset instance retrieval has been simplified to a single
get()
method that works for both GUIDs and qualifiedNames. - The
findByName()
method that exists for various assets (connections, glossary objects, etc) no longer requires an attributes argument (if none is provided it will default to the minimal set of attributes now). - With asset filters you can now quickly query assets of a given type, without needing to build or run a separate
IndexSearchRequest
. For example:Table.all().filter(...).stream().forEach(a -> log.info("Found asset: {}", a));
- All operations that you can invoke against an endpoint now have variants that can take an (optional)
RequestOptions
object. When provided, these allow you to override default client settings for that individual request β timeouts, max retries, etc. - Bumps all library dependencies to their latest stable versions.
v0.8.0
π New features
- Adds the ability to manage arbitrary files as assets
- Adds an abstraction layer for easing implementation of real-time event-processing through webhooks, in particular when implemented using AWS Lambda functions
- Index searches have been simplified through their
builder()
method, defaults for paging, and providing other shortcut factory methods (likeof()
for the DSL) - Lineage listings have been simplified through their
builder()
method and defaults for parameters - Index search and lineage listing responses now provide
Iterable<Asset>
and.stream()
interfaces to simplify iterating through results. These make looping much simpler while simultaneously more efficient β In both cases the next pages of results are automatically fetched as-needed, lazily, so you can always break-out early without retrieving unnecessary results.
βοΈ Breaking changes
- Search builders now have a required argument (the DSL or query to search), rather than starting blank.
- Lineage list builders now have a required argument (the GUID of the asset to start from), rather than starting blank.
- AssetBatch operations now throw any checked exceptions rather than swallowing and logging them.
- Implements multi-inheritance polymorphism for assets.
- This means a given asset type can now inherit its full set of attributes and relationships (across multiple supertypes).
- However, it means we have removed the abstract classes that represented supertypes of assets and replaced these with interfaces. For example, there is no longer a
SQL
or aCatalog
class, but anISQL
and anICatalog
interface. - Anywhere you previously were checking an asset using
if (a instanceof Catalog)
with such an abstract class, you'll now need to do the same check against the interface instead (e.g.if (a instanceof ICatalog)
). - The benefit of this change is that the asset types in the SDK now accurately reflect multi-inheritance of Atlan itself. So a
SnowflakeTag
for example will be bothinstanceof ITag
andinstanceof ISQL
. (With class-based inheritance, we had to choose just one of these to extend.) - Relationships are all now specified using their interface rather than their class, so for example
getAssignedTerms()
will now returnIGlossaryTerm
s rather thanGlossaryTerm
s. As long as you are accessing instance methods for your operations, this should be a transparent change for you in almost all cases, as the interface has all the same (non-static) methods defined on it that exist on the class.
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Fixes an issue introduced in the last release where the
assignedTerms
relationship was missing from generated test classes. - Fixes an issue that required an
attributes.csv
file to be present for injecting descriptions into generated POJOs β this is now optional as part of the code generation. - Fixes an issue for
TableauDatasource
objects, which can now return both calculated fields and datasource fields through the singlefields
relationship. - Fixes the level of caching attempted for roles β previously every role was cached, whereas now only the workspace-roles will be cached.
π₯ QOL improvements
- The
Iterable<Asset>
interface provided by index search and lineage listing responses allow you to:- Use a simple for-each loop:
for (Asset a : response) { ... }
- Use a
forEach()
lambda:response.forEach(a -> { ... });
- Use a simple for-each loop:
- The
.stream()
interface provides the results as a stream, allowing you to:- Use chain-able stream operations, such as:
response.stream().filter(...).limit(n).forEach(a -> {...}).collect()
- Use chain-able stream operations, such as:
- When the retry limit is reached for automated retries, any underlying exceptions or response body should now be logged in addition to the fact that the retry limit was reached.
- Every snapshot build and release now uploads a
lambda-layer.zip
file that can be uploaded directly as an AWS Lambda layer, providing the full Java SDK (and its dependencies). You can then implement any of your own logic through the abstraction layer and simply upload that minimal code directly, without needing to re-package the entire SDK and its dependencies. (This allows you to upload and manage the SDK layer independently of possibly many event handlers implemented on top of it.)
v0.7.0
π New features
- Adds higher-performance lineage list API
- Adds ability to manage an icon or image for tags (classifications)
- Adds new type definitions for:
- MicroStrategy objects
- Renames "classification" to "tag" to align with UI changes
- Adds new persona/purpose/policy management
βοΈ Breaking changes
- Renames "classifications" to "tags". Back-end payloads remain unchanged, but any properties or methods that referred to
classification
will now refer toatlanTag
. For example:mappedClassificationName
->mappedAtlanTagName
purposeClassifications
->purposeAtlanTags
ClassificationCache
->AtlanTagCache
Classification
->AtlanTag
ClassificationDef
->AtlanTagDef
classifications
->atlanTags
classificationNames
->atlanTagNames
appendClassifications
->appendAtlanTags
addClassifications
->addAtlanTags
- Updates persona, purpose and policy management to use new optimized internal changes. As a result, the specific objects and methods needed to manage personas, purposes and policies have changed:
Persona
andPurpose
have moved intocom.atlan.model.assets
- These now directly provide
createMetadataPolicy
andcreateDataPolicy
(andcreateGlossaryPolicy
inPersona
) helpers to manage the policies AuthPolicy
is the new base object for all policies (the helpers above now createAuthPolicy
objects)- Policy management is now more consistent across the two mechanisms, and policies can now be created in bulk for both as well
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π₯ QOL improvements
- Refactors the code generators used to build the SDK, so that they can be reused to create "extended" SDKs. All generated code is now also annotated (
javax.annotation.Generated
) for easier identification and simplifying common tasks like excluding generated code from coverage tests.
(Note: this is really just a re-issue of the v0.7.0 release that fixes a JavaDoc error that caused publishing to Maven Central to fail.)
v0.6.0
π New features
- New connector types (icons):
- Azure Data Lake
- Delta Lake
- Mini SQL (mSQL)
- Apache Iceberg
- Apache Impala
- Apache Spark SQL
- MariaDB
- Firebolt
- Cloudera Data Warehouse
- Starburst Galaxy
- Redis
- GraphQL
- Adds new type definitions for:
- Redash objects
- Monte Carlo objects
- Adds new crawlers for:
- Microsoft SQL Server
- Sigma
- Adds initial experimental exposure of Playbooks
βοΈ Breaking changes
- Deprecates the top-level
meanings
relationship, as it is not consistently present on responses. Instead use theassignedTerms
relationship. - Removes the parameter for custom metadata handling from the
upsert()
method. The behavior when true was very invasive without being obvious, so instead we have replaced it with two separate methods:upsertMergingCM()
will merge the provided custom metadata in a request with any existing custom metadata already on the asset. This is net-new functionality.upsertReplacingCM()
will overwrite all custom metadata on the asset. (Any custom metadata not provided in the request will result in that custom metadata being removed from the asset.) This is the same behavior as the originalupsert()
method when sending thereplaceCustomMetadata
parameter with a value oftrue
.- The same distinction in handling is now available when using an
AssetBatch
as well. (The default behavior when unspecified remains as before: all custom metadata in a batch is ignored, by default.)
- Removes
AbstractProcess
andAbstractColumnProcess
β instead, useLineageProcess
andColumnProcess
directly. - Simplifies the asset filtering provided to the
DbtCrawler
to be a string rather than series of complex nested objects.
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Fixes nested loops causing excessive retries on connection creation
- Fixes serde issues of complex embedded structures (internal maps and lists of maps)
- Fixes the outer
meanings
objects, which were previously missed out by deserialization - Fixes an issue where connections were improperly created due to missing empty lists for admin roles/users/groups
- Fixes the
replaceCustomMetadata()
method, so it should now properly replace only the properties within the specified custom metadata set on an asset - Fixes handling of deleted objects (like classifications and custom metadata) for audit entries, which will be unable to resolve the human-readable names of these objects once they've been deleted.
- Fixes deserialization of Atlan structs (complex nested objects). Previously these would always be deserialized as empty.
π₯ QOL improvements
- Adds an idempotent
appendClassifications()
method, which will only add classifications that are not already associated with an asset. (The existing options would throw errors if any provided classifications in a request overlapped with existing classifications on an asset, so were not idempotent.) - Adds helpers for creating custom metadata with custom logos (whether emojis or entirely custom images)
v0.5.1
π New features
- New connector types (icons):
- AlloyDB
- Aurora
- Azure Analysis Services
- Azure Cosmos DB
- Clari
- ClickHouse
- CockroachDB
- Firebird
- Greenplum
- Hex
- IBM Informix
- Kafka
- Marketo
- MonetDB
- MongoDB
- MuleSoft
- Oracle TimesTen
- Percona Server
- QuickSight
- Rockset
- SAP IQ
- SAP MaxDB
- SAP SQL
- SingleStore
- SQLite
- Teradata
- ThoughtSpot
- YugabyteDB
- Adds crawler for dbt Core
π Bug fixes
- Adds metadata annotations to all crawlers
- Fixes integration tests for inclusion in CI/CD-driven daily regression testing
- Fixes bundled JRE for container image, to include all necessary base Java dependencies for logging
π₯ QOL improvements
- Adds an
AtlanAsyncMutator
interface to expose ablock()
method, to force waiting until asynchronous operations are completed. Currently implemented for both connection creation and asset deletion. - Adds a more flexible
addClassifications()
method that allows controlling propagation settings of all provided classifications.
v0.5.0
π New features
- Models for new asset types:
- Kafka
- Qlik
- QuickSight
- New connector types:
- Sigma
- Synapse
- Airflow
- OpenLineage
- Dataflow
- Qlik Sense
- Models for events that are generated by Atlan
- Badges and badge conditions complete with
creator()
, factory methods, and related enumerations - New
AssetBatch
class to abstract and encode processing batches of assets at the same time - (Experimental) Initial objects and endpoints for requests handling
- (Experimental) Builds a shadow jar file that includes Numaflow, for potential event-based integration
βοΈ Breaking changes
- Renames the
sqlAsset
property toprimarySqlAsset
inDbtSource
β to allow setting the new multi-valuedsqlAssets
property using the singular builder pattern - Renames the
MEANINGS
searchable field enum toASSIGNED_TERMS
- Aligns to latest ADLS model (removes attributes
adlsAccountCreationTime
,adlsContainerLastModifiedTime
,adlsObjectCreationTime
,adlsObjectLastModifiedTime
- S3 objects now require
bucketQualifiedName
andbucketName
for creation - Removes the
updater()
method for personas and purposes β these should instead always be retrieved first, updated, then sent back to Atlan as an update - Makes
name
a mandatory parameter for theupdateCertificate()
andupdateAnnouncement()
operations on a glossary - Enforces immutability of objects (must use the builder pattern):
- Removes all setter methods (
set...()
) from the models. - Objects can only be built and modified now using the
builder()
s (and relatedcreator()
andupdater()
methods that return builders) - If you were previously modifying objects in-place, you now need to build an immutable copy of them to update them:
asset.setDescription("abc123");
->asset = asset.toBuilder().description("abc123").build();
- Note that collection-based fields will only append using this pattern, to replace you first need to chain the equivalent
clear...()
method:term.toBuilder().clearCategories().category(GlossaryCategory.refByGuid("...")).build();
would replace all categories defined on the term with the single category provided.
- Removes all setter methods (
- Adds the ability to limit a dbt crawler to a specific connection (an extra parameter in the fully-parameterized crawler method)
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Fixed an issue where an aggregation that had no content would cause an IOException during deserialization β now instead simply returns the empty aggregation result
- Fixes custom metadata creation, which now requires a name and description in all payloads (even if empty strings)
- Fixes an issue where the contents of a README were left unencoded during serialization
- Removes unused
connectionName
property when creating a lineage process - Pass-through internal server error details (500 response codes)
- Fixes a problem where if a persona or purpose was not supplied with a default blank description for creation, a knock-on issue occurred when trying to add policies to those objects
- Invalidates and refreshes caches any time a type definition is created, updated, or deleted via the SDK
- Drops the optional
entityGuid
when setting classification assignments - Fixes an issue where a trailing
/
on the base URL would have caused non-obvious failures (returning HTML instead of API responses) - Changes the debugging output for internal server errors (>500 response codes), to differentiate them from permission errors (403 response codes) since both will retry
- Fixes an issue related to the deserialization of popularity details within event payloads
- Fixes an issue where a null classification assignment could cause an IOException by trying to resolve a classification with the name
null
π₯ QOL improvements
- Adds an
isArchived()
helper for AttributeDefs, to make it easier to determine whether they are active or soft-deleted - Adds an
archive()
helper for AttributeDefs, to abstract away the steps needed to archive (soft-delete) an attribute definition - Adds a factory method for creating classification assignments without requiring entity details (
Classification.of()
) - Reduces logging level for API retries from
INFO
toDEBUG
- Adds
excludeMeanings
andexcludeClassifications
options for further optimizing searches that do not require these details in the results - Adds
generateQualifiedName()
helpers to objects that havecreators()
andupdaters()
, to encode creating the appropriate qualifiedNames for these objects
v0.4.0
π New features
- Adds the ability to provide your own unique ID for lineage processes, for more consistent updates (see below)
βοΈ Breaking changes
LineageProcess
creator method signature has changed, to:- Simplify its inputs β rather than requiring a connector type, connection name and connection qualifiedName you can now provide only the connection qualifiedName and the rest will be determined from this automatically.
- Allow a unique ID to be provided β previously a unique ID was always generated for you, as a hash of the inputs and outputs of the process. By providing your own ID you can now directly match the unique ID in the orchestrating system. This will allow you to update the same lineage process over time, even if its inputs and outputs change, without creating new nodes in lineage. (Previously any change to the inputs or outputs of a process would have resulted in a different generated hash ID, and thus a different lineage process.) This remains an optional input, though, so if you send the argument as
null
a hashed ID will still be generated (as before).
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Fixed an issue where an aggregation that had no content would cause an IOException during deserialization β now instead simply returns the empty aggregation result
π₯ QOL improvements
- Adds helper method to reverse-engineer connector type from an asset's qualifiedName
v0.3.0
π New features
- Adds simplified set of helpers to construct searches through the
QueryFactory
- Adds validation of connection admins during both creation and update of a connection
- Adds validation of users and groups specified in purpose policies
βοΈ Breaking changes
QueryFactory
methods have changed to match the naming convention used for the simplification and the fact that searchable fields are now enumerated rather than simple strings:active()
->beActive()
archived()
->beArchived()
withLineage()
->haveLineage()
withType()
->beOfType()
withSuperType()
->haveSuperType()
withCertificate()
has been removed. Instead usehave(KeywordFields.CERTIFICATE_STATUS).eq()
withExactName()
has been removed. Instead usehave(KeywordFields.NAME).eq()
whereQualifiedNameStartsWith()
has been removed. Instead usehave(KeywordFields.QUALIFIED_NAME).startingWith()
withAnyValueFor()
has been removed. Instead usehave(<field>).present()
withAtLeastOneClassification()
->beClassifiedByAtLeastOneOf()
, which now takes a collection of human-readable classification names rather than internal hashed-string classification IDswithAtLeastOneTerm()
->beDefinedByAtLeastOneOf()
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Fixed an issue where all attributes of type
PopularityInsights
were incorrectly (de)serialized - Fixed an issue where a typedef with certain characters in its name (like spaces) could not be deleted
π₯ QOL improvements
- Enumerates all searchable fields to avoid typos or choosing the wrong indexed field for a particular type of search
- Simplified helpers in
QueryFactory
for building complex queries, aggregations, and sorting - Simplified helpers in
QueryFactory
for retrieving results of metrics-based aggregations - It should no longer be possible to create a connection that you are locked out of due to making a typo in the details of any roles, groups or users you set as connection admins
- It should no longer be possible to create a policy on a purpose that applies to non-existent users or groups
v0.2.0
π New features
- Adds management of user membership in groups
- Adds recently released metadata types: ADLS, Sigma
- Adds Tableau crawler package
- Adds Athena crawler package
- Adds
findByName
helpers for retrieving glossary objects by their human-readable names - Adds management of purposes
βοΈ Breaking changes
- Fixes the generation of
qualifiedName
for process assets, which is not backwards-compatible. (For any existing processes created via the SDK, you'll end up with duplicates due to the newqualifiedName
. To avoid this, if possible, remove those existing processes and re-run using this updated SDK to generate the correctqualifiedName
s, and then going forward these will be used and there will be no duplication.)
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Enumerates errors even when they are passed-through directly, to avoid an internal fatal exception related to formatting of error messages
π₯ QOL improvements
- Adds
creator
methods for policies to ensure minimal required information is always present - Extends the base error class to capture details from differing error response formats
v0.1.0
π New features
- Adds Snowflake Miner package
- Adds enumerations (options) for custom metadata
- Adds the purge deletion type, which can be applied to both active and archived assets
- Adds helpers to re-run existing workflows
- Adds latest model revisions, including usage / popularity metadata
βοΈ Breaking changes
- Flattens
Entity
andAsset
β now all operations returnAsset
s directly (anywhere you previously usedEntity
orEntityMutationResponse
now needs to be replaced with theirAsset
-equivalent) - Caches now throw exceptions (
NotFoundException
) when lookups fail, rather than simply returning null
(Note: we're reserving the right to make breaking changes to the SDK prior to a 1.0.0 release, without incrementing the major version. Once we hit 1.0.0, we'll adhere to https://semver.org)
π Bug fixes
- Corrects include/exclude filters for crawlers
π₯ QOL improvements
- With the
Entity
->Asset
flattening, you no longer need to cast objects to retrieve things likequalifiedName
,name
,certificateStatus
, and so on - Enumerates all error messages with a unique code, and simplifies exception creation (default status code per exception)