Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,23 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

### Updated

## [v6.8.1] - 2025-12-15

### Changed

- Implemented a safety-first deletion policy for the catalogs endpoint to prevent accidental data loss. Collections are now never deleted through the catalogs route; they are only unlinked and automatically adopted by the root catalog if they become orphans. Collection data can only be permanently deleted via the explicit `/collections/{collection_id}` DELETE endpoint. This ensures a clear separation between container (catalog) deletion and content (collection/item) deletion, with data always being preserved through the catalogs API.

### Removed

- Removed `cascade` parameter from `DELETE /catalogs/{catalog_id}` endpoint. Collections are no longer deleted when a catalog is deleted; they are unlinked and adopted by root if orphaned.

## [v6.8.0] - 2025-12-15

### Added

- Environment variable `VALIDATE_QUERYABLES` to enable/disable validation of queryables in search/filter requests. When set to `true`, search requests will be validated against the defined queryables, returning an error for any unsupported fields. Defaults to `false` for backward compatibility.[#532](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/532)
- Environment variable `QUERYABLES_CACHE_TTL` to configure the TTL (in seconds) for caching queryables. Default is `1800` seconds (30 minutes) to balance performance and freshness of queryables data. [#532](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/532)
- Added optional `/catalogs` route support to enable federated hierarchical catalog browsing and navigation. [#547](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/547)
- Added optional `/catalogs` route support to enable hierarchical catalog browsing and navigation. [#547](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/547)
- Added DELETE `/catalogs/{catalog_id}/collections/{collection_id}` endpoint to support removing collections from catalogs. When a collection belongs to multiple catalogs, it removes only the specified catalog from the collection's parent_ids. When a collection belongs to only one catalog, the collection is deleted entirely. [#554](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/554)
- Added `parent_ids` internal field to collections to support multi-catalog hierarchies. Collections can now belong to multiple catalogs, with parent catalog IDs stored in this field for efficient querying and management. [#554](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/554)
- Added GET `/catalogs/{catalog_id}/children` endpoint implementing the STAC Children extension for efficient hierarchical catalog browsing. Supports type filtering (?type=Catalog|Collection), pagination, and returns numberReturned/numberMatched counts at the top level. [#558](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/558)
Expand Down Expand Up @@ -689,7 +699,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Use genexp in execute_search and get_all_collections to return results.
- Added db_to_stac serializer to item_collection method in core.py.

[Unreleased]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.8.0...main
[Unreleased]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.8.1...main
[v6.8.1]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.8.0...v6.8.1
[v6.8.0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.6...v6.8.0
[v6.7.6]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.5...v6.7.6
[v6.7.5]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v6.7.4...v6.7.5
Expand Down Expand Up @@ -730,3 +741,4 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
[v0.3.0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v0.2.0...v0.3.0
[v0.2.0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v0.1.0...v0.2.0
[v0.1.0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v0.1.0

70 changes: 50 additions & 20 deletions README.md

Large diffs are not rendered by default.

129 changes: 57 additions & 72 deletions stac_fastapi/core/stac_fastapi/core/extensions/catalogs.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ def register(self, app: FastAPI, settings=None) -> None:
response_class=self.response_class,
status_code=204,
summary="Delete Catalog",
description="Delete a catalog. Optionally cascade delete all collections in the catalog.",
description="Delete a catalog. All linked collections are unlinked and adopted by root if orphaned.",
tags=["Catalogs"],
)

Expand Down Expand Up @@ -337,22 +337,21 @@ async def get_catalog(self, catalog_id: str, request: Request) -> Catalog:
status_code=404, detail=f"Catalog {catalog_id} not found"
)

async def delete_catalog(
self,
catalog_id: str,
request: Request,
cascade: bool = Query(
False,
description="If true, delete all collections linked to this catalog. If false, only delete the catalog.",
),
) -> None:
"""Delete a catalog.
async def delete_catalog(self, catalog_id: str, request: Request) -> None:
"""Delete a catalog (The Container).

Deletes the Catalog document itself. All linked Collections are unlinked
and adopted by Root if they become orphans. Collection data is NEVER deleted.

Logic:
1. Finds all Collections linked to this Catalog.
2. Unlinks them (removes catalog_id from their parent_ids).
3. If a Collection becomes an orphan, it is adopted by Root.
4. PERMANENTLY DELETES the Catalog document itself.

Args:
catalog_id: The ID of the catalog to delete.
request: Request object.
cascade: If true, delete all collections linked to this catalog.
If false, only delete the catalog.

Returns:
None (204 No Content)
Expand All @@ -361,58 +360,42 @@ async def delete_catalog(
HTTPException: If the catalog is not found.
"""
try:
# Get the catalog to verify it exists
# Verify the catalog exists
await self.client.database.find_catalog(catalog_id)

# Use reverse lookup query to find all collections with this catalog in parent_ids.
# This is more reliable than parsing links, as it captures all collections
# regardless of pagination or link truncation.
# Find all collections with this catalog in parent_ids
query_body = {"query": {"term": {"parent_ids": catalog_id}}}
search_result = await self.client.database.client.search(
index=COLLECTIONS_INDEX, body=query_body, size=10000
)
children = [hit["_source"] for hit in search_result["hits"]["hits"]]

# Process each child collection
# Safe Unlink: Remove catalog from all children's parent_ids
# If a child becomes an orphan, adopt it to root
root_id = self.settings.get("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi")

for child in children:
child_id = child.get("id")
try:
if cascade:
# DANGER ZONE: User explicitly requested cascade delete.
# Delete the collection entirely, regardless of other parents.
await self.client.database.delete_collection(child_id)
logger.info(
f"Deleted collection {child_id} as part of cascade delete for catalog {catalog_id}"
)
else:
# SAFE ZONE: Smart Unlink - Remove only this catalog from parent_ids.
# The collection survives and becomes a root-level collection if it has no other parents.
parent_ids = child.get("parent_ids", [])
if catalog_id in parent_ids:
parent_ids.remove(catalog_id)
child["parent_ids"] = parent_ids

# Update the collection in the database
# Note: Catalog links are now dynamically generated, so no need to remove them
await self.client.database.update_collection(
collection_id=child_id,
collection=child,
refresh=False,
parent_ids = child.get("parent_ids", [])
if catalog_id in parent_ids:
parent_ids.remove(catalog_id)

# If orphan, move to root
if len(parent_ids) == 0:
parent_ids.append(root_id)
logger.info(
f"Collection {child_id} adopted by root after catalog deletion."
)

# Log the result
if len(parent_ids) == 0:
logger.info(
f"Collection {child_id} is now a root-level orphan (no parent catalogs)"
)
else:
logger.info(
f"Removed catalog {catalog_id} from collection {child_id}; still belongs to {len(parent_ids)} other catalog(s)"
)
else:
logger.debug(
f"Catalog {catalog_id} not in parent_ids for collection {child_id}"
logger.info(
f"Removed catalog {catalog_id} from collection {child_id}; still belongs to {len(parent_ids)} other catalog(s)"
)

child["parent_ids"] = parent_ids
await self.client.database.update_collection(
collection_id=child_id, collection=child, refresh=False
)
except Exception as e:
error_msg = str(e)
if "not found" in error_msg.lower():
Expand Down Expand Up @@ -929,11 +912,11 @@ async def get_catalog_children(
async def delete_catalog_collection(
self, catalog_id: str, collection_id: str, request: Request
) -> None:
"""Delete a collection from a catalog.
"""Delete a collection from a catalog (Unlink only).

If the collection has multiple parent catalogs, only removes this catalog
from the parent_ids. If this is the only parent catalog, deletes the
collection entirely.
Removes the catalog from the collection's parent_ids.
If the collection becomes an orphan (no parents), it is adopted by the Root.
It NEVER deletes the collection data.

Args:
catalog_id: The ID of the catalog.
Expand All @@ -959,37 +942,39 @@ async def delete_catalog_collection(
detail=f"Collection {collection_id} does not belong to catalog {catalog_id}",
)

# If the collection has multiple parents, just remove this catalog from parent_ids
if len(parent_ids) > 1:
parent_ids.remove(catalog_id)
collection_db["parent_ids"] = parent_ids
# SAFE UNLINK LOGIC
parent_ids.remove(catalog_id)

# Update the collection in the database
# Note: Catalog links are now dynamically generated, so no need to remove them
await self.client.database.update_collection(
collection_id=collection_id, collection=collection_db, refresh=True
# Check if it is now an orphan (empty list)
if len(parent_ids) == 0:
# Fallback to Root / Landing Page
# You can hardcode 'root' or fetch the ID from settings
root_id = self.settings.get(
"STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"
)

parent_ids.append(root_id)
logger.info(
f"Removed catalog {catalog_id} from collection {collection_id} parent_ids"
f"Collection {collection_id} unlinked from {catalog_id}. Orphaned, so adopted by root ({root_id})."
)
else:
# If this is the only parent, delete the collection entirely
await self.client.database.delete_collection(
collection_id, refresh=True
)
logger.info(
f"Deleted collection {collection_id} (only parent was catalog {catalog_id})"
f"Removed catalog {catalog_id} from collection {collection_id}; still belongs to {len(parent_ids)} other catalog(s)"
)

# Update the collection in the database
collection_db["parent_ids"] = parent_ids
await self.client.database.update_collection(
collection_id=collection_id, collection=collection_db, refresh=True
)

except HTTPException:
raise
except Exception as e:
logger.error(
f"Error deleting collection {collection_id} from catalog {catalog_id}: {e}",
f"Error removing collection {collection_id} from catalog {catalog_id}: {e}",
exc_info=True,
)
raise HTTPException(
status_code=500,
detail=f"Failed to delete collection from catalog: {str(e)}",
detail=f"Failed to remove collection from catalog: {str(e)}",
)
2 changes: 1 addition & 1 deletion stac_fastapi/core/stac_fastapi/core/version.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""library version."""
__version__ = "6.8.0"
__version__ = "6.8.1"
8 changes: 4 additions & 4 deletions stac_fastapi/elasticsearch/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ keywords = [
]
dynamic = ["version"]
dependencies = [
"stac-fastapi-core==6.8.0",
"sfeos-helpers==6.8.0",
"stac-fastapi-core==6.8.1",
"sfeos-helpers==6.8.1",
"elasticsearch[async]~=8.19.1",
"uvicorn~=0.23.0",
"starlette>=0.35.0,<0.36.0",
Expand All @@ -48,7 +48,7 @@ dev = [
"httpx>=0.24.0,<0.28.0",
"redis~=6.4.0",
"retry~=0.9.2",
"stac-fastapi-core[redis]==6.8.0",
"stac-fastapi-core[redis]==6.8.1",
]
docs = [
"mkdocs~=1.4.0",
Expand All @@ -58,7 +58,7 @@ docs = [
"retry~=0.9.2",
]
redis = [
"stac-fastapi-core[redis]==6.8.0",
"stac-fastapi-core[redis]==6.8.1",
]
server = [
"uvicorn[standard]~=0.23.0",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@
app_config = {
"title": os.getenv("STAC_FASTAPI_TITLE", "stac-fastapi-elasticsearch"),
"description": os.getenv("STAC_FASTAPI_DESCRIPTION", "stac-fastapi-elasticsearch"),
"api_version": os.getenv("STAC_FASTAPI_VERSION", "6.8.0"),
"api_version": os.getenv("STAC_FASTAPI_VERSION", "6.8.1"),
"settings": settings,
"extensions": extensions,
"client": CoreClient(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""library version."""
__version__ = "6.8.0"
__version__ = "6.8.1"
8 changes: 4 additions & 4 deletions stac_fastapi/opensearch/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ keywords = [
]
dynamic = ["version"]
dependencies = [
"stac-fastapi-core==6.8.0",
"sfeos-helpers==6.8.0",
"stac-fastapi-core==6.8.1",
"sfeos-helpers==6.8.1",
"opensearch-py~=2.8.0",
"opensearch-py[async]~=2.8.0",
"uvicorn~=0.23.0",
Expand All @@ -49,15 +49,15 @@ dev = [
"httpx>=0.24.0,<0.28.0",
"redis~=6.4.0",
"retry~=0.9.2",
"stac-fastapi-core[redis]==6.8.0",
"stac-fastapi-core[redis]==6.8.1",
]
docs = [
"mkdocs~=1.4.0",
"mkdocs-material~=9.0.0",
"pdocs~=1.2.0",
]
redis = [
"stac-fastapi-core[redis]==6.8.0",
"stac-fastapi-core[redis]==6.8.1",
]
server = [
"uvicorn[standard]~=0.23.0",
Expand Down
2 changes: 1 addition & 1 deletion stac_fastapi/opensearch/stac_fastapi/opensearch/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@
app_config = {
"title": os.getenv("STAC_FASTAPI_TITLE", "stac-fastapi-opensearch"),
"description": os.getenv("STAC_FASTAPI_DESCRIPTION", "stac-fastapi-opensearch"),
"api_version": os.getenv("STAC_FASTAPI_VERSION", "6.8.0"),
"api_version": os.getenv("STAC_FASTAPI_VERSION", "6.8.1"),
"settings": settings,
"extensions": extensions,
"client": CoreClient(
Expand Down
2 changes: 1 addition & 1 deletion stac_fastapi/opensearch/stac_fastapi/opensearch/version.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""library version."""
__version__ = "6.8.0"
__version__ = "6.8.1"
2 changes: 1 addition & 1 deletion stac_fastapi/sfeos_helpers/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ keywords = [
]
dynamic = ["version"]
dependencies = [
"stac-fastapi.core==6.8.0",
"stac-fastapi.core==6.8.1",
]

[project.urls]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""library version."""
__version__ = "6.8.0"
__version__ = "6.8.1"
Loading