feat(pipeline): add ACSA setup for ROS2 bag sync to Blob

.cspell/azure-services.txt

@@ -31,11 +31,15 @@ arcbox
 arcgis
 arck
 arcsight
+arccontainerstorage
 armttk
 azacsnap
 azapi
 azcmagent
 azcopy
+certmanagement
+connectedk8s
+containerstorage
 azdo
 azmk
 azmon
@@ -68,5 +72,8 @@ servicebus
 sharepoint
 snet
 southeastasia
+storageaccountendpoint
 wasbs
 westus
+edgevolume
+edgevolumes

.cspell/general-technical.txt

@@ -953,6 +953,7 @@ pipelines
 pitj
 pivottable
 pkgs
+pkill
 platform
 platformops
 platforms

data-pipeline/arc/README.md

@@ -9,3 +9,13 @@ Kubernetes manifests and configuration for Arc-connected edge data pipeline comp
 | K8s manifests      | Deployments, services, and config maps for recording workloads |
 | RBAC policies      | Service accounts and role bindings for edge agents             |
 | Flux configuration | GitOps sync definitions for automated edge deployment          |
+| ACSA manifests     | PVC and IngestSubvolume templates for cloud-backed edge storage |
+
+## 📄 ACSA Manifests
+
+| File                          | Description                                           |
+|-------------------------------|-------------------------------------------------------|
+| `acsa-pvc.yaml`               | ReadWriteMany PVC backed by ACSA `cloud-backed-sc`   |
+| `acsa-ingest-subvolume.yaml`  | IngestSubvolume CRD defining Blob sync policy         |
+
+These templates use `envsubst` variables rendered by `data-pipeline/setup/deploy-acsa.sh`. See the [ACSA setup guide](../../docs/data-pipeline/acsa-setup.md) for full deployment instructions.

data-pipeline/arc/acsa-ingest-subvolume.yaml

@@ -0,0 +1,20 @@
+---
+apiVersion: arccontainerstorage.azure.net/v1
+kind: IngestSubvolume
+metadata:
+  name: ${SUBVOLUME_NAME}
+  namespace: ${EDGE_NAMESPACE}
+spec:
+  edgevolume: ${ACSA_PVC_NAME}
+  path: ${SUBVOLUME_PATH}
+  authentication:
+    authType: MANAGED_IDENTITY
+  storageAccountEndpoint: "https://${STORAGE_ACCOUNT_NAME}.blob.core.windows.net/"
+  containerName: ${BLOB_CONTAINER_NAME}
+  ingest:
+    order: ${ACSA_INGEST_ORDER}
+    minDelaySec: ${ACSA_INGEST_MIN_DELAY_SEC}
+  eviction:
+    order: ${ACSA_EVICTION_ORDER}
+    minDelaySec: ${ACSA_EVICTION_MIN_DELAY_SEC}
+  onDelete: ${ACSA_ON_DELETE}

data-pipeline/arc/acsa-pvc.yaml

@@ -0,0 +1,13 @@
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: ${ACSA_PVC_NAME}
+  namespace: ${EDGE_NAMESPACE}
+spec:
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: ${ACSA_PVC_SIZE}
+  storageClassName: ${ACSA_STORAGE_CLASS}

data-pipeline/setup/README.md

@@ -9,3 +9,12 @@ Deployment scripts for Arc-connected edge agents that run the ROS 2 recording se
 | Arc agent provisioning  | Connect edge devices to Azure Arc-enabled Kubernetes    |
 | Connectivity validation | Verify cloud connectivity and service endpoints         |
 | Runtime configuration   | Deploy recording configuration and service dependencies |
+| ACSA deployment         | Install Azure Container Storage for Arc and configure Blob sync |
+
+## 📜 Scripts
+
+| Script             | Purpose                                                                          |
+|--------------------|-----------------------------------------------------------------------------------|
+| `deploy-acsa.sh`   | Install cert-manager + ACSA extensions, assign Blob role, apply PVC/subvolume manifests |
+
+See the [ACSA setup guide](../../docs/data-pipeline/acsa-setup.md) for deployment instructions.

data-pipeline/setup/defaults.conf

@@ -0,0 +1,31 @@
+# Default configuration for data-pipeline setup deployment scripts
+# Override via command-line arguments or environment variables
+
+# Default Terraform Directory (relative to data-pipeline/setup)
+DEFAULT_TF_DIR="${DEFAULT_TF_DIR:-../../infrastructure/terraform}"
+
+# Arc Namespace Configuration
+EDGE_NAMESPACE="${EDGE_NAMESPACE:-data-pipeline}"
+
+# ACSA Extension Configuration
+ACSA_EXTENSION_NAME="${ACSA_EXTENSION_NAME:-azure-arc-containerstorage}"
+ACSA_EXTENSION_VERSION="${ACSA_EXTENSION_VERSION:-2.11.2}"
+ACSA_RELEASE_TRAIN="${ACSA_RELEASE_TRAIN:-stable}"
+ACSA_DISK_STORAGE_CLASS="${ACSA_DISK_STORAGE_CLASS:-default,local-path}"
+
+# ACSA Volume Configuration
+ACSA_STORAGE_CLASS="${ACSA_STORAGE_CLASS:-cloud-backed-sc}"
+ACSA_PVC_NAME="${ACSA_PVC_NAME:-recording-data}"
+ACSA_PVC_SIZE="${ACSA_PVC_SIZE:-50Gi}"
+
+# Blob Sync Configuration
+BLOB_CONTAINER_NAME="${BLOB_CONTAINER_NAME:-datasets}"
+SUBVOLUME_NAME="${SUBVOLUME_NAME:-recordings}"
+SUBVOLUME_PATH="${SUBVOLUME_PATH:-recordings}"
+
+# IngestSubvolume synchronization behavior
+ACSA_INGEST_ORDER="${ACSA_INGEST_ORDER:-oldest-first}"
+ACSA_INGEST_MIN_DELAY_SEC="${ACSA_INGEST_MIN_DELAY_SEC:-30}"
+ACSA_EVICTION_ORDER="${ACSA_EVICTION_ORDER:-unordered}"
+ACSA_EVICTION_MIN_DELAY_SEC="${ACSA_EVICTION_MIN_DELAY_SEC:-600}"
+ACSA_ON_DELETE="${ACSA_ON_DELETE:-trigger-immediate-ingest}"