Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ MWAA](https://aws.amazon.com/managed-workflows-for-apache-airflow/) uses to run
You can also use it locally if you want to run a MWAA-like environment for testing, experimentation,
and development purposes.

Currently, Airflow v2.9.2 is supported. Future versions in parity with Amazon MWAA will be added as
Currently, Airflow v2.9.2 and v2.10.1 are supported. Future versions in parity with Amazon MWAA will be added as
well. _Notice, however, that we do not plan to support previous Airflow versions supported by MWAA._

## Using the Airflow Image
Expand All @@ -26,7 +26,7 @@ To experiment with the image using a vanilla Docker setup, follow these steps:
python3 create_venvs.py --target <development | production>
```

3. Build the Airflow v2.9.2 Docker image
3. Build a supported Airflow version Docker image
- `cd <amazon-mwaa-docker-images path>/images/airflow/2.9.2`
- Update `run.sh` file with your account ID, environment name and account credentials. The permissions associated
with the provided credentials will be assigned to the Airflow components that would be started with the next step.
Expand All @@ -46,8 +46,7 @@ Airflow should be up and running now. You can access the web server on your loca

When you build the Docker images of a certain Airflow version, using either `build.sh` or `run.sh`
(which automatically also calls `build.sh` for you), multiple Docker images will actually be
generated. For example, for Airflow 2.9 (the only currently supported version), you will notice the
following images:
generated. For example, for Airflow 2.9, you will notice the following images:

| Repository | Tag |
| --------------------------------- | ----------------------------- |
Expand Down
16 changes: 15 additions & 1 deletion images/airflow/2.10.1/python/mwaa/config/airflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,19 @@ def _get_opinionated_airflow_secrets_config() -> Dict[str, str]:
"AIRFLOW__SECRETS__BACKEND_KWARGS": json.dumps(connection_lookup_pattern),
}

def _get_opinionated_airflow_usage_data_config() -> Dict[str, str]:
"""
Retrieve the environment variables for Airflow's usage data configuration section.

This config can be overridden by the user.

:returns A dictionary containing the environment variables.
"""

return {
"AIRFLOW__USAGE_DATA_COLLECTION__ENABLED": "False",
Comment thread
michaelfu1029 marked this conversation as resolved.
}


def _get_essential_airflow_webserver_config() -> Dict[str, str]:
"""
Expand Down Expand Up @@ -310,12 +323,13 @@ def get_opinionated_airflow_config() -> Dict[str, str]:
"""
Retrieve the environment variables required to set Airflow configurations.

These environment variables are essential and cannot be overridden by the customer.
These environment variables can be overridden by the customer.

:returns A dictionary containing the environment variables.
"""
return {
**_get_opinionated_airflow_core_config(),
**_get_opinionated_airflow_scheduler_config(),
**_get_opinionated_airflow_secrets_config(),
**_get_opinionated_airflow_usage_data_config(),
}
2 changes: 1 addition & 1 deletion images/airflow/2.9.2/python/mwaa/config/airflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,7 @@ def get_opinionated_airflow_config() -> Dict[str, str]:
"""
Retrieve the environment variables required to set Airflow configurations.

These environment variables are essential and cannot be overridden by the customer.
These environment variables can be overridden by the customer.

:returns A dictionary containing the environment variables.
"""
Expand Down