Skip to content

Add Scarf based telemetry#39510

Merged
kaxil merged 7 commits into
apache:mainfrom
astronomer:scarf
May 16, 2024
Merged

Add Scarf based telemetry#39510
kaxil merged 7 commits into
apache:mainfrom
astronomer:scarf

Conversation

@kaxil
Copy link
Copy Markdown
Member

@kaxil kaxil commented May 9, 2024

To take the discussion about adding Telemetry forward, I am creating a draft PR that adds some basic telemetry to send to Scarf.

Voting thread: https://lists.apache.org/thread/h1x2glvnd42rbj2q2rgpfo3pjhmpt307

I have added docs on the data collection as well as a way to opt-out of it.

Telemetry added at:

Data collected:

  • Airflow version
  • Python version
  • Platform System info: Linux/Darwin
  • Machine type: arm64/aarch64
  • Airflow Metadata DB: Postgres/MySQL
  • DB: sqlite/postgres
  • DB version: 12.6.3
  • Number of DAGs
image image image image image

My proposal will be that this data will only be provided to PMC members on request to start with. For now, I am using the trial version.

We should also add it to the Airflow website similar to apache/superset#25639.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg Bot added area:CLI area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues kind:documentation labels May 9, 2024
Comment thread airflow/utils/scarf.py Outdated
Comment thread airflow/config_templates/config.yml Outdated
Comment thread airflow/config_templates/config.yml Outdated
Comment thread airflow/config_templates/config.yml Outdated
Comment thread docs/apache-airflow/faq.rst Outdated
Comment thread docs/apache-airflow/faq.rst Outdated
Comment thread docs/apache-airflow/faq.rst Outdated
Comment thread docs/apache-airflow/installation/installing-from-pypi.rst Outdated
Comment thread docs/apache-airflow/installation/installing-from-pypi.rst Outdated
Comment thread docs/apache-airflow/installation/installing-from-pypi.rst Outdated
Comment thread airflow/config_templates/config.yml Outdated
Comment thread airflow/config_templates/config.yml Outdated
@kaxil kaxil force-pushed the scarf branch 5 times, most recently from 0479db7 to b16caed Compare May 9, 2024 20:28
Comment thread docs/apache-airflow/faq.rst Outdated
Comment thread docs/apache-airflow/installation/installing-from-pypi.rst Outdated
Comment thread airflow/utils/scarf.py Outdated
Comment thread tests/utils/test_scarf.py Outdated
Comment thread airflow/settings.py Outdated
Comment thread airflow/www/views.py Outdated
Comment thread airflow/config_templates/config.yml Outdated
Comment thread docs/apache-airflow/faq.rst Outdated
Comment thread airflow/config_templates/config.yml Outdated
Comment thread airflow/cli/commands/scheduler_command.py Outdated
Comment thread airflow/cli/commands/scheduler_command.py Outdated
@kaxil kaxil force-pushed the scarf branch 2 times, most recently from af2184e to ab1855a Compare May 10, 2024 16:04
Comment thread airflow/config_templates/config.yml Outdated
Comment thread airflow/utils/scarf.py Outdated
Comment thread docs/apache-airflow/faq.rst
Comment thread docs/apache-airflow/faq.rst
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:CLI area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues kind:documentation type:new-feature Changelog: New Features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants