Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make stack traces more comprehensive yet simple #43178

Open
1 of 2 tasks
omkar-foss opened this issue Oct 18, 2024 · 0 comments
Open
1 of 2 tasks

Make stack traces more comprehensive yet simple #43178

omkar-foss opened this issue Oct 18, 2024 · 0 comments
Labels
area:core kind:feature Feature Requests kind:meta High-level information important to the community

Comments

@omkar-foss
Copy link
Collaborator

omkar-foss commented Oct 18, 2024 •
edited
Loading

Description

As per users' feedback in the Airflow Debugging Survey 2024, user feedback around incomplete stack traces is as follows:

  • Stack traces do not always show the full path of the source file, which can lead to confusion with files that have the same name across installed packages.
  • Stack traces for failing DAGs often lack complete information when displayed in the UI's red error bar.
  • Stack traces are generally helpful but can be misleading when they arise from dependency conflicts after upgrades.

Use case/motivation

Goals for this issue are the following:

  • Automatically dump relevant variables and environment states at the time of failure, including task parameters and Airflow configurations. Ensure to cleanup the dump to remove redundant content.
  • Add time information to stack traces, detailing the time spent on different operations within a task.
  • Include other DAG execution state information that could be useful to the user. Preferably keep it configurable to user can disable if not required.
  • Attempt to always show the full path of the source file, in order to avoid confusion during debugging.

Related issues

Parent Issue: #40975

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@omkar-foss omkar-foss converted this from a draft issue Oct 18, 2024
@omkar-foss omkar-foss added area:core kind:meta High-level information important to the community kind:feature Feature Requests labels Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:core kind:feature Feature Requests kind:meta High-level information important to the community
Projects
Debugging Improvements - Airflow 3
Status: Planning
Milestone
No milestone
Development

No branches or pull requests
1 participant
@omkar-foss