build(prek-hook): Check whether new "raise AirflowException" is added#55416

Lee-W · 2025-09-09T16:05:23Z

Why

https://lists.apache.org/thread/t8bnhyqy77kq4fk7fj3fmjd5wo9kv6w0

What

Add prek hook to detect raise AirflowException

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

jscheffl

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

scripts/ci/prek/check_new_airflow_exception_usage.py

.pre-commit-config.yaml

Lee-W · 2025-09-14T23:40:43Z

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Yep, we probably could exclude that file if needed. But this is the best I can think of now.

.pre-commit-config.yaml

scripts/ci/prek/check_new_airflow_exception_usage.py

potiuk · 2025-10-13T23:04:19Z

The currect check strategy implies that if I fix an AirflowException-typo that I am not allowed to.I asusme this is for the moment acceptable... just am thinking about this. But have no other good strategy idea in mind.

Maybe better solution:

dump (automatically) list of used Exceptions to a .txt file
fail if a found exception is not on the list
add a flag to the script to clean-up removed exceptions when --all-files are used
prek run check_no_new_airflow_exceptions_added (default - fails if not on list)
prek run check_no_new_airflow_exceptions_added --all-files (check all)
uv run ./scripts/ci/prek/check_no_new_airflow_exceptions_added.py --cleanup - removes all exceptions from .txt file when already removed in the code

Or similar.

github-actions · 2025-12-25T00:19:10Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

Lee-W · 2026-02-15T07:40:41Z

@Lee-W would it be ok if I take over of the implementation? I just reviewed a PR where I almost missed new usage of AirflowException, and I want to promote an action to prevent it for the next times. Thank you!

That would be awesome! I really want to go back and get this one done, but never actaully have the bandwidth to do so.

github-actions · 2026-04-04T00:26:39Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

Lee-W · 2026-04-07T08:08:52Z

@potiuk @shahar1 I tried to wrap up this PR created long long long ago (with Claude). Please take a look when you have some time :)

shahar1

Thanks for creating it, and apologies that I couldn't finish it up on my own.
Few comments :)

scripts/ci/prek/check_new_airflow_exception_usage.py

.pre-commit-config.yaml

potiuk · 2026-04-09T12:38:39Z

Oh wow. We have that many ?

jason810496

Nice! I like this simple and smart way to keep track of all existing overly broad ‎AirflowException usages. It should help avoid adding too many broad ‎AirflowException usages and will serve as a final warning for contributors.

jscheffl

Oh, wow, really many. Let me fix the edge3 provider one :-D

…apache#55416) * build(prek-hook): Check whether new "raise AirflowException" is added * feat: add white list * refactor: simplify the record to use count instead of raw content * fixup! refactor: simplify the record to use count instead of raw content * refactor: filter python files that matters * fixup! refactor: filter python files that matters (cherry picked from commit 6f56aa1)

…apache#55416) * build(prek-hook): Check whether new "raise AirflowException" is added * feat: add white list * refactor: simplify the record to use count instead of raw content * fixup! refactor: simplify the record to use count instead of raw content * refactor: filter python files that matters * fixup! refactor: filter python files that matters

…s added (apache#55416)

…on" is added (#55416) (#64992)

boring-cyborg bot added area:dev-tools backport-to-v3-0-test labels Sep 9, 2025

Lee-W force-pushed the script-to-check-new-airflow-except branch from f67ff02 to 23f571c Compare September 13, 2025 06:45

Lee-W marked this pull request as ready for review September 13, 2025 06:52

Lee-W requested review from ashb, gopidesupavan and potiuk as code owners September 13, 2025 06:52

jscheffl reviewed Sep 13, 2025

View reviewed changes

eladkal reviewed Sep 13, 2025

View reviewed changes

scripts/ci/prek/check_new_airflow_exception_usage.py Show resolved Hide resolved

eladkal reviewed Sep 13, 2025

View reviewed changes

.pre-commit-config.yaml Outdated Show resolved Hide resolved

kaxil removed the backport-to-v3-0-test label Sep 14, 2025

amoghrajesh reviewed Sep 15, 2025

View reviewed changes

.pre-commit-config.yaml Outdated Show resolved Hide resolved

scripts/ci/prek/check_new_airflow_exception_usage.py Outdated Show resolved Hide resolved

scripts/ci/prek/check_new_airflow_exception_usage.py Outdated Show resolved Hide resolved

Lee-W force-pushed the script-to-check-new-airflow-except branch 3 times, most recently from 506f630 to 9cd9e6d Compare September 22, 2025 08:04

Lee-W requested review from amoghrajesh, eladkal and jscheffl September 28, 2025 02:59

Lee-W force-pushed the script-to-check-new-airflow-except branch from 9cd9e6d to 30615ae Compare September 28, 2025 02:59

Lee-W force-pushed the script-to-check-new-airflow-except branch 4 times, most recently from 288a6a3 to 73f587c Compare October 13, 2025 02:03

Lee-W force-pushed the script-to-check-new-airflow-except branch 2 times, most recently from aaab427 to 621e7bc Compare October 13, 2025 13:02

Lee-W marked this pull request as draft October 15, 2025 10:10

Lee-W force-pushed the script-to-check-new-airflow-except branch from 621e7bc to b109e01 Compare October 22, 2025 08:41

github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Apr 4, 2026

Lee-W removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Apr 4, 2026

Lee-W force-pushed the script-to-check-new-airflow-except branch from b109e01 to 5c26c0b Compare April 7, 2026 07:53

Lee-W marked this pull request as ready for review April 7, 2026 08:05

Lee-W requested review from bugraoz93 and jason810496 as code owners April 7, 2026 08:05

shahar1 reviewed Apr 7, 2026

View reviewed changes

scripts/ci/prek/check_new_airflow_exception_usage.py Show resolved Hide resolved

scripts/ci/prek/check_new_airflow_exception_usage.py Show resolved Hide resolved

.pre-commit-config.yaml Outdated Show resolved Hide resolved

Lee-W force-pushed the script-to-check-new-airflow-except branch from 5c26c0b to c3b6c00 Compare April 7, 2026 09:16

Lee-W requested a review from shahar1 April 7, 2026 11:30

Lee-W force-pushed the script-to-check-new-airflow-except branch from 9ebcfb6 to 99ada8e Compare April 9, 2026 03:27

Lee-W added 6 commits April 9, 2026 17:33

build(prek-hook): Check whether new "raise AirflowException" is added

cf9f5b3

feat: add white list

f444fe3

refactor: simplify the record to use count instead of raw content

d574353

fixup! refactor: simplify the record to use count instead of raw content

854fcf2

refactor: filter python files that matters

47d1e7a

fixup! refactor: filter python files that matters

4724c32

Lee-W force-pushed the script-to-check-new-airflow-except branch from 99ada8e to 4724c32 Compare April 9, 2026 09:33

potiuk approved these changes Apr 9, 2026

View reviewed changes

jason810496 approved these changes Apr 9, 2026

View reviewed changes

jscheffl approved these changes Apr 9, 2026

View reviewed changes

jscheffl merged commit 6f56aa1 into apache:main Apr 9, 2026
142 checks passed

jscheffl mentioned this pull request Apr 9, 2026

Remove AirflowException from Edge Example Dag #64975

Merged

1 task

Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 10, 2026

fixup! build(prek-hook): Check whether new "raise AirflowException" i…

8b703c7

…s added (apache#55416)

Lee-W added a commit that referenced this pull request Apr 10, 2026

[v3-2-test] build(prek-hook): Check whether new "raise AirflowExcepti…

59cb280

…on" is added (#55416) (#64992)

Conversation

Lee-W commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Uh oh!

jscheffl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Lee-W commented Sep 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

potiuk commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 25, 2025

Uh oh!

Lee-W commented Feb 15, 2026

Uh oh!

github-actions bot commented Apr 4, 2026

Uh oh!

Lee-W commented Apr 7, 2026

Uh oh!

shahar1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

potiuk commented Apr 9, 2026

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

jscheffl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Lee-W commented Sep 9, 2025 •

edited

Loading

potiuk commented Oct 13, 2025 •

edited

Loading