Add backport reminder and reorganize the backporting actions.

[NickDris]

Introducing backport.py script that interacts with GH API to

• Label merged PRs lacking a 'backport pending' or a version vX(.Y) label.
• Post a backporting reminder as a comment in the PR.

Github Workflow changes:

• Backporting action is separated from the rest of backporting, like it was initially.
• Backporting reminder and labeler added in a workflow that can be also run manually.

Backport reminder workflow:

• Mainly uses backport.py to add labels and post reminders in merged PRs.
• Runs either on schedule, manually or on closing PR, discerning the mode of operation
  with the github.event_name.
• Based on the mode of operation, backport.py either processes a single PR, or
  fetches the list of PRs, merged the last X days from GH API
• Checking PR merged status is made inside backport.py.

Inputs:

• lookback_days: Look for merged PRs within a lookback window.
• pending_label_age_days: Post reminder based on how old is the last reminder.

[NickDris]

This needs a bit of optimization, in order to apply the DRY principle, as the functionality added in #888 has some significant overlaps.
Please wait before reviewing it.

[gbanasiak]

Initial high-level feedback/questions:

1. Having a CLI tool which can be used in GH actions as well as independently is a great idea.
2. A dry-run option is highly recommended. It would allow for testing (part of PR review).
3. Token should not be passed through command-line option, see below.
4. The tool uses standard libraries only it seems. Is this intentional? Example: why argparse and not something newer, like click?
5. backport.action.yml still references scripts/backport-pending.py which gets removed.

[NickDris]

All of the review comments were much help. --dry-run added, --token removed and coded to be set only through the environment and backport.action.yml cleaned.
Using click instead of argparse was a pretty good idea, since it is in alignment with our other tools, I followed that.
I also added logging and provided a --debug option, which for now prints the Github API calls.
Thanks a lot for reviewing @gbanasiak

[gbanasiak]

Many thanks for these exhaustive tests. Personally I do find some of them too complex to follow easily. One thing that really adds to the complexity is the attempt to verify entire sequences of GH API calls. I think I would keep this simpler looking for key API call depending on the test case. But that's fine, let's not spend more time simplifying these.

I'd like to request the following 2 changes:

• change filtering logic for reminders - they should apply to PRs with backport pending label
• make newly added tests discoverable by pytest (and hatch -v -e unit run test), they are currently not covered; the problem is all .* folders are by default excluded from pytest due to default norecursedirs setting (doc) - my suggestion would be to rename .ci to tools and remove scripts sub-dir (tools/scripts seems redundant now, we only have 2 scripts); we also need an extra __init__.py file under tools to make pytest happy

Reminder comments will accumulate in time, especially in case of PRs waiting for the first supported release. It would be nice to remove older comments after adding a new one.

There's a bunch of static type checker (mypy) failures. I've started mentioning them in review but gave up quickly. I recommend using type checker integrated with IDE for now.

My heart sank for a moment when looking at dry-run outputs. I haven't noticed this earlier :)

# these print-outs do not mention dry-run and "added" suggests it was already done
% ./tools/scripts/backport.py --dry-run --repo=elastic/rally-tracks label --lookback-days=7 
2025-11-18 17:21:29,111 INFO __main__ Added label 'backport pending' to PR #917
2025-11-18 17:21:29,112 INFO __main__ Added label 'backport pending' to PR #916
2025-11-18 17:21:29,112 INFO __main__ Added label 'backport pending' to PR #915
2025-11-18 17:21:29,112 INFO __main__ Added label 'backport pending' to PR #904

# this output sometimes says "would" which is nice but mixes this with "posted", confusing
% ./tools/scripts/backport.py --dry-run --repo=elastic/rally-tracks remind --lookback-days=7
2025-11-18 17:27:36,473 INFO __main__ prefetched=12 lookback_days=7 pending_reminder_age_days=7 now=2025-11-18T16:27:36.473747+00:00 threshold=2025-11-11T16:27:36.473747+00:00
2025-11-18 17:27:36,802 INFO __main__ Would post comment to PR #918
2025-11-18 17:27:36,802 INFO __main__ PR #918: initial reminder posted
2025-11-18 17:27:36,802 INFO __main__ PR #917: cooling period not elapsed)
2025-11-18 17:27:36,802 INFO __main__ PR #916: cooling period not elapsed)
2025-11-18 17:27:36,802 INFO __main__ PR #915: cooling period not elapsed)
2025-11-18 17:27:37,146 INFO __main__ Would post comment to PR #911
2025-11-18 17:27:37,146 INFO __main__ PR #911: initial reminder posted
2025-11-18 17:27:37,491 INFO __main__ Would post comment to PR #910
2025-11-18 17:27:37,491 INFO __main__ PR #910: initial reminder posted
2025-11-18 17:27:37,810 INFO __main__ Would post comment to PR #909
2025-11-18 17:27:37,811 INFO __main__ PR #909: initial reminder posted
2025-11-18 17:27:38,212 INFO __main__ Would post comment to PR #908
2025-11-18 17:27:38,212 INFO __main__ PR #908: initial reminder posted
2025-11-18 17:27:38,656 INFO __main__ Would post comment to PR #907
2025-11-18 17:27:38,657 INFO __main__ PR #907: initial reminder posted
2025-11-18 17:27:38,657 INFO __main__ PR #904: cooling period not elapsed)
2025-11-18 17:27:39,065 INFO __main__ Would post comment to PR #902
2025-11-18 17:27:39,065 INFO __main__ PR #902: initial reminder posted
2025-11-18 17:27:39,455 INFO __main__ Would post comment to PR #898
2025-11-18 17:27:39,455 INFO __main__ PR #898: initial reminder posted

[fressi-elastic]

I have the filing that this PR got lot of unnecessary complexity and technical debt just because we didn't reused pre-existing 3rd party code [1]. It is nice to write code to learn and get code reviewed to be able to exchange ideas and experiences. But for something like this I would have preferred using a pre-existing library. It would have been much less work than writing (and testing) this code. Given the quality of the code of this PR is not very critical to me it LGTM but please read my comments so you can do different the next time (I hope I am not been rude, it has never been my intention).

[1] https://github.com/PyGithub/PyGithub

[gbanasiak]

@fressi-elastic

I have the filing that this PR got lot of unnecessary complexity and technical debt just because we didn't reused pre-existing 3rd party code

Extra context: We've been there before with discussion around click. The decision was taken to use standard Python library only. It eliminates extra dependencies in GH workflows using this script.

[NickDris]

@elasticmachine update branch

[elasticmachine]

merge conflict between base and head

[NickDris]

I have the filing that this PR got lot of unnecessary complexity and technical debt just because we didn't reused pre-existing 3rd party code [1]. It is nice to write code to learn and get code reviewed to be able to exchange ideas and experiences. But for something like this I would have preferred using a pre-existing library. It would have been much less work than writing (and testing) this code. Given the quality of the code of this PR is not very critical to me it LGTM but please read my comments so you can do different the next time (I hope I am not been rude, it has never been my intention).

[1] https://github.com/PyGithub/PyGithub

Nothing rude about it. I do have the same expectations from myself in the future but it's a learning process and you are actually helping it. Thank you .
I will read your comments carefully and change some parts that I consider critical.

[gbanasiak]

LGTM, I've left final cosmetic changes after testing

[esbenchmachine]

💚 All backports created successfully

Status	Branch	Result
✅	9.2

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details