[CI] Keep diffusion-only kernels and tests out of the srt stages - #42706

On November 2, 2026, GitHub will restrict `pull_request_target` on public repositories by default. To continue allowing the event trigger, configure an Actions policy. Learn more: https://gh.io/securely-using-pull_request_target#default-policy-for-pull_request_target

Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/labeler@v5. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/