Add runner utilization report workflow

[alisonshao]

Tracks idle/active time per runner label to understand machine utilization. Helps identify when to add more runners (high utilization) or when runners are over-provisioned (low utilization).

Runs daily at 8 AM UTC. Can also trigger manually with custom time window and label filter.

Example output:

Label	Runners	Jobs	Active (hrs)	Idle (hrs)	Utilization
1-gpu-5090	16	64	12.5	371.5	3.3% ░░░░░░░░░░
1-gpu-h200	8	156	89.2	102.8	46.5% ████░░░░░░

[gemini-code-assist]

Summary of Changes

Hello @alisonshao, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new tool to provide critical insights into the performance and cost-efficiency of GitHub Actions self-hosted runners. By automatically generating detailed utilization reports, it empowers teams to make informed decisions about scaling their runner infrastructure, ensuring optimal resource allocation and reducing unnecessary operational costs.

Highlights

• New Runner Utilization Script: Introduces a new Python script, runner_utilization_report.py, designed to analyze GitHub Actions job data and calculate self-hosted runner utilization metrics.
• Utilization Metrics: The script reports active time, idle time, and utilization percentage for runners, grouped by their custom labels. It helps identify over-provisioned or under-provisioned runner pools.
• GitHub API Integration: Leverages the gh CLI to interact with the GitHub API, fetching workflow runs, jobs, and runner details within a specified time window.
• Markdown Report Generation: Generates a formatted Markdown report that includes a summary table and interpretation guidelines for high and low utilization, and can output to stdout, a file, or GITHUB_STEP_SUMMARY.
• Configurable Parameters: Supports command-line arguments for specifying the GitHub repository, time window (in hours), and filtering runner labels.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/runner-utilization.yml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a helpful script for monitoring runner utilization. The implementation is solid, but I've identified several areas for improvement, primarily concerning the handling of API pagination, which could lead to incorrect reports. I've also noted some opportunities for performance optimization and code cleanup. Addressing these points will make the script more robust and reliable.

[gemini-code-assist]

The get_runners function does not handle pagination. It only fetches the first 100 runners because of the per_page=100 parameter. If the repository has more than 100 self-hosted runners, the report will be incomplete and inaccurate because it will not account for all runners. You should implement a pagination loop, similar to the one in get_workflow_runs, to fetch all runners.

Suggested change

	data = run_gh_command([f"repos/{repo}/actions/runners", "-f", "per_page=100"])
	return data.get("runners", [])
	all_runners = []
	page = 1
	while True:
	data = run_gh_command([
	f"repos/{repo}/actions/runners",
	"-f", "per_page=100",
	"-f", f"page={page}",
	])
	runners = data.get("runners", [])
	all_runners.extend(runners)
	if len(runners) < 100:
	break
	page += 1
	return all_runners

[gemini-code-assist]

The while loop for fetching workflow runs has a hardcoded safety limit of 10 pages. For a very active repository, there could be more than 1000 workflow runs (10 pages * 100 per page) in the specified time window. If this limit is reached, the script will silently stop fetching runs, leading to incomplete data and underestimated utilization metrics. It's safer to remove this limit and rely on the API's pagination mechanism to determine when to stop.

[gemini-code-assist]

Similar to get_workflow_runs, this function has a hardcoded safety limit of 5 pages for fetching jobs. A workflow run with a large matrix strategy could easily exceed 500 jobs (5 pages * 100 per page). Reaching this limit will cause the script to under-report active time for that run. It's better to remove this arbitrary limit and let the pagination loop complete naturally based on the number of results returned by the API.

[gemini-code-assist]

The current approach to map a job's runner to a label involves iterating through all labels for every job. This has a complexity of O(Jobs * Labels) and can be inefficient for a large number of jobs and labels. A more performant approach is to create a reverse mapping from runner name to label once, before processing the jobs. This changes the lookup to an O(1) operation inside the loop.

First, create this map after label_runners is finalized (around line 108):

runner_to_label_map = {
    runner_name: label
    for label, runner_list in label_runners.items()
    for runner_name in runner_list
}

Then, you can replace this loop with a more efficient dictionary lookup as suggested.

            if label := runner_to_label_map.get(runner_name):
                label_jobs[label].append(job_info)

[gemini-code-assist]

The list of default labels to ignore ("self-hosted", "Linux", "X64") is hardcoded. This could be brittle if GitHub changes these default labels in the future. For better maintainability, consider moving this list to a constant defined at the top of the file.

For example, you could add this at the top of the script:

DEFAULT_LABELS_TO_IGNORE = {"self-hosted", "Linux", "X64"}

And then use the constant here.

Suggested change

	if label_name not in ["self-hosted", "Linux", "X64"]: # Skip default labels
	if label_name not in DEFAULT_LABELS_TO_IGNORE: # Skip default labels

[gemini-code-assist]

The runner_jobs dictionary is initialized here and populated on line 138, but its value is never read or used. This is dead code and should be removed to improve clarity. You should also remove line 138 where it is populated.

[gemini-code-assist]

This line populates the runner_jobs dictionary, which is unused. This line should be removed along with the dictionary's initialization on line 110.

[gemini-code-assist]

The window_start variable is calculated but never used elsewhere in the code. This unused variable should be removed to improve code clarity.

[alisonshao]

24 hours: https://github.com/sgl-project/sglang/actions/runs/21083722190

[alisonshao]

added queue time: https://github.com/sgl-project/sglang/actions/runs/21084406194