Skip to content

Commit

Permalink
Bundle action using esbuild
Browse files Browse the repository at this point in the history
Remove node_modules from repository to reduce cost to download
repository.

Because node_modules is no longer included:
* If `npm` isn't available (e.g. in a container), install it
* Run `npm install` before performing various tasks

Change pr-checks to not be particularly picky about the generated
content because it will differ between different versions as everything
is bundled together.
  • Loading branch information
jsoref committed Nov 6, 2024
1 parent cbe1897 commit 594f2e3
Show file tree
Hide file tree
Showing 18,887 changed files with 3,125 additions and 4,740,773 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
6 changes: 6 additions & 0 deletions .github/actions/prepare-test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ outputs:
runs:
using: composite
steps:
- name: npm install
shell: bash
run: |
if command -v npm >/dev/null 2>/dev/null; then
npm ci
fi
- name: Move codeql-action
shell: bash
run: |
Expand Down
4 changes: 4 additions & 0 deletions .github/actions/update-bundle/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ runs:
shell: bash
run: npm install -g ts-node

- name: Install
shell: bash
run: npm ci

- name: Run update script
working-directory: ${{ github.action_path }}
shell: bash
Expand Down
19 changes: 19 additions & 0 deletions .github/workflows/pr-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ jobs:
- name: Checkout
uses: actions/checkout@v4

- name: Install
run: npm install

- name: Lint
id: lint
run: npm run-script lint-ci
Expand All @@ -52,6 +55,16 @@ jobs:
# `npm install` on Linux.
npm install
(
echo '*/*-action.js';
echo '*/*-action-post.js'
) >> .gitignore
for action in $(
find * -mindepth 1 -maxdepth 1 -type f -name action.yml
); do
git rm -f "$(dirname "$action")"/*-action*.js
done
if [ ! -z "$(git status --porcelain)" ]; then
git config --global user.email "[email protected]"
git config --global user.name "github-actions[bot]"
Expand Down Expand Up @@ -112,6 +125,12 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: npm install
run: |
npm ci
- name: Build
run: |
npm run build
- name: npm test
run: |
# Run any commands referenced in package.json using Bash, otherwise
Expand Down
8 changes: 5 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Ignore for example failing-tests.json from AVA
node_modules/.cache/
# actions are bundled to make this repository lightweight for consumers
node_modules/
# lib is generated by tsc
lib
# Java build files
.gradle/
*.class
Expand All @@ -8,4 +10,4 @@ node_modules/.cache/
# eslint sarif report
eslint.sarif
# for local incremental compilation
tsconfig.tsbuildinfo
tsconfig.tsbuildinfo
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Note that the only difference between `v2` and `v3` of the CodeQL Action is the
## [UNRELEASED]

- The CodeQL Action now downloads bundles compressed using Zstandard on GitHub Enterprise Server when using Linux or macOS runners. This speeds up the installation of the CodeQL tools. This feature is already available to GitHub.com users. [#2573](https://github.com/github/codeql-action/pull/2573)
- The CodeQL Action no longer includes node_modules. This should drammatically decrease the download size/increase download speed for `uses: github/codeql-action@...`.

## 3.27.0 - 22 Oct 2024

Expand Down
13 changes: 4 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,15 @@ Before you start, ensure that you have a recent version of node (16 or higher) i

### Common tasks

* Transpile the TypeScript to JavaScript: `npm run build`. Note that the JavaScript files are committed to git.
* Run tests: `npm run test`. You’ll need to ensure that the JavaScript files are up-to-date first by running the command above.
* Run the linter: `npm run lint`.
* Set up node: `npm ci`
* Transpile the TypeScript to JavaScript: `npm run build`. Note that the bundled action files are committed to git.
* Run tests: `npm run test`. You’ll need to ensure that the `node_modules` are available andJavaScript files are up-to-date first by running the commands above.
* Run the linter: `npm run lint` (requires the first command).

This project also includes configuration to run tests from VSCode (with support for breakpoints) - open the test file you wish to run and choose "Debug AVA test file" from the Run menu in the Run panel.

You may want to run `tsc --watch` from the command line or inside of vscode in order to ensure build artifacts are up to date as you are working.

### Checking in compiled artifacts and `node_modules`

Because CodeQL Action users consume the code directly from this repository, and there can be no build step during an GitHub Actions run, this repository contains all compiled artifacts and node modules. There is a PR check that will fail if any of the compiled artifacts are not up to date. Compiled artifacts are stored in the `lib/` directory. For all day-to-day development purposes, this folder can be ignored.

Only run `npm install` if you are explicitly changing the set of dependencies in `package.json`. The `node_modules` directory should be up to date when you check out, but if for some reason, there is an inconsistency use `npm ci && npm run removeNPMAbsolutePaths` to ensure the directory is in a state consistent with the `package-lock.json`. Note that due to a macOS-specific dependency, this command should be run on a macOS machine. There is a PR check to ensure the consistency of the `node_modules` directory.

### Running the action

To see the effect of your changes and to test them, push your changes in a branch and then look at the [Actions output](https://github.com/github/codeql-action/actions) for that branch. You can also exercise the code locally by running the automated tests.
Expand Down
4 changes: 2 additions & 2 deletions analyze/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,5 +92,5 @@ outputs:
description: The ID of the uploaded SARIF file.
runs:
using: node20
main: "../lib/analyze-action.js"
post: "../lib/analyze-action-post.js"
main: "analyze-action.js"
post: "analyze-action-post.js"
379 changes: 379 additions & 0 deletions analyze/analyze-action-post.js

Large diffs are not rendered by default.

230 changes: 230 additions & 0 deletions analyze/analyze-action.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion autobuild/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ inputs:
required: false
runs:
using: node20
main: '../lib/autobuild-action.js'
main: 'autobuild-action.js'
180 changes: 180 additions & 0 deletions autobuild/autobuild-action.js

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ export default [
"tests/**/*",
"eslint.config.mjs",
".github/**/*",
"*/*-action.js",
"*/*-action-post.js",
],
},
...fixupConfigRules(
Expand Down
4 changes: 2 additions & 2 deletions init/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -147,5 +147,5 @@ outputs:
description: The version of the CodeQL binary used for analysis
runs:
using: node20
main: '../lib/init-action.js'
post: '../lib/init-action-post.js'
main: 'init-action.js'
post: 'init-action-post.js'
379 changes: 379 additions & 0 deletions init/init-action-post.js

Large diffs are not rendered by default.

187 changes: 187 additions & 0 deletions init/init-action.js

Large diffs are not rendered by default.

Loading

0 comments on commit 594f2e3

Please sign in to comment.