Merge branch 'main' into issue-22492

.dockerignore

@@ -7,7 +7,6 @@ docs/
 node_modules/
 script/
 tests/
-lib/rest/static/dereferenced
 # Folder is cloned during the preview + prod workflows, the assets are merged into other locations for use before the build
 docs-early-access/
 # During the preview deploy untrusted user code may be cloned into this directory

.github/actions-scripts/enterprise-server-issue-templates/deprecation-issue.md

@@ -9,15 +9,15 @@ The following large repositories are used throughout this checklist, it may be u
 - `github/github`
 - `github/docs-internal`
 
-Additionally, you may want to download:
+Additionally, download:
 
 - [Azure Storage Explorer](https://aka.ms/portalfx/downloadstorageexplorer)
 
 ## Step 0: Remove deprecated version numbers from docs-content issue forms
 
 **Note**: This step can be performed independently of all other steps, and can be done several days before or along with the other steps.
 
-- [ ] In the `docs-content` repo, remove the deprecated GHES version number from the `options` list in the following files (in the `.github/ISSUE_TEMPLATE/` directory): [`release-tier-1-or-2-tracking.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tier-1-or-2-tracking.yml) and [`release-tier-3-or-tier-4.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tier-3-or-tier-4.yml).
+- [ ] In the `docs-content` repo, remove the deprecated GHES version number from the `options` list in [`release-tracking.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tracking.yml).
 - [ ] When the PR is approved, merge it in. This can be merged independently from all other steps.
 
 ## Step 1: Scrape the docs and archive the files
@@ -40,7 +40,7 @@ Additionally, you may want to download:
 
   **Note:** You can pass the `--dry-run` flag to scrape only the first 10 pages plus their redirects for testing purposes. **If you use the dry run command, be sure to run the full script without `--dry-run` before you commit the changes.**
 
-## Step 3: Manually remove the search input from the archived docs
+## Step 2: Manually remove the search input from the archived docs
 
 - [ ] Search for `site-search-input` in the compressed Javascript files (should find the file in the `_next` directory). When you find it, use something like https://beautifier.io/ or VSCode to reformat it to be readable. To reformat using VSCode, use the "Format document" option or <kbd>Shift</kbd>+<kbd>Option</kbd>+<kbd>F</kbd>. Find `site-search-input` in the file, the result will be enclosed in a function that looks something like... `(0,f.jsx)("input",{"data-testid":"site-search-input",` Delete the entire function. For example, this:
   ```
@@ -88,7 +88,7 @@ Additionally, you may want to download:
   ```
 - [ ] Save the file. If using beautifier, copy and paste the updated file back into your temp directory with the scraped archive content.
 
-## Step 2: Upload the scraped content directory to Azure storage
+## Step 3: Upload the scraped content directory to Azure storage
 
 - [ ] Log in to the Azure portal from Okta. Navigate to the [githubdocs Azure Storage Blob resource](https://portal.azure.com/#@githubazure.onmicrosoft.com/resource/subscriptions/fa6134a7-f27e-4972-8e9f-0cedffa328f1/resourceGroups/docs-production/providers/Microsoft.Storage/storageAccounts/githubdocs/overview).
 - [ ] Click the "Open in Explorer" button to the right of search box. If you haven't already, click the download link to download "Microsoft Azure Storage Explorer." To login to the app, click the plug icon in the left sidebar and click the option to "add an azure account." When you login, you'll need a yubikey to authenticate through Okta.
@@ -97,7 +97,11 @@ Additionally, you may want to download:
 - [ ] Check the log to ensure all files were uploaded successfully.
 - [ ] Remove the temporarily created directory from your `github/docs-internal` checkout.
 
-## Step 4: Deprecate the version in docs-internal
+## Step 4: Create a tag and long-running branch
+
+Create a new tag for the most recent commit on the `main` branch so that we can keep track of where in commit history we removed the GHES release. Create a tag called `enterprise-<release number>-release`. Then from that commit on `main` create a new branch called `enterprise-<release number>-release`. This branch will be long-lived and used to rerender and scrape content that is added after a release is deprecated.
+
+## Step 5: Deprecate the version in docs-internal
 
 In your `docs-internal` checkout:
 
@@ -111,7 +115,7 @@ You can test that the static pages were generated correctly on localhost and on
 
 Poke around several pages, ensure that the stylesheets are working properly, images are rendering properly, and that the search functionality was disabled.
 
-## Step 5: Remove static files for the version
+## Step 6: Remove static files for the version
 
 **Note:** We do not remove the old content for GHES release notes. New release notes can be added after we perform a deprecation in some rare cases, and not removing this content makes it easier for us to re-scrape the content to add to Azure Blob Storage.
 
@@ -121,7 +125,7 @@ Poke around several pages, ensure that the stylesheets are working properly, ima
 - [ ] Open a new PR.
 - [ ] Get a review from docs-engineering and merge. This step can be merged independently from step 6. The purpose of splitting up steps 5 and 6 is to focus the review on specific files.
 
-## Step 6: Remove the liquid conditionals and content for the version
+## Step 7: Remove the liquid conditionals and content for the version
 
 - [ ] In your `docs-internal` checkout, create a new branch `remove-<version>-markup` branch: `git checkout -b remove-<version>-markup` (you can branch off of `main` or from your `deprecate-<version>` branch, up to you).
 - [ ] Remove the outdated Liquid markup and frontmatter.
@@ -131,17 +135,31 @@ Poke around several pages, ensure that the stylesheets are working properly, ima
   - [ ] Debug any test failures or unexpected results -- it's very likely manual updates will be necessary, the script does a lot of work but doesn't automate everything and can't 100% replace human intent.
 - [ ] When the PR is approved, merge it in to complete the deprecation. This can be merged independently from step 5.
 
-## Step 7: Deprecate the OpenAPI description in `github/github`
+## Step 8: Deprecate the OpenAPI description in `github/github`
 
 - [ ] In `github/github`, edit the release's config file in `app/api/description/config/releases/`, and change `deprecated: false` to `deprecated: true`.
 - [ ] Open a new PR, and get the required code owner approvals. A docs-content team member can approve it for the docs team.
 - [ ] When the PR is approved, [deploy the `github/github` PR](https://thehub.github.com/epd/engineering/devops/deployment/deploying-dotcom/). If you haven't deployed a `github/github` PR before, work with someone that has -- the process isn't too involved depending on how you deploy, but there are a lot of details that can potentially be confusing as you can see from the documentation.
 
 **Note**: you can do this step independently of the other steps after a GHES version is deprecated since it should no longer get updates in github/github. You should plan to get this PR merged as soon as possible, otherwise if you wait too long our OpenAPI automation may re-add the static files that you removed in step 5.
 
-## Step 5: Continue to deprecate the version in docs-internal
+## Step 9: Continue to deprecate the version in docs-internal
 
 - [ ] Open a new PR. Make sure to check the following:
   - [ ] Tests are passing (you may need to include the changes in step 6 to get tests to pass).
   - [ ] The deprecated version renders in preview as expected. You should be able to navigate to `docs.github.com/enterprise/<DEPRECATED VERSION>` to access the docs. You should also be able to navigate to a page that is available in the deprecated version and change the version in the URL to the deprecated version, to test redirects.
-  - [ ] The new oldest supported version renders on staging as expected. You should see a banner on the top of every page for the oldest supported version that notes when the version will be deprecated.
+  - [ ] The new oldest supported version renders on staging as expected. You should see a banner on the top of every page for the oldest supported version that notes when the version will be deprecated.
+
+## Re-scraping a page or all pages
+
+Occasionally, a change will need to be added to our archived enterprise versions. If this occurs, you can check out the `enterprise-<release number>-release` branch and re-scrape the page or all pages using `script/enterprise-server-deprecations/archive-version.js`. To scrape a single page you can use the `—page <page relative path>` option.
+
+For each language, upload the new file to Azure blob storage in the `enterprise` container.
+
+After uploading the new files, you will need to purge the Fastly cache for the single page. From Okta, go to Fastly and select `docs`. Click `Purge` then `Purge URL`. If you need to purge a whole path, just do a `Purge All`
+
+![](/assets/images/fastly_purge.jpg)
+
+Enter the URL or path and do a soft purge.
+
+![](/assets/images/fastly_purge_url.jpg)

.github/actions-scripts/enterprise-server-issue-templates/release-issue.md

@@ -18,7 +18,7 @@ If you aren't comfortable going through the steps alone, sync up with a docs eng
     ```
     script/update-enterprise-dates.js
     ```
-- [ ] Create REST files based on previous version. Copy the latest GHES version of the dereferenced file from `lib/rest/static/dereferenced` to a new file in the same directory for the new GHES release. Ex, `cp lib/rest/static/dereferenced/ghes-3.4.deref.json lib/rest/static/dereferenced/ghes-3.5.deref.json`. Then run `script/rest/update-files.js --decorate-only` and check in the resulting files.
+- [ ] Create REST files based on previous version. Copy the latest GHES version of the decorate file from `lib/rest/static/decorated` to a new file in the same directory for the new GHES release. Ex, `cp lib/rest/static/decorated/ghes-3.4.json lib/rest/static/decorated/ghes-3.5.json`.
 
 - [ ] Create GraphQL files based on previous version:
 
@@ -76,16 +76,6 @@ If you aren't comfortable going through the steps alone, sync up with a docs eng
 
 ### Troubleshooting
 
-#### `OpenAPI dev mode check / check-schema-versions` failures
-
-If the `OpenAPI dev mode check / check-schema-versions` check fails with the following message:
-
->  :construction::warning: Your decorated and dereferenced schema files don't match. Ensure you're using decorated and dereferenced schemas from the automatically created pull requests by the 'github-openapi-bot' user. For more information, see 'script/rest/README.md'
-
-- run `git checkout origin/main lib/rest/static/*`
-- run `script/enterprise-server-releases/create-rest-files.js --oldVersion enterprise-server@<LATEST PUBLIC RELEASE NUMBER> --newVersion enterprise-server@<NEW RELEASE NUMBER>`
-- push the resulting changes
-
 #### `Node.js tests / test content` failures
 
 If the `Node.js tests / test content` check fails with the following message, the `lib/enterprise-dates.json` file is not up-to-date:
@@ -104,7 +94,7 @@ This file should be automatically updated, but you can also run `script/update-e
 - [ ] [Freeze the repos](https://github.com/github/docs-content/blob/main/docs-content-docs/docs-content-workflows/freezing.md) at least 1-2 days before the release, and post an announcement in Slack so everybody knows. It's helpful to freeze the repos before doing the OpenAPI merges to avoid changes to the megabranch while preparing and deploying.
 - [ ] Alert the Neon Squad (formally docs-ecosystem team)  1-2 days before the release to deploy to `github/github`. A PR should already be open in `github/github`, to change the OpenAPI schema config `published` to `true` in `app/api/description/config/releases/ghes-<NEXT RELEASE NUMBER>.yaml`. They will need to:
   - [ ] Get the required approval from `@github/ecosystem-api-reviewers` then deploy the PR to dotcom. This process generally takes 30-90 minutes.
-  - [ ] Once the PR merges, make sure that the auto-generated PR titled "Update OpenAPI Descriptions" in doc-internal contains both the dereferenced and decorated JSON files for the new GHES release. If everything looks good, merge the "Update OpenAPI Description" PR into the GHES release megabranch. **Note:** Be careful about resolving the conflicts correctly—you may wish to delete the existing OpenAPI files for the release version from the megabranch (that is, delete the GHES release version `lib/rest/static` decorated and dereferenced JSON files), so there are no conflicts to resolve and to ensure that the incoming artifacts are the correct ones.
+  - [ ] Once the PR merges, make sure that the auto-generated PR titled "Update OpenAPI Descriptions" in doc-internal contains the decorated JSON files for the new GHES release. If everything looks good, merge the "Update OpenAPI Description" PR into the GHES release megabranch. **Note:** Don't attempt to resolve conflicts for changes to the `lib/rest/static/decorated` files. Instead delete the existing OpenAPI files for the release version from the megabranch (that is, revert the changes to the `lib/rest/static` decorated JSON files, e.g., from the megabranch do a `git checkout origin/main lib/rest/static/*`), so there are no conflicts to resolve and to ensure that the incoming artifacts are the correct ones.
 - [ ] Alert the Ecosystem-API team in #ecosystem-api about the pending release freeze and incoming blocking review of OpenAPI updates in the public REST API description (the `rest-api-descriptions` repo). They'll need to block any future "Update OpenAPI Descriptions" PRs in the public REST API description until after the ship.
   - [ ] Add a blocking review to the auto-generated "Update OpenAPI Descriptions" PR in the public REST API description. (You or they will remove this blocking review once the GHES release ships.)
 
@@ -129,4 +119,4 @@ This file should be automatically updated, but you can also run `script/update-e
   - [ ] You (or they) can now remove your blocking review on the auto-generated "Update OpenAPI Descriptions" PR in public REST API description (the `rest-api-descriptions` repo). (although it's likely newer PRs have been created since yours with the blocking review, in which case the Ecosystem-API team will close your PR and perform the next step on the most recent PR).
   - [ ] The Ecosystem-API team will merge the latest auto-generated "Update OpenAPI Descriptions" PR (which will contain the OpenAPI schema config that changed `published` to `true` for the release).
 - [ ] After unfreezing, if there were significant or highlighted GraphQL changes in the release, consider manually running the [GraphQL update workflow](https://github.com/github/docs-internal/actions/workflows/update-graphql-files.yml) to update our GraphQL schemas. By default this workflow only runs once every 24 hours.
-- [ ] After the release, in the `docs-content` repo, add the now live version number to the "Specific GHES version(s)" section in the following files: [`.github/ISSUE_TEMPLATE/release-tier-1-or-2-tracking.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tier-1-or-2-tracking.yml) and [`.github/ISSUE_TEMPLATE/release-tier-3-or-tier-4.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tier-3-or-tier-4.yml). When the PR is approved, merge it in. 
+- [ ] After the release, in the `docs-content` repo, add the now live version number to the "Specific GHES version(s)" section in [`.github/ISSUE_TEMPLATE/release-tracking.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tracking.yml). When the PR is approved, merge it in. 

.github/actions-scripts/purge-fastly-edge-cache-per-language.js

@@ -0,0 +1,33 @@
+#!/usr/bin/env node
+
+import { languageKeys } from '../../lib/languages.js'
+
+import { makeLanguageSurrogateKey } from '../../middleware/set-fastly-surrogate-key.js'
+import purgeEdgeCache from '../../script/deployment/purge-edge-cache.js'
+
+// This covers things like `/api/webhooks` which isn't language specific.
+await purgeEdgeCache(makeLanguageSurrogateKey())
+
+const languages = process.env.LANGUAGES
+  ? languagesFromString(process.env.LANGUAGES)
+  : // Make sure `en` is first because contributors write mostly in English
+    // and they're most likely wanting to see their landed changes appear
+    // in production as soon as possible.
+    languageKeys.sort((a) => (a === 'en' ? -1 : 1))
+
+for (const language of languages) {
+  await purgeEdgeCache(makeLanguageSurrogateKey(language))
+}
+
+function languagesFromString(str) {
+  const languages = str
+    .split(/,/)
+    .map((x) => x.trim())
+    .filter(Boolean)
+  if (!languages.every((lang) => languageKeys.includes(lang))) {
+    throw new Error(
+      `Unrecognized language code (${languages.find((lang) => !languageKeys.includes(lang))})`
+    )
+  }
+  return languages
+}