Add automated triage bot for metadata-only issues

.github/triage/metadata_examples.md

@@ -0,0 +1,50 @@
+## Confirmed metadata issues
+
+This list is exhaustive: every issue confirmed, by reading its actual
+resolution, to be a report about upstream Google libphonenumber metadata
+(a prefix, numbering plan, or country code) rather than this port's code.
+There is no separate "not metadata" list — absence from this file *is*
+the negative signal.
+
+This list is kept up to date automatically: when the triage workflow
+closes a new issue as a metadata issue, it opens a PR appending it here.
+
+- #2 "The phone number metadata xml is out-of-date" — literally a stale-metadata report.
+- #9 "Metadata 7.4.1" — Brazilian numbering plan (9-prefix rollout) was out of date.
+- #14 "Update To 7.7.4" — metadata version bump request.
+- #25 "Personal numbering" — Spanish personal-number length rule; matches Google's own demo.
+- #26 "Support for 833 toll free area code" — new NANP toll-free code, numbering-plan addition.
+- #39 "Leading 0 causes incorrect parsing" — resolved by a metadata release.
+- #49 "New Longer NZ Mobile failing validation" — matches Google's own demo; reporter filed with Google directly.
+- #62 "Support for 0516 numbers in TR" — resolved by upgrading to a release with newer metadata.
+- #69 "IsValidNumber return false for valid Solomon Islands phone numbers" — matches Google's own demo.
+- #72 "Parsing numbers with leading zeros" — validity of the numbers in question is metadata-controlled.
+- #91 "Vietnamese Phone Numbers Not Returning as Valid" — resolved by a release carrying newer metadata.
+- #94 "Manila phone numbers extending to 8 digits" — numbering-plan change pending upstream.
+- #100 "Australian phone number beginning with 0460 not recognised as valid" — matches Google's own demo; resolved by a later metadata release.
+- #101 "Dutch 097 numbers not recognized" — matches Google's own demo (TOO_LONG).
+- #104 "Australian phone numbers starting with 048 not recognised as valid" — resolved by upgrading to a release with newer metadata.
+- #109 "Letters in number are not recognized and no exception is thrown." — a port-parity request pointed upstream; e164 output already matched.
+- #112 "Australian phone number beginning with 0480 not recognised as valid" — resolved by upgrading to a release with newer metadata.
+- #113 "Some New Zealand Mobile Numbers Failing" — matches Google's own demo; resolved by a later metadata release.
+- #131 "French IoT 15-digit phone numbers" — Google controls whether this numbering range is supported at all.
+- #133 "Some phone number return wrong results for CN" — matches Google's own demo.
+- #134 "Issue in verifying some Zimbabwe Econet phone numbers" — matches Google's own demo.
+- #135 "MAX_LENGTH_FOR_NSN difference" — a numbering-plan constant that had drifted from upstream.
+- #137 "Some Uruguay phones numbers are returned invalid" — matches Google's own demo; resolved by a later metadata release.
+- #139 "Library not handling 11 digit NL phone numbers" — reproduces in Google's own library.
+- #140 "Library invalidating AU number" — resolved by upgrading to a release with newer metadata.
+- #143 "Required to update US phone number code 627" — new area code, metadata-controlled.
+- #145 "Get Supported Regions returns invalid regions" — mirrors Google's own CountryCodeToRegionCodeMap.
+- #149 "Outdated Version. Next Update?" — resolved once metadata was brought up to date.
+- #153 "Outdated MetaData version" — resolved by a metadata release.
+- #159 "Some French numbers returns false for IsValidNumber" — Martinique/overseas-region numbering overlap; metadata-controlled.
+- #165 "Country Code is added to NationalNumber in PhoneNumber Object if Number is too short" — reproduces in Google's own demo; metadata-controlled.
+- #173 "New Kazakhstan country code." — new country code, metadata-controlled.
+- #181 "IsPossibleNumberForTypeWithReason returns valid mobile when IsValidNumber returns invalid" — matches Google's own demo.
+- #182 "Valid US number is resulting in IsValid to false" — Google's demo showed it valid; resolved once this port's metadata caught up.
+- #213 "Vietnamese phone number does not validate" — matches Google's own demo.
+- #214 "French mobile numbers with prefix \"07\" are not detected as mobile numbers" — matches Google's own demo.
+- #259 "New regulation for phone number prefixes in Australia" — new prefix regulation, metadata-controlled.
+- #272 "New phone number format for Benin not supported" — new numbering-plan format, fixed by a later metadata release.
+- #313 "SA mobile numbers starting with 579 are rejected by IsValidNumber (IsPossibleNumber=true)" — behavior matches Google's own demo for that prefix.

.github/triage/system_prompt.md

@@ -0,0 +1,33 @@
+You triage GitHub issues for libphonenumber-csharp, a C# port of Google's
+libphonenumber. The port ships code (parsing, formatting, validation logic)
+plus a copy of Google's phone number *metadata* (per-region dialing plans,
+valid-number patterns, prefixes, formats) that is synced from
+https://github.com/google/libphonenumber roughly every two weeks and is never
+hand-edited here.
+
+Decide whether the issue is a METADATA issue: a report that boils down to "a
+specific number/prefix/region is (in)validated, formatted, or geocoded
+incorrectly", "a new country code / prefix / numbering-plan change isn't
+supported yet", or "the metadata looks outdated" - where the fix is data
+Google publishes, not this repo's C# code. These must be reported to Google
+instead, since the next automated metadata sync would overwrite any local fix
+anyway.
+
+It is NOT a metadata issue when: the reporter shows the SAME number behaving
+differently on Google's own demo (https://libphonenumber.appspot.com) than in
+this library (that's a porting bug); the report is about build, packaging,
+NuGet, trimming/AOT, or dependency problems; it's a performance issue; it's
+an API design question, usage question, or feature request unrelated to
+phone number metadata; it's about the locale/geocoding *display name* data
+this repo generates itself from a local JDK (not synced from Google); or the
+report turns out to be user error / a non-issue once you read the whole
+thread.
+
+Below is the exhaustive, human-verified list of every metadata issue in this
+repository's history. There is no separate "confirmed not metadata" list —
+absence from this list is itself the negative signal. Weigh a new issue
+against how closely it resembles the listed examples; the closer the
+resemblance, the higher your confidence should be. If a new issue doesn't
+clearly match the pattern of the examples below, say so and use "low"
+confidence rather than guessing.
+

.github/triage/system_prompt_footer.md

@@ -0,0 +1,3 @@
+
+Respond with ONLY a single-line JSON object, no markdown fences, no extra text:
+{"is_metadata_issue": true|false, "confidence": "high"|"medium"|"low", "reason": "<one short sentence>"}

.github/workflows/triage_metadata_issues.yml

@@ -0,0 +1,220 @@
+name: Triage metadata issues
+
+# Uses the GitHub Copilot CLI to spot new issues that are actually reports
+# about upstream Google libphonenumber *metadata* (country dialing plans,
+# valid ranges, prefixes, formatting) rather than bugs in this C# port. Those
+# can't be fixed here - see CONTRIBUTING.md and the bug_report.md checklist -
+# so this closes them with a canned pointer to the upstream repo, and opens a
+# PR adding the issue to .github/triage/metadata_examples.md so future
+# classifications stay grounded in real history.
+#
+# Requires the repository (or the owner's personal account, since this repo
+# has no org) to have GitHub Copilot CLI usage enabled; see
+# https://docs.github.com/en/copilot/how-tos/copilot-cli/use-copilot-cli-in-actions
+# No extra secret is needed - the built-in GITHUB_TOKEN is used together with
+# the copilot-requests permission below.
+on:
+  issues:
+    types: [opened]
+
+permissions:
+  contents: write
+  issues: write
+  pull-requests: write
+  copilot-requests: write
+
+concurrency:
+  group: triage-metadata-issue-${{ github.event.issue.number }}
+  cancel-in-progress: true
+
+jobs:
+  triage:
+    # x64, not the arm runner used elsewhere in this repo: the Copilot CLI's
+    # npm package does not reliably ship arm64 Linux binaries.
+    runs-on: ubuntu-latest
+    # Don't second-guess the maintainers' own issues (e.g. tracking/planning
+    # issues like #375), only third-party bug reports.
+    if: github.event.issue.author_association != 'OWNER' && github.event.issue.author_association != 'MEMBER' && github.event.issue.author_association != 'COLLABORATOR'
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Read issue
+        id: issue
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
+        with:
+          script: |
+            const truncate = (s, max) => (s && s.length > max ? `${s.slice(0, max)}\n[...truncated]` : (s || ''));
+            core.setOutput('title', truncate(context.payload.issue.title, 500));
+            core.setOutput('body', truncate(context.payload.issue.body, 6000));
+
+      - name: Build classifier system prompt
+        run: |
+          cat .github/triage/system_prompt.md \
+              .github/triage/metadata_examples.md \
+              .github/triage/system_prompt_footer.md \
+              > ${{ runner.temp }}/triage_system_prompt.md
+
+      - name: Install Copilot CLI
+        uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
+        with:
+          node-version: 22
+      - run: npm install -g @github/copilot@1.0.80
+
+      - name: Classify issue
+        id: classify
+        continue-on-error: true
+        uses: actions/ai-inference@2c43c91ae16266ca159d311430343c67a5ffa222 # v3
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          system-prompt-file: ${{ runner.temp }}/triage_system_prompt.md
+          prompt: |
+            Issue title:
+            ${{ steps.issue.outputs.title }}
+
+            Issue body:
+            ${{ steps.issue.outputs.body }}
+
+      - name: Act on classification
+        id: act
+        if: steps.classify.outcome == 'success'
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
+        with:
+          script: |
+            const raw = (${{ toJSON(steps.classify.outputs.response) }} || '').trim();
+            // The system prompt asks for a single-line JSON object with no surrounding text,
+            // but models don't always comply - take the last line that looks like a complete
+            // JSON object rather than greedily spanning from the first '{' to the last '}',
+            // which would swallow any brace that shows up in stray preamble text.
+            const lines = raw.split('\n').map(l => l.trim()).filter(Boolean);
+            const jsonLine = lines.slice().reverse().find(l => l.startsWith('{') && l.endsWith('}')) || raw;
+
+            let verdict;
+            try {
+              verdict = JSON.parse(jsonLine);
+            } catch (err) {
+              core.warning(`Could not parse classifier response as JSON: ${err.message}`);
+              core.summary.addHeading('Metadata issue triage', 3).addRaw(`Skipped: unparseable response:\n\n${raw}`).write();
+              return;
+            }
+
+            core.summary
+              .addHeading('Metadata issue triage', 3)
+              .addRaw(`is_metadata_issue=${verdict.is_metadata_issue}, confidence=${verdict.confidence}\n\n${verdict.reason || ''}`)
+              .write();
+
+            const isMetadataIssue = verdict.is_metadata_issue === true && verdict.confidence !== 'low';
+            core.setOutput('is_metadata_issue', String(isMetadataIssue));
+            core.setOutput('reason', verdict.reason || '');
+            if (!isMetadataIssue) {
+              return;
+            }
+
+            const owner = context.repo.owner;
+            const repo = context.repo.repo;
+            const issue_number = context.payload.issue.number;
+
+            const label = 'metadata';
+            try {
+              await github.rest.issues.createLabel({
+                owner, repo, name: label, color: 'c5def5',
+                description: 'Report about upstream Google libphonenumber data, not this port',
+              });
+            } catch (err) {
+              // 422 just means the label already exists - nothing to do. Anything else is
+              // unexpected, but addLabels below still works whether or not creation succeeded,
+              // so don't let it abort labeling/commenting/closing.
+              if (err.status !== 422) {
+                core.warning(`Could not create label '${label}': ${err.message}`);
+              }
+            }
+            await github.rest.issues.addLabels({ owner, repo, issue_number, labels: [label] });
+
+            await github.rest.issues.createComment({
+              owner, repo, issue_number,
+              body: [
+                'Thanks for filing this! This looks like a phone number **metadata** issue',
+                '(country dialing plans, valid ranges, prefixes, or formatting for a specific',
+                'region) rather than a bug in the `libphonenumber-csharp` code itself.',
+                '',
+                "This library republishes Google's [libphonenumber](https://github.com/google/libphonenumber)",
+                'metadata automatically about every two weeks - it is not hand-edited here, so a',
+                'fix in this repo would just be overwritten by the next sync.',
+                '',
+                'Please check whether the same behavior reproduces on the',
+                '[Google demo](https://libphonenumber.appspot.com/), and if so, file it with the',
+                'upstream project instead: https://github.com/google/libphonenumber/issues',
+                '(see also the [bug report checklist](https://github.com/twcclegg/libphonenumber-csharp/blob/main/.github/ISSUE_TEMPLATE/bug_report.md)).',
+                '',
+                '---',
+                '_This issue was flagged and closed automatically. If this doesn\'t look right -',
+                "for example the behavior differs from Google's own demo, or this isn't actually",
+                'a metadata question - please reopen it and a maintainer will take another look._',
+              ].join('\n'),
+            });
+
+            await github.rest.issues.update({
+              owner, repo, issue_number,
+              state: 'closed',
+              state_reason: 'not_planned',
+            });
+
+      - name: Propose adding this issue to the examples list
+        if: steps.act.outputs.is_metadata_issue == 'true'
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
+        with:
+          script: |
+            const owner = context.repo.owner;
+            const repo = context.repo.repo;
+            const issue_number = context.payload.issue.number;
+            const defaultBranch = context.payload.repository.default_branch;
+            const path = '.github/triage/metadata_examples.md';
+            const branch = `triage/metadata-example-issue-${issue_number}`;
+
+            const clean = (s, max) => (s || '').replace(/[\r\n"]+/g, ' ').replace(/\s+/g, ' ').trim().slice(0, max);
+            const title = clean(context.payload.issue.title, 150);
+            const reason = clean(${{ toJSON(steps.act.outputs.reason) }}, 200) || 'Flagged by automated triage as a metadata issue.';
+            const newLine = `- #${issue_number} "${title}" — ${reason}`;
+
+            try {
+              const existingPrs = await github.rest.pulls.list({ owner, repo, head: `${owner}:${branch}`, state: 'all' });
+              if (existingPrs.data.length > 0) {
+                core.info(`A PR for ${branch} already exists, skipping.`);
+                return;
+              }
+
+              const { data: file } = await github.rest.repos.getContent({ owner, repo, path, ref: defaultBranch });
+              const content = Buffer.from(file.content, 'base64').toString('utf8');
+              if (content.includes(`#${issue_number} "`)) {
+                core.info(`Issue #${issue_number} is already listed, skipping.`);
+                return;
+              }
+              const updatedContent = `${content.trimEnd()}\n${newLine}\n`;
+
+              const { data: ref } = await github.rest.git.getRef({ owner, repo, ref: `heads/${defaultBranch}` });
+              await github.rest.git.createRef({ owner, repo, ref: `refs/heads/${branch}`, sha: ref.object.sha });
+
+              await github.rest.repos.createOrUpdateFileContents({
+                owner, repo, path, branch,
+                message: `Add issue #${issue_number} to metadata-issue triage examples`,
+                content: Buffer.from(updatedContent).toString('base64'),
+                sha: file.sha,
+              });
+
+              await github.rest.pulls.create({
+                owner, repo, base: defaultBranch, head: branch,
+                title: `Add issue #${issue_number} to metadata-issue triage examples`,
+                body: [
+                  `Closed #${issue_number} as a metadata issue during automated triage.`,
+                  'Adding it to the examples list so future classifications stay grounded in real history.',
+                  '',
+                  '_Opened automatically by the triage_metadata_issues workflow - please review the summarized reason before merging._',
+                ].join('\n'),
+              });
+            } catch (err) {
+              core.warning(`Could not open examples-list PR for issue #${issue_number}: ${err.message}`);
+              core.summary
+                .addHeading('Metadata issue triage', 3)
+                .addRaw(`Closed #${issue_number} as a metadata issue, but could not open the examples-list PR: ${err.message}`)
+                .write();
+            }

README.md

@@ -226,6 +226,10 @@ UPSTREAM_TAG=v9.0.33 DEPLOYED_VERSION=9.0.32 \
 
 Nothing about the target repository is hard-coded. The script commits and pushes through whatever checkout it runs in, and takes the repository to release from `GITHUB_REPOSITORY` — set automatically by GitHub Actions, and otherwise derived from the `origin` remote. So a fork releases to itself, and the dry-run summary names the repository it would publish to. `UPSTREAM_REPOSITORY` (default `google/libphonenumber`) and `NUGET_PACKAGE_ID` (default `libphonenumber-csharp`) are overridable the same way.
 
+### Automated triage of metadata issues
+
+A large share of the issues filed here turn out to be reports about Google's phone number metadata itself (an unrecognized prefix, an outdated numbering plan) rather than a bug in this port's code — see the checklist in [`bug_report.md`](.github/ISSUE_TEMPLATE/bug_report.md). The [`triage_metadata_issues`](.github/workflows/triage_metadata_issues.yml) workflow uses the GitHub Copilot CLI, grounded in [`.github/triage/metadata_examples.md`](.github/triage/metadata_examples.md), to spot these on issue creation, closes them with a comment pointing to [google/libphonenumber](https://github.com/google/libphonenumber), and labels them `metadata`. Every closure appends to that examples file, so the classifier keeps learning from real outcomes instead of drifting from a fixed prompt.
+
 ## Contributing
 See [CONTRIBUTING.md](CONTRIBUTING.md)