Support attributes excluded in code-generation annotations in JQ helpers

CHANGELOG.md

@@ -9,6 +9,7 @@ What's changed
 * Add support for annotations on attributes and groups. ([#645](https://github.com/open-telemetry/weaver/pull/645) by @lquerel).
 * 💥 BREAKING CHANGE 💥 - Upgrade to version 0.4.0 of regorus [requires all v0 policies to be modified](https://github.com/microsoft/regorus/pull/373). Policy upgrade instructions [here](https://www.openpolicyagent.org/docs/latest/v0-upgrade/#upgrading-rego) may help. ([#651](https://github.com/open-telemetry/weaver/pull/651) by @jerbly).
 * Stability level `Deprecated` is deprecated. Conventions should be deprecated via `deprecated` field and should keep the original stability. ([#607](https://github.com/open-telemetry/weaver/pull/607) by @lmolkova).
+* Exclude attributes declared with `code_generation.exclude` annotations in `semconv_attributes` and other JQ attribute helpers. ([#662](https://github.com/open-telemetry/weaver/pull/662) by @lmolkova)
 * Sort metrics by name in all JQ helpers. ([#573](https://github.com/open-telemetry/weaver/issues/573) by @lmolkova)
 
 ## [0.13.2] - 2025-02-13

crates/weaver_forge/README.md

@@ -387,6 +387,25 @@ The `semconv_attributes` function extracts the registry attributes and applies t
 The `semconv_group_attributes_by_root_namespace` function groups the attributes by root namespace. It's
 possible to combine these two functions with your own JQ filters if needed.
 
+The `semconv_grouped_attributes` helper applies code-generation annotations provided on attributes by default.
+The following annotations are supported:
+
+* exclude attribute
+
+  ```yaml
+    - id: some.attribute.name
+      stability: development
+      type: string
+      brief: Some attribute
+      examples: ["foo"]
+      annotations:
+        code_generation:
+          exclude: true
+  ```
+
+To ignore all code generation annotations, pass `ignore_code_generation_annotations = true` option to  
+the `semconv_grouped_attributes`.
+
 **Process Metrics**
 
 The following JQ filter extracts the metrics from the resolved registry, sorted by group

crates/weaver_forge/data/registry-db.yaml

@@ -536,3 +536,22 @@ groups:
           This is useful in cases where the database is running in a clustered environment and the instrumentation is able to record the node executing the query.
           The client may obtain this value in databases like MySQL using queries like `select @@hostname`.
         examples: 'mysql-e26b99z.example.com'
+      - id: db.excluded_attribute
+        stability: development
+        type: string
+        brief: >
+          An attribute that is excluded from code generation.
+        examples: ["foo"]
+        annotations:
+          code_generation:
+            exclude: true
+      - id: db.sensitive_attribute
+        stability: development
+        type: string
+        brief: >
+          An attribute that contains sensitive information.
+        examples: ["bar"]
+        annotations:
+          code_generation:
+            exclude: false # part of the test, do not remove
+          privacy_sensitivity: PII

crates/weaver_forge/expected_output/test/attribute_group/db.md

@@ -436,6 +436,21 @@ The index of the database being accessed as used in the [`SELECT` command](https
 - Stability: Stable
 
 
+#### Attribute `db.sensitive_attribute`
+
+An attribute that contains sensitive information.
+
+
+- Requirement Level: Recommended
+
+- Type: string
+- Examples: [
+    "bar",
+]
+
+- Stability: Development
+
+
 #### Attribute `db.sql.table`
 
 The name of the primary table that the operation is acting upon, including the database name (if applicable).

crates/weaver_forge/expected_output/test/attribute_groups.md

@@ -459,6 +459,23 @@ The index of the database being accessed as used in the [`SELECT` command](https
 - Stability: Stable
 
 
+#### Attribute `db.sensitive_attribute`
+
+An attribute that contains sensitive information.
+
+
+
+- Requirement Level: Recommended
+
+- Type: string
+- Examples: [
+    "bar",
+]
+
+- Stability: Development
+
+- Annotations: {"code_generation": {"exclude": false}, "privacy_sensitivity": "PII"}
+
 #### Attribute `db.sql.table`
 
 The name of the primary table that the operation is acting upon, including the database name (if applicable).

crates/weaver_forge/templates/semconv_jq_fn/weaver.yaml

@@ -7,7 +7,7 @@ templates:
     application_mode: single
   - template: semconv_grouped_attributes.json
     filter: >
-      semconv_grouped_attributes({ "stable_only": false })
+      semconv_grouped_attributes({ "stable_only": false, "ignore_code_generation_annotations": true })
     application_mode: single
   - template: semconv_grouped_attributes_stable.json
     filter: >

crates/weaver_forge/templates/test/attribute_groups.md

@@ -43,5 +43,8 @@
   {% if attribute.stability %}
 - Stability: {{ attribute.stability | capitalize }}
   {% endif %}
+  {%- if attribute.annotations %}
+- Annotations: {{ attribute.annotations }}
+  {%- endif %}
   {% endfor %}
   {% endfor %}

defaults/jq/semconv.jq

@@ -27,6 +27,22 @@ def stability_filter($options):
         .
     end;
 
+# Filters out attributes based on code generation annotations.
+# $options is an object that can contain:
+# - ignore_code_generation_annotations: a boolean to ignore code generation annotations.
+def code_generation_exclude_filter($options):
+    if ($options | has("ignore_code_generation_annotations")) then
+        .
+    else
+        # null coalescence is not supported in jaq (but supported in jq)
+        map(select(
+            .annotations == null 
+            or .annotations.code_generation == null 
+            or .annotations.code_generation.exclude == null 
+            or .annotations.code_generation.exclude == false
+        ))
+    end;
+
 #####################
 # Attribute functions
 #####################
@@ -37,11 +53,13 @@ def stability_filter($options):
 # - exclude_root_namespace: a list of root namespaces to exclude.
 # - stable_only: a boolean to exclude all non-stable attributes.
 # - exclude_stability: a list of stability statuses to exclude. Use `stable_only` to exclude all non-stable attributes instead.
+# - ignore_code_generation_annotations: a boolean to ignore code generation annotations.
 def semconv_attributes($options):
     .groups
     | map(select(.type == "attribute_group" and (.id | startswith("registry."))))
     | map(.attributes) | add
     | stability_filter($options)
+    | code_generation_exclude_filter($options)
     | if ($options | has("exclude_deprecated") and $options.exclude_deprecated == true) then
         map(select(has("deprecated") | not))
       else