fix(m365): treat unknown Teams license as unlicensed; adopt semver - #1875
Merged
mitasovr merged 3 commits intoJul 23, 2026
Merged
Conversation
Follow-up to constructorfabric#736 (shipped in constructorfabric#1869), addressing the CodeRabbit review. 1. Teams feeders (meeting, chat): switch `coalesce(isLicensed, true)` to `coalesce(isLicensed, false) = true`, so rows with a NULL/unknown license flag are dropped alongside explicit `isLicensed=false` — matching the constructorfabric#736 proposal. The e2e teams fixture template now defaults `isLicensed: true` so existing fixtures stay counted; the regression test adds an `isLicensed=null` case asserting it is excluded. The email/onedrive/sharepoint feeders keep their conservative `assignedProducts` filter (drop only explicitly-empty lists, keep NULL): those reports expose no `isLicensed`, and the JSON-string serialization is inferred rather than verified, so a NULL there means "unknown/unsynced" and must not nuke the whole feeder. 2. descriptor.yaml: migrate version "2026.05.04" -> "1.0.0" (strict semver, ADR-0015). Because the current deployed value is non-semver, reconcile classifies this as a `migration` (republish + catalog re-discover, NO full-refresh) — it only establishes the semver baseline. A follow-up MAJOR bump (1.0.0 -> 2.0.0) will dispatch the one-shot `dbt --full-refresh` over `dbt_select` (tag:m365+) that purges collab rows materialized before the constructorfabric#736 filter landed, since incremental models do not retro-delete already-written rows. Refs constructorfabric#736. Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
📝 WalkthroughWalkthroughM365 collaboration chat and meeting models now treat NULL ChangesM365 license filtering
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Checkov (3.3.8)src/ingestion/connectors/collaboration/m365/descriptor.yamlTraceback (most recent call last): src/ingestion/tests/e2e/metrics/collab_meeting_unlicensed_excluded.test.yamlTraceback (most recent call last): src/ingestion/tests/e2e/metrics/templates/m365_teams.yamlTraceback (most recent call last): Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/ingestion/connectors/collaboration/m365/dbt/m365__collab_chat_activity.sql (1)
75-77: 🗄️ Data Integrity & Integration | 🔵 TrivialEnsure the staged version migration actually completes the data cleanup.
The new predicates only affect rows processed after deployment; existing unlicensed rows remain in these incremental models. The documented
1.0.0 → 2.0.0full refresh must therefore be guaranteed.
src/ingestion/connectors/collaboration/m365/dbt/m365__collab_chat_activity.sql#L75-L77: retain the explicit-true filter and ensure historical rows are refreshed or deleted.src/ingestion/connectors/collaboration/m365/dbt/m365__collab_meeting_activity.sql#L50-L52: retain the explicit-true filter and ensure historical rows are refreshed or deleted.src/ingestion/connectors/collaboration/m365/descriptor.yaml#L2-L8: enforce the subsequent major-version full-refresh rollout.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/connectors/collaboration/m365/dbt/m365__collab_chat_activity.sql` around lines 75 - 77, Retain the explicit-true licensing filters in m365__collab_chat_activity.sql (75-77) and m365__collab_meeting_activity.sql (50-52), and ensure historical rows are refreshed or deleted. Update src/ingestion/connectors/collaboration/m365/descriptor.yaml (2-8) to enforce the documented 1.0.0-to-2.0.0 full-refresh rollout so the cleanup completes for existing data.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/ingestion/connectors/collaboration/m365/descriptor.yaml`:
- Around line 2-8: Update the operator-facing reconcile baseline in ingestion
README documentation from 2026.05.04 to 1.0.0, keeping it consistent with the
descriptor’s version and preserving the surrounding migration guidance.
---
Nitpick comments:
In
`@src/ingestion/connectors/collaboration/m365/dbt/m365__collab_chat_activity.sql`:
- Around line 75-77: Retain the explicit-true licensing filters in
m365__collab_chat_activity.sql (75-77) and m365__collab_meeting_activity.sql
(50-52), and ensure historical rows are refreshed or deleted. Update
src/ingestion/connectors/collaboration/m365/descriptor.yaml (2-8) to enforce the
documented 1.0.0-to-2.0.0 full-refresh rollout so the cleanup completes for
existing data.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 44e35fa4-c23b-4005-8c93-4bc88f809955
📒 Files selected for processing (5)
src/ingestion/connectors/collaboration/m365/dbt/m365__collab_chat_activity.sqlsrc/ingestion/connectors/collaboration/m365/dbt/m365__collab_meeting_activity.sqlsrc/ingestion/connectors/collaboration/m365/descriptor.yamlsrc/ingestion/tests/e2e/metrics/collab_meeting_unlicensed_excluded.test.yamlsrc/ingestion/tests/e2e/metrics/templates/m365_teams.yaml
Comment on lines
+2
to
+8
| # 1.0.0 (semver migration): adopt strict MAJOR.MINOR.PATCH (ADR-0015). The prior | ||
| # value "2026.05.04" is not semver, so reconcile classifies current→1.0.0 as a | ||
| # `migration` (republish + catalog re-discover, NO full-refresh). This only | ||
| # establishes the semver baseline. A follow-up MAJOR bump (1.0.0 → 2.0.0) will | ||
| # dispatch the one-shot `dbt --full-refresh` over `dbt_select` (tag:m365+) that | ||
| # purges collab rows materialized before the #736 unlicensed-user filter landed. | ||
| version: "1.0.0" |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Update the documented reconcile baseline.
src/ingestion/README.md still describes 2026.05.04 as the current baseline, which will contradict this descriptor’s 1.0.0 version after merge. Update the operator-facing documentation in the same change.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/ingestion/connectors/collaboration/m365/descriptor.yaml` around lines 2 -
8, Update the operator-facing reconcile baseline in ingestion README
documentation from 2026.05.04 to 1.0.0, keeping it consistent with the
descriptor’s version and preserving the surrounding migration guidance.
ktursunov
approved these changes
Jul 23, 2026
mitasovr
enabled auto-merge (squash)
July 23, 2026 12:54
ktursunov
reviewed
Jul 23, 2026
| # coalesce(isLicensed, false) != true, so an unset flag would otherwise | ||
| # exclude every fixture row. Tests exercising the unlicensed path set this | ||
| # explicitly to false (or null). See #736. | ||
| isLicensed: true |
Contributor
There was a problem hiding this comment.
let's check all cases - true/false/null
mitasovr
added a commit
that referenced
this pull request
Jul 27, 2026
) Step 2 of the #736 cleanup. 1.0.0 (#1875) established the semver baseline as a `migration` with no data action. This MAJOR bump makes reconcile dispatch a one-shot `dbt --full-refresh` over `dbt_select` (tag:m365+) per ADR-0015, which re-materializes the m365 staging + downstream silver from append-only bronze and thereby purges the unlicensed collab rows that incremental models left behind before the #736 filter landed. PREREQUISITE: deploy only after the 1.0.0 release has reconciled to the live environment. If the Airbyte-side current version is still the legacy non-semver "2026.05.04", classify_bump treats this as `migration` and NO full-refresh is dispatched. Refs #736. Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech> Co-authored-by: Roman Mitasov <Roman.Mitasov@constructor.tech> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #736 (shipped in #1869), addressing the CodeRabbit review on that PR.
1. Teams license filter — exclude unknown (NULL) too
CodeRabbit flagged that the merged Teams filter kept
isLicensed IS NULLrows instead of excluding them as the issue proposed. Switched both Teams feeders:Now an unlicensed or unknown-license row is dropped; only an explicit
isLicensed=truerow is kept (m365__collab_meeting_activity,m365__collab_chat_activity).isLicensed: trueso existing fixtures stay counted.collab_meeting_unlicensed_excluded.test.yamlgains anisLicensed=nullcase asserting exclusion (alicetrue→3, frankfalse→null, gracenull→null).The email/onedrive/sharepoint feeders keep their conservative
assignedProductsfilter (drop only explicitly-empty lists, keep NULL): those Graph reports expose noisLicensed, and the JSON-string serialization is inferred rather than verified, so a NULL there means "unknown/unsynced" and must not nuke the whole feeder.2. descriptor.yaml → strict semver (ADR-0015), step 1 of 2
CodeRabbit also noted the incremental models don't retro-delete rows materialized before the #736 filter. The blessed cleanup per ADR-0015 is a MAJOR-bump full-refresh over
dbt_select(tag:m365+) — but that only fires when the current deployed version is already semver.m365 is currently
version: "2026.05.04"(non-semver), soclassify_bump.pywould treat any bump as amigration(republish + re-discover, no full-refresh). This PR is step 1: migrate to1.0.0to establish the semver baseline. A step 2 PR (1.0.0 → 2.0.0) will then dispatch the one-shotdbt --full-refreshthat purges the pre-#736 unlicensed rows.Testing
Dockerized e2e rig, affected Teams metrics:
(new 3-way exclusion test +
collab_meetings/collab_messages/collab_meeting_free_days)Refs #736. Follows #1869.
🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Tests
Other
1.0.0.