Skip to content

Communityupdates - #3

Merged
timbMSFT merged 3 commits into
masterfrom
CommunityUpdates
Dec 3, 2018
Merged

Communityupdates#3
timbMSFT merged 3 commits into
masterfrom
CommunityUpdates

Conversation

@timbMSFT

@timbMSFT timbMSFT commented Dec 3, 2018

Copy link
Copy Markdown
Contributor

Update community github with fixes and new office queries.

@timbMSFT
timbMSFT requested a review from juliango2100 December 3, 2018 12:30
…n structure of underlying office activity data

@juliango2100 juliango2100 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@zhzhao8888 zhzhao8888 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind. I see you trying to check into the original folders.

@zhzhao8888 zhzhao8888 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I see you trying to check into the original folders.

@timbMSFT
timbMSFT merged commit d392c5e into master Dec 3, 2018
zhzhao8888 added a commit that referenced this pull request Feb 27, 2019
Shain (shainw) pushed a commit that referenced this pull request May 14, 2020
Sarah Young (sarah-yo) pushed a commit that referenced this pull request Feb 2, 2021
…IntelImprovements

Minor fixes for nullable fields from Guardicore ThreatIntel feed
Shain (shainw) pushed a commit that referenced this pull request Feb 18, 2021
Shain (shainw) pushed a commit that referenced this pull request Mar 22, 2021
v-maudan pushed a commit that referenced this pull request Sep 30, 2021
Armorblox Sentinel Solution Azure Function changes
Ofer Shezaf (oshezaf) added a commit that referenced this pull request Jan 16, 2022
v-ntripathi (NikTripathi) pushed a commit that referenced this pull request Mar 4, 2022
Initial Commit with the first version
v-ntripathi (NikTripathi) pushed a commit that referenced this pull request Apr 27, 2022
…47385-cs-47271

Bumps version. Updates code & solution zip file.
v-ntripathi (NikTripathi) pushed a commit that referenced this pull request Jul 22, 2022
v-ntripathi (NikTripathi) pushed a commit that referenced this pull request Aug 3, 2022
Merging upstream into master
Anthony Ledesma (arledesma) pushed a commit to mindstream-it/Azure-Sentinel that referenced this pull request Sep 9, 2022
XDR-1172: Set up Upstream Sync for Azure-Sentinel using CI with Azure Pipelines (feat)
mayank88mahajan added a commit to mayank88mahajan/Azure-Sentinel that referenced this pull request Jan 13, 2023
mayank88mahajan added a commit to mayank88mahajan/Azure-Sentinel that referenced this pull request Jan 14, 2023
v-dvedak pushed a commit that referenced this pull request Aug 2, 2023
Merge master back to fork repo
v-atulyadav pushed a commit that referenced this pull request Aug 14, 2023
v-atulyadav pushed a commit that referenced this pull request Nov 1, 2023
v-atulyadav pushed a commit that referenced this pull request Mar 12, 2024
…dpointconnector-ccp

Adding tables to .script/tests/KqlvalidationsTests/CustomTables
v-atulyadav pushed a commit that referenced this pull request Sep 10, 2024
Pull changes from Azure Main Repo
v-atulyadav pushed a commit that referenced this pull request Dec 31, 2024
Corrected validations and re-packaged the solution.
v-dvedak pushed a commit that referenced this pull request Sep 18, 2025
…zureCredential

[MXDR-3374] fix default azure credential
Jason Heard (jlheard) added a commit that referenced this pull request Dec 22, 2025
Co-authored-by: jlheard <1328792+jlheard@users.noreply.github.com>
v-atulyadav pushed a commit that referenced this pull request Jan 16, 2026
Co-authored-by: anthonylamark <3209818+anthonylamark@users.noreply.github.com>
v-dvedak pushed a commit that referenced this pull request Jan 22, 2026
Co-authored-by: v-maheshbh <207855009+v-maheshbh@users.noreply.github.com>
v-dvedak pushed a commit that referenced this pull request Jan 22, 2026
v-atulyadav pushed a commit that referenced this pull request Feb 23, 2026
* recompile using v3 tool

* recompile using v3 tool

* self review

* follow up from claude on depends for contentPackages

* revert

* cleanup

* minimize diff
Itai Margalit (ItaiMargalit) pushed a commit to ItaiMargalit/Azure-Sentinel that referenced this pull request May 24, 2026
Fixes two deploy-time bugs in v3.1.0 (PR Azure#13524).

Bug Azure#1 — Paging:
Switched pagingType from "NextPageUrl" to "Offset", matching the IONIX
API's standard DRF LimitOffsetPagination shape. The old config failed
CCF connectivity validation with "Missing NextPageParaName from config"
(RES40002) for every customer on 3.1.0.

Bug Azure#3 — Table declaration:
The original v3.1.1 attempt removed IONIX_Table.json on the theory that
Log Analytics would auto-create the custom table on first DCR write.
That's not how it works — Sentinel's DCR validator requires the custom
output table to already exist, and Connect failed with:

  InvalidOutputTable: Table for output stream
  'Custom-CyberpionActionItems_CL' is not available for destination
  'IONIX-Destination'.

Restored IONIX_Table.json with the 12-column declaration (same shape as
v3.1.0) so the table is created when the solution is installed via
Content Hub (which auto-executes the contentTemplates nested mainTemplate).
This matches the pattern used by every other CCF solution in the catalog
(Auth0, Cloudflare CCF, SAP BTP, CyrenThreatIntelligence — all have the
table declared inside the contentTemplates wrapper, none at top level).

Package regenerated via Tools/Create-Azure-Sentinel-Solution/V3
(createSolutionV3.ps1) — mainTemplate.json + 3.1.1.zip are canonical
V3-tool output.

Verified end-to-end on a fresh Sentinel workspace:
- Solution installs via Content Hub
- CyberpionActionItems_CL table created
- CCF connector connects (green), no RES40002
- First poll ingested 64 deduped action items via Offset paging
- Workbook + analytics rule template both render

Note: customers migrating from the deprecated push connector who already
have a populated CyberpionActionItems_CL table with the legacy 15-18
column set will still hit "Invalid output table schema" on this version
(same behavior as v3.1.0). Migration handling is tracked separately for
v3.1.2 — recommended approach is a new table name for the CCF connector
with a union shim during the transition period.
Elakkuvan Rajamani (elakkuvan-r) added a commit to elakkuvan-r/Azure-Sentinel that referenced this pull request Jun 4, 2026
…zure#3, Azure#4)

Azure#2 — Renamed Whisper - BGP Anomaly Hunt → Whisper - ASN Reputation Score Hunt
to match the query's actual logic (compares ASN reputation scores across
24-hour windows, flags increases >20 points). Filename kept stable.

Azure#3 — Standardized the playbook parameter naming across all 10 playbooks
from `playbook-name` (kebab) to `PlaybookName` (PascalCase) per the
Azure-Sentinel repo convention. Updated both the parameter declaration
and every `parameters('playbook-name')` reference.

Azure#4 — Moved the Whisper Security custom API connector ARM template into
its own folder per repo convention:
  Solutions/Whisper/Playbooks/WhisperSecurityConnector.json
    → Solutions/Whisper/Playbooks/WhisperSecurityConnector/azuredeploy.json

Package regenerated to 3.0.29 via Create-Azure-Sentinel-Solution V3.

Azure#1 (parse_json on comma-separated strings) — replied on the thread
explaining deferral until we have live data to verify the actual
ingestion-pipeline output format; not changing code in this round.

Signed-off-by: Elakkuvan Rajamani <elakkuvan@whisper.security>
noodlemctwoodle (noodlemctwoodle) added a commit to noodlemctwoodle/Azure-Sentinel that referenced this pull request Jul 21, 2026
…#14726

Applies fixes for every Copilot inline comment on
Azure#14726, retaining the Logstash
container semantics that are already validated in production.

Logstash pipeline (deployed to prod, verified end-to-end):

- Solutions/UniFi Syslog (CCF)/Logstash/logstash.conf
  - Azure#2/Azure#3: Section 0's ruby PRI decoder now also maps facility/severity
    integer codes to their RFC 5424 name strings (kern/user/mail/... and
    emergency/alert/critical/error/warning/notice/informational/debug).
    Without this, section 6's rename filter (`syslog_facility -> Facility`,
    `syslog_severity -> SyslogSeverity`) was a silent no-op and every
    row landed with null Facility/SyslogSeverity. Consequence in prod:
    `UnifiSyslogCriticalSeverityEvent` never matched an event since the
    solution's initial commit — its `where SyslogSeverity in~ ("critical")`
    predicate could never be true. Confirmed post-fix: 100% of events
    populate both fields.
  - Azure#2/Azure#3 extended: added a CEF-without-syslog-header fallback right
    before section 6's rename. Raw CEF sent by the UDM cloud-key API
    direct to UDP/TCP 514 bypasses the syslog header parse (no <pri>
    prefix), so section 0's ruby decode skips them. New block:
      * if EventFormat == "cef": derive Facility=daemon (network daemon
        origin) + SyslogSeverity from CEF LogSeverity range mapping
        (0-3=informational, 4-5=notice, 6=warning, 7-8=error, 9-10=critical)
      * otherwise: default to "unknown"/"unknown"
    Confirmed post-fix: 0/157 recent fresh events land with a null.
  - Azure#4: replaced always-on `stdout { codec => rubydebug }` with a
    conditional gated on ENABLE_STDOUT_RUBYDEBUG env var (default
    "false"). Logstash env-var interpolation syntax
    `"${ENABLE_STDOUT_RUBYDEBUG:false}" == "true"` means production
    containers never dump per-event debug to `docker logs` unless the
    operator explicitly opts in via .env. Confirmed post-fix: log
    volume dropped from thousands of lines/min to 39 lines / 3min
    (startup only, zero rubydebug event blocks).
  - Azure#12: typo `slient` -> `silent` in a code comment (line 583).

- Solutions/UniFi Syslog (CCF)/Logstash/.env.example
  - Documented the new ENABLE_STDOUT_RUBYDEBUG env var with a warning
    about production log volume.

- Solutions/UniFi Syslog (CCF)/Logstash/docker-compose.yml
  - Azure#9/Azure#10: rewrote for portability. Old file required an external
    `syslog-net` macvlan network + hardcoded `10.0.0.21` IP that would
    fail immediately for anyone else. New default publishes 514/udp +
    514/tcp from the host via `ports:`, which works out of the box on
    any Docker host. The macvlan pattern is documented as a commented
    optional section at the bottom of the file for advanced users who
    need a dedicated LAN IP (e.g. hosts already running rsyslog on 514,
    or UniFi devices that prefer target-discovery on-LAN). Also passes
    ENABLE_STDOUT_RUBYDEBUG through with a `:-false` default so the
    guard evaluates correctly even if the env var is unset.

DCR / mainTemplate:

- Solutions/UniFi Syslog (CCF)/Data Connectors/UnifiSyslog_ccf/UnifiSyslog_DCR.json
  - Azure#5: added isnotempty() guards on the CEF transformKql's SourcePort
    and DestinationPort projections. Before: `SourcePort = toint(SourcePort)`
    which converts an empty string ('') to `0`, meaning downstream
    queries filtering `where SourcePort > 0` silently drop rows that
    genuinely had no source port emitted by CEF (e.g. ICMP flows). After:
    `SourcePort = iff(isnotempty(SourcePort), toint(SourcePort), int(null))`.
    Matches the guard pattern already used elsewhere in the same
    transformKql for DeviceCustomNumber1 / ThreatSeverity /
    MaliciousIPLatitude / MaliciousIPLongitude. Same fix applied to
    DestinationPort.

Analytic rules + hunting queries (11 files):

- Azure#7: added `| where DeviceVendor =~ "Ubiquiti"` scope filter right after
  the `CommonSecurityLog` table reference on every rule/hunt whose query
  starts with CommonSecurityLog. Without this scope, the rules/hunts
  would fire on any TI match / SSH brute force / Tor exit hit / port
  scan / etc. from any Sentinel data source that lands in
  CommonSecurityLog (Fortinet, Cisco, Palo Alto, etc.) — creating
  cross-solution false positives with generic "UniFi:" alert titles.
  Files updated:
    Analytic Rules/UnifiSyslogSshBruteForce.yaml
    Analytic Rules/UnifiSyslogThreatIntelMatch.yaml
    Analytic Rules/UnifiSyslogTorExitTraffic.yaml
    Hunting Queries/UnifiSyslogApplicationPortScanners.yaml
    Hunting Queries/UnifiSyslogDhcpNakDecline.yaml
    Hunting Queries/UnifiSyslogFirstSeenExternalSources.yaml
    Hunting Queries/UnifiSyslogPortScannerSweep.yaml
    Hunting Queries/UnifiSyslogSshTargetUsers.yaml
    Hunting Queries/UnifiSyslogSudoActivityAudit.yaml
    Hunting Queries/UnifiSyslogTiFeedEffectiveness.yaml
    Hunting Queries/UnifiSyslogTopExternalAttackers.yaml
    Hunting Queries/UnifiSyslogWifiDeauthFlood.yaml
  Post-audit: `grep DeviceVendor` confirms all 11 files now have
  `DeviceVendor =~ "Ubiquiti"` filter; audit script exits clean.

- Solutions/UniFi Syslog (CCF)/Hunting Queries/UnifiSyslogDhcpNakDecline.yaml
  - Azure#8: entityMappings block was declaring a Host entity with HostName
    mapped to `SampleClient`, but `SampleClient = take_any(SourceMACAddress)`
    is a MAC address, not a hostname. Sentinel would try to resolve the
    Host entity against something like "aa:bb:cc:dd:ee:ff" — invalid
    entity resolution. Fix: added `SampleHost = take_any(SourceHostName)`
    and `SampleIP = take_any(SourceIP)` to the summarize output; entity
    mappings now declare an IP entity (Address -> SampleIP) and a Host
    entity (HostName -> SampleHost). Real hostnames + real IPs, valid
    Sentinel entity resolution.

Workbook:

- Solutions/UniFi Syslog (CCF)/Workbooks/UnifiSyslog.json
  - Azure#6: the "Rows landing per 5-min bucket" timechart on the Pipeline
    Health tab had TWO `union` clauses of the same `CommonSecurityLog`
    CEF count query. Result: the CEF series was double-counted and the
    timechart under-represented Unifi_Syslog_CL's share of ingestion.
    Removed the trailing `| union (...)` block.

Not changed (Copilot false positive):

- Azure#1: Copilot flagged doubled `||` in ReleaseNotes.md line 3. Confirmed
  by byte inspection — file has clean single `|` pipes throughout. Will
  reply to the comment marking as invalid.

Package regenerated:

- Solutions/UniFi Syslog (CCF)/Package/mainTemplate.json
- Solutions/UniFi Syslog (CCF)/Package/3.0.0.zip
  - Rebuilt via createSolutionV3.ps1. Version reset 3.0.1 -> 3.0.0
    (script always auto-bumps; post-process restores). Byte-parity
    verified: disk mainTemplate.json and inside-zip mainTemplate.json
    both sha256=237b8b91d947c246..., both 249,625 bytes. Explicitly
    prevents the "outside and inside zip file mainTemplates are
    different" finding that hit PR Azure#14253.

Validation:

- KQL validation: PASS (22 files)
- ARM-TTK: PASS 48/48
- Field Types: PASS
- Classic App Insights: PASS
- Hyperlink Validation: PASS
- (Skipped: .NET Core 3.1 detection-schema + non-ASCII + TruffleHog —
  local tooling not installed, upstream CI covers these.)

End-to-end prod verification (Logstash relay on 10.0.10.2):

- Container: healthy (Up 37 seconds after last rebuild)
- Pipeline: in=3 filtered=3 out=3 (clean 1:1 processing)
- Docker log volume: 39 lines / 3min (was thousands/min pre-Azure#4-fix)
- Facility/SyslogSeverity population: 157/157 fresh events (0 nulls)

Breaking changes: none.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants