[o365] Better handling of address formats

[adriansr]

What does this PR do?

• Supports bracketed format in ClientIP, ClientIPAddress, and OriginatingServer fields.
• Fixes grok error with invalid IPs. o365: Grok parsing errors due to invalid IP addresses #2660
• Cleanup: Remove unnecessary event.ingested from pipeline.
• Better logic for populating {client,server}.address.
• Improves? some grok expressions ((%{NOTSPACE:...}|%{HOSTNAME:...}))
• Safer handling of IPv4 mapped IPv6 addresses (::ffff:A.B.C.D).

Closes #2660
Relates #2519

[legoguy1000]

Looks like u made additional changes than I did. I can close mine if u want?

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-02-16T18:10:27.388+0000

• Duration: 23 min 23 sec

Test stats 🧪

Test	Results
Failed	0
Passed	28
Skipped	0
Total	28

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[adriansr]

Thanks for your PR, @legoguy1000, much appreciated.

I took the opportunity to make more changes that I felt necessary in this case, so yes, better keep this one instead.

• Support "host (ip)" pair in brackets: [HOSTNAME (IP)].
• Populate .address fields directly. The temporary ._address is unnecessary.
• Simplify a few grok expressions:
  • Better not to use grok's IP type at all and rely on the later convert processor (This only applies to the "host (ip)" formats, as those are the only ones observed to include invalid IPs).
  • Replace usages of (%{NOTSPACE:...}|%{HOSTNAME:...}) with %{NOTSPACE:...}. NOTSPACE is a superset of HOSTNAME.
• Review potentially wrong "::ffff:" gsub expression.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)