[Rule Tuning] Deprecate Individual MSFT Compliance Rules / Create BBR MSFT Purview Compliance Rule

[terrancedejesus]

Fixes #5678

Pull Request

Issue link(s):

• [Rule Tuning] Deprecate Individual MSFT Defender Rules / Create BBR MSFT Defender Rule #5678

Summary - What I changed

Begins deprecation process for all MSFT Purview (M365 Security and Compliance) related rules in favor of a single BBR rule. Please see linked issue for more details.

How To Test

Query for BBR rule can be used in TRADE or telemetry stack.

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[github-actions]

Rule: New - Guidelines

These guidelines serve as a reminder set of considerations when proposing a new rule.

Documentation and Context

• Detailed description of the rule.
• List any new fields required in ECS/data sources.
• Link related issues or PRs.
• Include references.

Rule Metadata Checks

• creation_date matches the date of creation PR initially merged.
• min_stack_version should support the widest stack versions.
• name and description should be descriptive and not include typos.
• query should be inclusive, not overly exclusive, considering performance for diverse environments. Non ecs fields should be added to non-ecs-schema.json if not available in an integration.
• min_stack_comments and min_stack_version should be included if the rule is only compatible starting from a specific stack version.
• index pattern should be neither too specific nor too vague, ensuring it accurately matches the relevant data stream (e.g., use logs-endpoint.process-* for process data).
• integration should align with the index. If the integration is newly introduced, ensure the manifest, schemas, and new_rule.yaml template are updated.
• setup should include the necessary steps to configure the integration.
• note should include any additional information (e.g. Triage and analysis investigation guides, timeline templates).
• tags should be relevant to the threat and align/added to the EXPECTED_RULE_TAGS in the definitions.py file.
• threat, techniques, and subtechniques should map to ATT&CK always if possible.

New BBR Rules

• building_block_type should be included if the rule is a building block and the rule should be located in the rules_building_block folder.
• bypass_bbr_timing should be included if adding custom lookback timing to the rule.

Testing and Validation

• Provide evidence of testing and detecting the expected threat.
• Check for existence of coverage to prevent duplication.

[Mikaayenson]

🤔 we may want to include in the bbr coverage for dlp if it doesn't already

[terrancedejesus]

Moving this back into draft, pending some additional UAL matrix review.

[imays11]

I like the planned phases and coming back to build on top of this BBR, was worried about losing the value of the data here but your plan makes sense.