Microsoft recently released new guidance on how to configure Exchange Online protection and Office 365 Advanced Threat Protection in a secure way. You can find these recommendations here:
Recommended settings for EOP and Office 365 ATP security
Microsoft also released a new tool to run against your tenant to make sure EOP and Office 365 ATP is configured correct and follows the new best practices. This tools is called ORCA (The Office 365 ATP Recommended Configuration Analyzer) and I’ve tried it out against a couple of tenants with good results.
More info about ORCA on GitHub.
ORCA will check:
- Configuration in EOP which can impact ATP
- SafeLinks configuration
- SafeAttachments configuration
- Antiphish and antispoof policies
I recommend everybody to run ORCA and it’s super easy to get started!
Generate ORCA report
First install the ORCA PowerShell Module:
Install-Module -Name ORCA
Then connect to Exchange Online with PowerShell as you’re used to.
Generate the ORCA report:
Get-ORCAReport
Example PowerShell-output:
That’s it. The CMDlet will ask you which HTML-viewer you would like to open report in.
The ORCA Report
The report is easy to understand and it will list recommended actions in yellow.
Example report:
The report includes information on which policys needs to be changed and an explanation on why.
All in all, this is what ORCA checks (version 1.3.2, January 2020):
- Bulk Complaint Level threshold is between 4 and 6
- Bulk is marked as spam
- Advanced Spam filter options are runed off
- Outbound spam filter policy settings configured
- No IP Allow Lists have been configured
- Domains are not being whitelisted in an unsafe manner
- Spam action set to Move message to Junk Email Folder
- Change High Confidence Spam action to Quarantine message
- Bulk action set to Move message to Junk Email Folder
- Change Phish action to Quarantine message
- High Confidence Phish action set to Quarantine message
- Safety Tips are enabled
- Set up DKIM signing to sign your emails
- DNS Records have been set up to support DKIM
- Domains are not being whitelisted in an unsafe manner
- Zero Hour Autopurge is Enabled
- Zero Hour Autopurge is Enabled
- Zero Hour Autopurge is Enabled
- Supported filter policy action used
- Unified Audit Log is enabled
- External Sender notifications are disabled
- Common attachment type filter is enabled
- Safe Links Policies are tracking when user clicks on safe links
- Safe Attachments is enabled for SharePoint and Teams
- Safe Links is enabled for Office ProPlus, Office for iOS and Android
- Safe Links is enabled intra-organization
- Safe Links Synchronous URL detonation is enabled
- Anti-phishing policy exists and EnableAntiSpoofEnforcement is true
- Safe Attachments is not bypassed
- Safe Links is not bypassed
- Advanced Phish filter Threshold level is adequate
- Mailbox intelligence is enabled in anti-phishing policies
- Domain Impersonation action is set to move to Quarantine
- User impersonation action is set to move to Quarantine
- Your policy is configured to notify users with a tip
Please follow me here, on LinkedIn and on Twitter!
Daniel Chronlund Cloud Security Blog 