We’ve all deployed our new and shiny, well tested, well piloted Conditional Access policy designs just to find out that a couple of legacy systems still connects to Azure AD with legacy protocols, one office in a small-town without a name haven’t domain joined their computers and another office buys their smartphones from a guy named Joe without ever enrolling them in MDM. Authentication fails and we need to roll back. Sad face 😦
Well, that’s history (happy face 🙂 ). The new Conditional Access Report-only mode (announced at Ignite 2019) will help you to better test and predict potential issues that new policy designs might cause. Conditional Access Report-Only mode gives you the possibility to put your CA policies in a state where there are no policy enforcement, but the policy engine still reports what would have happened if policies were active. This is all logged in the Azure AD Sign-ins log. You can analyze every authentication and how the user would have experienced the logon if the policies were active. Very powerful!
Report-only mode is just a click away. Go into your CA policy (or create a new one) and set it to Report-only. It’s as easy as that. See the results in the Azure AD Sign-ins log.
But what’s even more powerful is the new Conditional Access Insights dashboard. This dashboard lives in Azure Monitor and integrates with the Sign-ins log of Azure AD. It will visualize the statistics on how users experience Conditional Access. You can filter on certain Conditional Access policies or certain users and you can dig into what will happen when you enable your new Conditional Access policies.
All you need to do to setup the Conditional Access Insights dashboard is to enable export of the Sign-ins log in Azure AD to Azure Monitor/your LogAnaytics workspace in Azure. You will find the dashboard, among other valuable reports, under Azure Active Directory > Workbooks.
Trust me, Conditional Access Report-only mode will be one of your most important tools in upcoming CA deployments and changes. Use it wisely!
Please follow me here, on LinkedIn and on Twitter!
Daniel Chronlund Cloud Security Blog 
One thought on “Safe Conditional Access Deployment with Report-Only Mode and the Insights Dashboard”