The Microsoft 365 simulator is not human risk management

Attack Simulation Training, the phishing simulation tool included in Microsoft Defender for Office 365, does what it promises: it sends realistic test emails, measures who clicks and assigns training to whoever falls. What it does not do is validate that the person's behavior changed after the failure, and that is precisely the question human risk management (HRM), the discipline that measures and reduces the risk that comes from people, exists to answer. The difference is not one of quality but of scope: measuring sends and clicks is one thing, proving that someone stopped being vulnerable is another. Confusing the two leaves an organization with full reports and the important question unanswered.

What Microsoft 365 Attack Simulation does well

Let's start by recognizing what the native simulator solves, because it is not trivial. The figure that justifies its existence comes from CISA, the United States cybersecurity agency: more than 90 percent of successful cyberattacks begin with a phishing email. Testing employees with simulated emails is not optional, and for a company that already lives in Microsoft 365, having that test built into its own console is a real advantage.

The scope Microsoft documents is serious. The social engineering techniques are curated from the MITRE ATT&CK framework, the public catalog of attacker tactics, and include credential harvesting, malware attachments, links inside attachments, links to malware, drive-by URLs and OAuth consent grant abuse; the link can be a URL or a QR code. It ships with a built-in payload catalog, lets you create custom payloads and can even capture harmless versions of real phishing emails detected in your own tenant to reuse as simulations. Automations schedule campaigns with several techniques at once, and the reports answer the operational questions: who received the email, who clicked, who entered credentials, who reported it and who completed the assigned training. It even estimates a predicted compromise rate per payload, calculated from aggregated historical data across Microsoft 365.

The requirement is the license: it comes with Microsoft Defender for Office 365 Plan 2, included in Microsoft 365 E5, and E3 only gets a trimmed trial version. Many midsize companies in LATAM living on Business plans do not have the full tool, and it is worth knowing that before taking it for granted. We covered what completing a course does and does not prove in how to measure if security training works.

Where it falls short: no retest, a single channel, no credential monitoring, no per-person matching

The most important limit is the one a report shows least. When someone falls for a simulation, the native simulator assigns training and measures whether they completed it. What Microsoft's documentation does not describe is a scheduled individual retest: testing that same person again weeks later, with a different template from the same category, to verify they learned to recognize the pattern rather than remembering one email. The closest Microsoft documents is the suggested user group of repeat offenders (users compromised by consecutive simulations), which the administrator can target in a new campaign; that is still a campaign built by hand, not an automatic validation that each person learned the lesson. Without that second test, the organization knows who fell and who finished the course, but not who remains vulnerable, which is the only thing that matters for risk.

The second limit is the channel. The entire simulation lives in email: the link, the attachment and even the QR code arrive in the inbox. Fake voice calls and fraudulent text messages, which target the same person through channels email never sees, remain outside the tool's scope.

The third is real exposure. The simulator does not monitor whether employee credentials have appeared in data breaches or on the dark web, so its tests cannot distinguish between the person whose password already circulates among attackers and the one with no known exposure. The simulation operates blind to the risk that already exists.

And the fourth is who picks the template. In the native simulator, the administrator selects the payloads and target groups per campaign; the predicted compromise rate helps compare payloads, but it is an aggregate prediction, not a per-person selection. Microsoft's own documentation suggests creating identical simulations scoped by department to compare which one is more vulnerable, which is to say the segmentation is manual. A human risk program inverts that logic: it crosses each person's role, failure history and exposure to send them the lure most likely to fool them, because that is where the test reveals something. We answered the most common questions about that model in our human risk management FAQ.

Table: native simulator versus a human risk management program

The scope comparison is clearest in a table. They are not rivals: one is a feature of an email suite, the other is a complete program built around people.

What a CISO who already lives in Microsoft 365 should demand

The short answer: do not discard the simulator, but do not confuse it with the program. Cisco's 90-5-5 framework, which estimates that around 90 percent of breaches involve a human factor, sets the bar for what that program must prove: if people are the main risk surface, the metric that matters is whether their behavior improves, not how many test emails went out.

The concrete demands follow from that. First, individual retest: whoever fell gets tested again with a different template from the same category, because that is the only evidence of real change. Second, consequence in the risk score: a failure should weigh on that person's indicator until they demonstrate sustained resistance, not until they finish a video. Third, connection to real exposure: an employee's leaked credentials should influence which simulation they receive and their remediation priority. And fourth, reports that speak of behavior: leadership does not need to know how many clicks there were, it needs to know whether the organization resists better than it did a quarter ago.

A CISO already paying for E5 can cover the baseline with the native tool and demand these four things from any platform evaluated on top of it. One without E5 should evaluate directly against these demands, because they will pay either way and might as well pay for the full scope.

That cycle of demands (monitoring of leaked credentials, per-person simulation and a retest that validates the change) is the use case Fensivo works on, as a complement to the Microsoft 365 environment rather than a replacement for its controls. The platform retests whoever fell and answers the question the native simulator leaves open. You can see how we approach it in Fensivo's use cases.

If leadership asked you today to prove, with data, that the person who fell last month would not fall again, what would you show them?

Sources and references

• CISA, "4 Things You Can Do To Keep Yourself Cyber Safe". cisa.gov
• Cisco, "The 90-5-5 Concept: Your Key to Solving Human Risk in Cybersecurity", May 27, 2025. blogs.cisco.com