Cybersecurity research · Human behavior · Enterprise risk
An organization can become better at phishing simulations without demonstrating that it has become harder to compromise.
Every year, employees complete awareness courses, receive simulated phishing messages, and get assigned remedial lessons after clicking. Security teams collect the results and present improvements to management. Sometimes those improvements are real. Sometimes the exercises mainly show that employees have adapted to the exercises.
What matters is whether the organization is less likely to suffer credential theft, fraudulent payments, unauthorized access, malware execution, or operational disruption. That question is considerably harder to answer—and too often missing from the training report.
01 / The evidence
The studies are challenging what companies think training achieves
The premise behind most phishing-awareness programs is reasonable: teach people to recognize suspicious messages and fewer attacks will succeed. The problem is that learning a rule, recognizing a simulated message, and resisting a real attack under pressure are different outcomes.
A 2025 paper presented at the IEEE Symposium on Security and Privacy examined annual training and simulation-triggered lessons at a large healthcare organization. In an eight-month randomized experiment involving more than 19,500 employees and ten simulated phishing campaigns, Ho and colleagues found no statistically significant relationship between recently completing annual awareness training and failing a phishing simulation. Embedded training produced very small absolute differences in later failure rates. Most participants spent little time engaging with the training material, and for some content types, repeated completion was associated with worse subsequent simulation outcomes.[1]
That is a serious result, but it is not proof that education never works. The researchers evaluated commonly deployed interventions in one healthcare setting, primarily through simulated phishing outcomes. They did not randomly remove all protective controls or measure the enterprise’s long-term breach reduction.
Was the lesson doing the work—or the reminder?
At ACM CCS 2024, Daniele Lain and colleagues studied the components of embedded phishing training: educational content, the reminder created by a simulation, and incentives or potential consequences. They found that the observed benefit came chiefly from the periodic reminder rather than lesson content, which employees often did not consume. Delaying the training was as effective as immediate delivery, and rewards did not improve secure behavior.[2]
That changes the interpretation of a successful campaign. If performance improves because staff have been reminded to watch for suspicious email, it does not follow that a lengthy remedial module was responsible.
Other field studies show why simple conclusions are risky
A separate 2022 USENIX Security study followed more than 14,000 employees over 15 months. It found that conventional embedded training did not reliably make users more resilient and could produce counterproductive effects. The same work found evidence that employees collectively reporting suspicious messages could help detect campaigns rapidly with manageable operational burden.[3]
In a 2025 field experiment covering approximately 33,000 employees across hundreds of Dutch small and medium-sized enterprises, Gonzalez-Jimenez and coauthors found that earlier phishing drills had a non-systematic, largely short-term effect on later click behavior. Some employees improved more than others. That variability matters: average results can conceal differences in context, susceptibility, and response.[4]
| Study | What was examined | What the result supports |
|---|---|---|
| Ho et al., 2025 | Annual and embedded training; 19,500+ healthcare employees; ten campaigns over eight months | Common training formats produced little measured improvement in this organization’s simulation outcomes. |
| Lain et al., 2024 | Embedded-training content, reminders, timing, incentives and employee perceptions | Reminder effects mattered more than training content; rewards did not improve secure behavior. |
| Lain et al., 2022 | 15-month enterprise field experiment, including employee reporting | Embedded training could disappoint; collective reporting could contribute meaningfully to detection. |
| Gonzalez-Jimenez et al., 2025 | Phishing drills across hundreds of SMEs and roughly 33,000 employees | Behavioral improvements were not systematic or durable across the sample. |
These studies differ in setting, intervention, and measured outcome. Their findings should not be pooled into a universal estimate of how much phishing training reduces real-world breaches.
02 / Measurement
A lower click rate is not the same as a lower breach rate
Most organizations can tell the board how many employees clicked a simulated message. Far fewer can explain whether a malicious link that reaches an employee can still result in stolen credentials, an approved fraudulent payment, or an attacker gaining access to a sensitive system.
The distinction is not semantic. A click, a credential submission, an authenticated session, an unauthorized action and a material incident represent different security outcomes. The controls between them matter.
A person opened a link or attachment. This alone does not prove compromise.
Credentials, data or code were exposed to the attacker.
Identity, session, endpoint, or application control was obtained.
Fraud, data loss, disruption, or another material outcome occurred.
Even the simulation click rate requires context. A message that closely matches someone’s actual work is generally harder to detect than an unfamiliar, implausible message. NIST researchers developed the Phish Scale specifically to evaluate human phishing detection difficulty and explain why click rates vary across simulations.[5]
A low click rate may reflect stronger recognition. It may also mean the simulation was easy, repetitive, irrelevant to the recipients, or unlike current attacker methods. Likewise, an increased click rate after changing to more realistic test scenarios is not necessarily evidence that staff became less security-conscious.
If the dashboard rewards an easier simulation, the organization can report progress without demonstrating better protection.
None of this makes simulation data useless. It means the metric should be used for the limited question it answers: how people interacted with a particular simulated message under particular conditions. It should not be promoted into a complete measure of organizational phishing resilience.
03 / Human factors
Why repetitive simulations and remedial lessons often disappoint
It is tempting to describe phishing failures as ignorance. An employee clicked because they did not know better; therefore the employee needs another lesson. But the field research does not support treating knowledge as the only—or always the dominant—constraint.
People process email while performing other work. An authentic-looking invoice may arrive during month-end close. A message about an urgent patient issue may appear in a healthcare workflow. A supplier payment request may resemble legitimate correspondence already in progress. In those situations, recognition is shaped by task demands and surrounding context, not simply whether the employee can recite the indicators of phishing.
The 2024 Lain study suggests that employees often spent little time consuming embedded lessons and that periodic reminders mattered more than the material itself.[2] The 2025 IEEE study similarly found weak practical effects from widely used training formats.[1]
“Click fatigue” is a useful editorial label, not a validated clinical or psychological diagnosis. The research warrants concern about repetitive exercises, attention, time pressure and the limited marginal value of more training. It does not establish a single condition that explains everyone who clicks.
The counterproductive incentive
When simulation scores become employee performance signals, security teams can inadvertently reward concealment, risk avoidance or gaming the exercise. This is a governance risk to evaluate—not a universal finding from the cited experiments. Punitive responses can also make employees hesitant to report a real mistake quickly, when timely disclosure may be essential to containment.
Good training should make the next protective action clear: how to report an email, how to verify a payment request, when to contact the help desk, and what to do after a suspicious click. It should be proportionate, relevant to actual work, and evaluated against defined outcomes. More modules, by themselves, do not meet that standard.
04 / Control design
Stop asking awareness training to carry controls it cannot provide
A phishing-awareness program should help employees identify, verify and report suspicious activity. It should not be the sole defense preventing a convincing message from becoming a security incident.
Use phishing-resistant authentication where compromise matters most
NIST SP 800-63B-4 defines phishing-resistant authentication in terms of protocols that prevent disclosure of reusable authentication secrets or valid authenticator outputs to an impostor verifier without relying on the user’s vigilance. WebAuthn/FIDO2 is one example. Manually entered one-time passcodes are not considered phishing-resistant under that definition.[6]
That makes it particularly important to evaluate phishing-resistant methods for privileged identities, sensitive applications and high-impact transactions. Implementation must also account for enrollment, device management, recovery, legacy authentication paths and other routes around the primary login mechanism.
Phishing-resistant MFA does not stop every social-engineering outcome. It does not, by itself, prevent an authorized user from making a fraudulent payment, disclosing sensitive information, authorizing a malicious application or being deceived through help-desk processes.
Design transaction controls that do not depend on perfect suspicion
Payment-detail changes, supplier-bank updates, sensitive data releases and privileged administrative actions should have verification paths that are independent of the message asking for them. Examples include callback to an independently maintained contact number, dual approval for defined transactions, and clear exception handling when verification fails.
These are InfoSecured’s risk-control recommendations based on the nature of the fraud and access pathways. The phishing-training experiments did not test a specific payment-verification workflow or establish its quantitative effectiveness.
Reduce exposure, and make reporting useful
Mail and collaboration security, attachment isolation, domain authentication, browser protection, endpoint restrictions, and least-privilege access can reduce the opportunities for a message to cause harm. They require configuration, maintenance and evaluation against the organization’s threats; merely purchasing tools does not establish effectiveness.
Reporting also deserves investment. The 2022 Lain study found that employees acting collectively as reporters could aid rapid detection of phishing campaigns.[3] That supports making suspicious-message reporting easy and connecting reports to triage, response, feedback and incident records.
A reporting button that sends messages into an unreviewed queue is not an effective reporting process. Nor is a training program that emphasizes recognizing suspicious messages but fails to tell employees what to do after they have acted on one.
| Control objective | Evidence worth reviewing | Remaining limitation |
|---|---|---|
| Protect sign-in | Phishing-resistant MFA coverage, fallback routes, recovery procedures and adversarial tests | Does not prevent authorized fraudulent actions or all session-abuse paths |
| Protect high-impact actions | Verified payment-change cases, dual approval records, exceptions and fraud investigations | Process bypasses and collusion remain possible |
| Limit malicious delivery | Mail-control configurations, detection tests, incident samples and quarantine performance | Some malicious messages will still reach users |
| Detect and contain | Report-to-triage time, containment records, escalation tests and incident reconstruction | Reporting and response vary by scenario and operational capacity |
| Build relevant awareness | Scenario difficulty, targeted objectives, reporting behavior and participant feedback | Simulation gains do not equal measured breach reduction |
Suggested assurance evidence, not a claim that the four training studies validated this specific control mix.
05 / Leadership decisions
What should the CISO ask before approving another awareness budget?
Security leadership does not need to eliminate training to take these findings seriously. It needs to stop equating training activity with effectiveness.
First, ask what risk the program is intended to reduce. The objective might be earlier reporting, fewer exposed credentials, fewer successful fraudulent transactions, or reduced time to containment. Those objectives are not interchangeable. Each needs different evidence.
Second, ask whether the metric can be gamed by the test design. Compare simulation difficulty, target group, job context and attacker scenario before treating click-rate movements as improvement or deterioration. NIST’s Phish Scale provides a published method for adding this context.[5]
Third, ask which consequences are already blocked by technical or procedural controls. A simulated click may be embarrassing, but the important exposure may be an unprotected login, a payment approval made without verification, or excessive privileges granted to a compromised account. Control assessment should test those paths directly.
Fourth, ask whether the organization learns from actual incidents. Reports, near misses, fraud attempts and confirmed intrusions can reveal where simulations are unrepresentative or where safeguards did not work as intended. The objective is not to collect the largest possible dataset; it is to retain reliable evidence that supports corrective decisions.
Finally, ask what the next dollar would buy. There may be cases where better role-specific education or clearer reporting guidance is the right investment. There may be others where authentication, transaction verification, identity recovery, privileged access, or response processes offer a more credible risk-reduction opportunity. That choice needs organization-specific threat, cost, and control evidence—not an arbitrary publication-wide 30/60/90-day timetable.
The conclusion
Training is not the security outcome
The uncomfortable finding in this research is not that employees are incapable of learning. It is that organizations can repeat familiar training activities for years while having limited evidence of how much those activities prevent consequential attacks.
Phishing risk management should give employees usable ways to recognize and report attacks while relying on authentication, workflow safeguards, preventive technology and incident response to contain human error. The CISO’s responsibility is to distinguish what the training program measures from what the organization actually needs to protect.
Direct answers
Phishing training: common questions
Does phishing-awareness training work?
Some programs can improve recognition, reporting or short-term simulated behavior. But large field studies of common annual and embedded-training approaches found weak or inconsistent effects. Results depend on design, context and the outcome measured; no single study establishes that all training is ineffective.
What is click fatigue?
In this article, “click fatigue” describes the concern that repetitive simulated phishing tests and follow-up lessons can lose relevance or attention. It is an editorial term, not a validated psychological diagnosis established by the cited studies.
Why isn’t the phishing-simulation click rate enough?
A click rate measures responses to a specific simulated message. It varies with message difficulty and recipient context, and does not directly measure successful account takeover, data loss, fraud or the effectiveness of the controls that interrupt those outcomes.
Should companies stop phishing simulations?
Not automatically. Simulation data can help characterize exposure, reporting and exercise difficulty. The question is whether its design is appropriate, its results are interpreted cautiously and its cost is justified relative to other controls and obligations.
What should security leaders prioritize instead?
Evaluate phishing-resistant authentication for material access, independent verification of high-risk transactions, effective mail and endpoint defenses, straightforward reporting, and rehearsed incident response. Priorities should reflect the organization’s actual threats and control weaknesses.
Evidence base
Research and official guidance
Peer-reviewed conference and journal studies are identified separately from official technical guidance. The suggested executive control approach is an InfoSecured editorial synthesis; it has not itself been evaluated as a single intervention in the studies below.
- Ho, G., Mirian, A., Luo, E., Tong, K., Lee, E., Liu, L., et al. (2025). Understanding the Efficacy of Phishing Training in Practice. IEEE Symposium on Security and Privacy, 37–54. DOI: 10.1109/SP61157.2025.00076.
- Lain, D., Jost, T., Matetić, S., Kostiainen, K., & Čapkun, S. (2024). Content, Nudges and Incentives: A Study on the Effectiveness and Perception of Embedded Phishing Training. ACM CCS 2024. DOI: 10.1145/3658644.3690348.
- Lain, D., Kostiainen, K., & Čapkun, S. (2022). Phishing in Organizations: Findings from a Large-Scale and Long-Term Study. USENIX Security Symposium. ETH Zürich research record.
- Gonzalez-Jimenez, D., Capozza, F., Dirkmaat, T., van de Veer, E., van Druten, A., & Baillon, A. (2025). Falling and failing (to learn): Evidence from a nation-wide cybersecurity field experiment with SMEs. Journal of Economic Behavior & Organization, 230, 106868. DOI: 10.1016/j.jebo.2024.106868.
- Steves, M., Greene, K., & Theofanos, M. (2020). Categorizing Human Phishing Detection Difficulty: A Phish Scale. Journal of Cybersecurity, 6(1), tyaa009. DOI: 10.1093/cybsec/tyaa009. See also the NIST Phish Scale User Guide.
- National Institute of Standards and Technology. (2025). Digital Identity Guidelines: Authentication and Authenticator Management. NIST SP 800-63B-4, Section 3.2.5: Phishing Resistance. Official NIST guidance.