Home  /  IT Security   /  Phishing simulations are ineffective

Phishing simulations are ineffective

During my career as a System Engineer, I’ve frequently encountered clients and colleagues eager to implement phishing exercises as a key part of their cybersecurity strategy. While the intent—to educate employees and reduce vulnerability to phishing attacks—is commendable, I’ve observed that these exercises often fall short of their goals and can even have unintended negative consequences. Recent research, including a study from ETH Zürich, supports this view, suggesting that phishing exercises may not only be ineffective but could also undermine trust within the organisation.

 

 

 

In many organizations, employees who fail phishing tests are subjected to punitive actions, such as mandatory training sessions. This approach fosters resentment and may encourage employees to find ways to game the system rather than genuinely engaging with security practices, for example by running training videos in the background, while doing something else. The focus shifts from building a security-conscious culture to avoiding punishment, which is counterproductive.

 

 

The Limitations Against Sophisticated Attacks

Phishing simulations used today frequently employ basic campaign templates and fail to replicate the sophistication of real-world attacks. In high-level targeted attacks, adversaries may spend weeks or months researching their targets to create phishing attempts that are highly personalized. They might use specific project details, mimic communication styles, or reference internal events. Such emails can be nearly impossible for employees to recognize as fraudulent, no matter how much training they’ve received.

Moreover, email isn’t the only factor at play. In targeted attacks, cybercriminals use other social engineering tactics as well. The U.S. National Institute of Standards and Technology (NIST) points out:

“Recognize that email isn’t the only way to get phished. You can also receive attacks through text messages, phone calls, social media messages, or even physical postal mail.”

 

— NIST (nist.gov)

Phishing exercises that focus solely on email fail to prepare employees for threats across these other channels, leaving significant gaps in the organisation’s defences.

 

Research Findings on the Ineffectiveness of Phishing Exercises

I’m not alone with the opinion, that phishing simulation tests are ineffective. The study “Phishing in Organizations: Findings from a Large-Scale and Long-Term Study” conducted by researchers at ETH Zürich provides compelling evidence on the shortcomings of phishing exercises. The researchers found that:

  • Voluntary Embedded Training Is Ineffective: The combination of simulated phishing exercises and voluntary embedded training did not improve employees’ phishing resilience. In fact, it made them more susceptible to phishing.
  • Continuous Exposure Increases Risk: Employees continuously exposed to phishing simulations may become desensitised, increasing the likelihood that they will eventually fall for a real phishing attempt.
  • Simple Warnings Are Effective: Placing straightforward warnings atop suspicious emails significantly aids employees in recognising potential threats. More detailed warnings were not more effective than simple ones.

To directly quote from the paper:

“Interestingly, contradicting prior research results and common industry practice, we found that the combination of simulated phishing exercises and voluntary embedded training (…)  not only failed to improve employee’s phishing resilience, but it actually even made the employees more susceptible to phishing. Our results suggest caution in the design of embedded training (…) and practical implications of this somewhat surprising and non-intuitive finding.”

 

— D. Lain, K. Kostiainen and S. Čapkun, “Phishing in Organizations: Findings from a Large-Scale and Long-Term Study,” 2022 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2022, pp. 842-859, doi: 10.1109/SP46214.2022.9833766.

Other research corroborates these findings:

  • Short-Term Gains, Long-Term Losses: Studies have shown that while training can initially reduce the likelihood of falling for phishing attacks, the effectiveness diminishes over time as employees revert to old habits.
  • Human Error Is Inevitable: Even knowledgeable employees can fall victim to phishing due to cognitive overload or persuasive content. Training alone may not overcome these factors.
  • Negative Impact on Trust: Employees often feel tricked and distrustful after phishing simulations, which can negatively impact morale and trust in the IT department.

Why Relying on Humans Isn’t Enough

Expecting employees to be the last line of defense against sophisticated cyber threats is unrealistic. Cybercriminals are constantly evolving their tactics, making phishing attempts increasingly convincing. A determined attacker can craft messages that are virtually indistinguishable from legitimate communications, especially when leveraging insider information.

With phishing attacks occurring across various platforms—emails, text messages, phone calls, social media, and even postal mail—it’s nearly impossible for employees to remain vigilant at all times. This omnipresence of threats requires a more robust solution than human awareness alone. We should follow the Swiss Cheese Model, where multiple layers of defence are in place, with humans as the last layer.

Schema of Swiss Cheese Model

User:BenAveling, CC BY-SA 4.0, via Wikimedia Commons

A Better Approach: Strengthening Technical Defences

Rather than over-relying on employee vigilance, organisations should invest in advanced technical solutions.

  • Email Filtering and Threat Detection: Implement sophisticated systems that can identify and block phishing attempts before they reach employees’ inboxes.
  • Anomaly Detection: Use algorithms to detect unusual patterns that may indicate an attack.
  • Multi-Factor Authentication (MFA): Require MFA to add an extra layer of security, ensuring that even if credentials are compromised, unauthorised access is prevented.
  • Zero-Trust Security Models: Adopt a zero-trust approach where every access request is authenticated, before accessing resources. This reduces reliance on perimeter defences and mitigates risks from both external and internal threats.

Nevertheless, sometimes a phishing e-mail still finds its way to a users mailbox. In this case it’s important that you encourage open communication, so employees feel comfortable reporting suspicious activities without fear of reprimand.
Be sure to include a warning banner for suspicious e-mails; as mentioned by the study, these show effectiveness. Keep them short and simple. Here are guides on how to configure them:

An open e-mail with a highlighted banner at the start, stating "[CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments unless you have verified the sender and know the content is safe."

Simple warning banner at the start of the message.

Conclusion

Phishing exercises, while well-intentioned, often fail to deliver meaningful improvements in organisational security. They can inadvertently increase susceptibility to attacks, damage employee trust, and divert resources from more effective security measures. As cyber threats continue to evolve in sophistication and span multiple communication channels, organisations must adapt by investing in advanced detection methods and fostering a security-aware culture that doesn’t overburden employees with unrealistic expectations. By rethinking our approach and focusing on systemic solutions, we can build a more resilient defence against the ever-changing landscape of cyber threats.

To end this post humorously, there is also another factor that plays into this, as this reddit post beautifully shows 🙂

indifferent keystrokes
byu/sellyourcomputer incomics


References

 

Written by:

I'm a system engineer with over 10+ years of experience, working with Networks, Linux, Windows, Citrix, Containers and much more. Currently I'm preparing for my exam to be a "Cyber Security Specialist with Federal Diploma of Higher Education" In my free time I do various IT projects and produce short films.

Leave a comment