Phishing simulations are ineffective
During my career as a System Engineer, I’ve frequently encountered clients and colleagues eager to implement phishing exercises as a key part of their cybersecurity strategy. While the intent—to educate employees and reduce vulnerability to phishing attacks—is commendable, I’ve observed that these exercises often fall short of their goals and can even have unintended negative consequences. Recent research, including a study from ETH Zürich, supports this view, suggesting that phishing exercises may not only be ineffective but could also undermine trust within the organisation.
my university sent an email about providing $7,500 in assistance to those experiencing financial hardship due to the pandemic….turns out it was a PHISHING exercise…
is this a joke???
— Nina Luong (@nina_luong) April 12, 2022
GoDaddy
June: laid off 800+ employees
November: reported 11% revenue increase & record number of new customers
Now: sent staff notice of $650 bonuses – turns out it was a phishing test; employees actually get $0 & must complete a “social engineering” testhttps://t.co/1zFG3nprBC— Dan Price (@DanPriceSeattle) December 25, 2020
I clicked on a justeat voucher email at work and got myself enrolled on a phishing course ffss
— F (@fa7_a96) February 21, 2024
In many organizations, employees who fail phishing tests are subjected to punitive actions, such as mandatory training sessions. This approach fosters resentment and may encourage employees to find ways to game the system rather than genuinely engaging with security practices, for example by running training videos in the background, while doing something else. The focus shifts from building a security-conscious culture to avoiding punishment, which is counterproductive.
Omg. Same. Thing. Happened to me…
Usually spammers don’t have an ‘@ Utah ‘ email. https://t.co/SxsI21dVbY
— Brandi Wynne, MS PhD FAHA (@brandimwynne) December 7, 2023
The Limitations Against Sophisticated Attacks
Phishing simulations used today frequently employ basic campaign templates and fail to replicate the sophistication of real-world attacks. In high-level targeted attacks, adversaries may spend weeks or months researching their targets to create phishing attempts that are highly personalized. They might use specific project details, mimic communication styles, or reference internal events. Such emails can be nearly impossible for employees to recognize as fraudulent, no matter how much training they’ve received.
Moreover, email isn’t the only factor at play. In targeted attacks, cybercriminals use other social engineering tactics as well. The U.S. National Institute of Standards and Technology (NIST) points out:
“Recognize that email isn’t the only way to get phished. You can also receive attacks through text messages, phone calls, social media messages, or even physical postal mail.”
— NIST (nist.gov)
Phishing exercises that focus solely on email fail to prepare employees for threats across these other channels, leaving significant gaps in the organisation’s defences.
Research Findings on the Ineffectiveness of Phishing Exercises
I’m not alone with the opinion, that phishing simulation tests are ineffective. The study “Phishing in Organizations: Findings from a Large-Scale and Long-Term Study” conducted by researchers at ETH Zürich provides compelling evidence on the shortcomings of phishing exercises. The researchers found that:
- Voluntary Embedded Training Is Ineffective: The combination of simulated phishing exercises and voluntary embedded training did not improve employees’ phishing resilience. In fact, it made them more susceptible to phishing.
- Continuous Exposure Increases Risk: Employees continuously exposed to phishing simulations may become desensitised, increasing the likelihood that they will eventually fall for a real phishing attempt.
- Simple Warnings Are Effective: Placing straightforward warnings atop suspicious emails significantly aids employees in recognising potential threats. More detailed warnings were not more effective than simple ones.
To directly quote from the paper:
“Interestingly, contradicting prior research results and common industry practice, we found that the combination of simulated phishing exercises and voluntary embedded training (…) not only failed to improve employee’s phishing resilience, but it actually even made the employees more susceptible to phishing. Our results suggest caution in the design of embedded training (…) and practical implications of this somewhat surprising and non-intuitive finding.”
— D. Lain, K. Kostiainen and S. Čapkun, “Phishing in Organizations: Findings from a Large-Scale and Long-Term Study,” 2022 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2022, pp. 842-859, doi: 10.1109/SP46214.2022.9833766.
Other research corroborates these findings:
- Short-Term Gains, Long-Term Losses: Studies have shown that while training can initially reduce the likelihood of falling for phishing attacks, the effectiveness diminishes over time as employees revert to old habits.
- Human Error Is Inevitable: Even knowledgeable employees can fall victim to phishing due to cognitive overload or persuasive content. Training alone may not overcome these factors.
- Negative Impact on Trust: Employees often feel tricked and distrustful after phishing simulations, which can negatively impact morale and trust in the IT department.
Why Relying on Humans Isn’t Enough
Expecting employees to be the last line of defense against sophisticated cyber threats is unrealistic. Cybercriminals are constantly evolving their tactics, making phishing attempts increasingly convincing. A determined attacker can craft messages that are virtually indistinguishable from legitimate communications, especially when leveraging insider information.
With phishing attacks occurring across various platforms—emails, text messages, phone calls, social media, and even postal mail—it’s nearly impossible for employees to remain vigilant at all times. This omnipresence of threats requires a more robust solution than human awareness alone. We should follow the Swiss Cheese Model, where multiple layers of defence are in place, with humans as the last layer.
A Better Approach: Strengthening Technical Defences
Rather than over-relying on employee vigilance, organisations should invest in advanced technical solutions.
- Email Filtering and Threat Detection: Implement sophisticated systems that can identify and block phishing attempts before they reach employees’ inboxes.
- Anomaly Detection: Use algorithms to detect unusual patterns that may indicate an attack.
- Multi-Factor Authentication (MFA): Require MFA to add an extra layer of security, ensuring that even if credentials are compromised, unauthorised access is prevented.
- Zero-Trust Security Models: Adopt a zero-trust approach where every access request is authenticated, before accessing resources. This reduces reliance on perimeter defences and mitigates risks from both external and internal threats.
Nevertheless, sometimes a phishing e-mail still finds its way to a users mailbox. In this case it’s important that you encourage open communication, so employees feel comfortable reporting suspicious activities without fear of reprimand.
Be sure to include a warning banner for suspicious e-mails; as mentioned by the study, these show effectiveness. Keep them short and simple. Here are guides on how to configure them:
- Microsoft Exchange (onPrem): Use Transport rules (Warning: This feature breaks S/MIME, PGP and DKIM signatures)
- Microsoft 365: Use the Set-ExternalInOutlook Cmdlet, or if you use different clients, transport rule as mentioned above.
- EXIM: Use transport filters (Warning: This feature breaks S/MIME, PGP and DKIM signatures)
Conclusion
Phishing exercises, while well-intentioned, often fail to deliver meaningful improvements in organisational security. They can inadvertently increase susceptibility to attacks, damage employee trust, and divert resources from more effective security measures. As cyber threats continue to evolve in sophistication and span multiple communication channels, organisations must adapt by investing in advanced detection methods and fostering a security-aware culture that doesn’t overburden employees with unrealistic expectations. By rethinking our approach and focusing on systemic solutions, we can build a more resilient defence against the ever-changing landscape of cyber threats.
To end this post humorously, there is also another factor that plays into this, as this reddit post beautifully shows 🙂
References
- D. Lain, K. Kostiainen and S. Čapkun, “Phishing in Organizations: Findings from a Large-Scale and Long-Term Study,” 2022 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2022, pp. 842-859, doi: 10.1109/SP46214.2022.9833766. keywords: {Industries;Training;Privacy;Atmospheric measurements;Phishing;Collaboration;Companies;phishing;training;education;user-study},
- Vishwanath, A., Harrison, B., & Ng, Y. J. (2018). Suspicion, Cognition, and Automaticity Model of Phishing Susceptibility. Communication Research, 45(8), 1146-1166. https://doi.org/10.1177/0093650215627483
- Steve Sheng, Bryant Magnien, Ponnurangam Kumaraguru, Alessandro Acquisti, Lorrie Faith Cranor, Jason Hong, and Elizabeth Nunge. 2007. Anti-Phishing Phil: the design and evaluation of a game that teaches people not to fall for phish. In Proceedings of the 3rd symposium on Usable privacy and security (SOUPS ’07). Association for Computing Machinery, New York, NY, USA, 88–99. https://doi.org/10.1145/1280680.1280692