I was working at a certain security company (my first “multi”) and I was a few months into the job, when I received an email. I don’t remember what exactly it was about, but it smelled phishy and I thought I’ll investigate it. I fired up a vanilla VM, pasted the url there and opened the linked webpage, which resembled the Office 365 login pretty well. Obviously, it was fake. I wondered how far the author went, so I typed in some fake logon information and hit submit. Of course, it responded with some fake error message, but nothing else happened.

Unamused, I destroyed the VM and went back to my work. Then my mailbox pinged, I got an email from security. It was a phishing test and it seems like, I failed. Not only failed, but failed heavily, since I provided credentials. What the hell. And now it was on me to “prove” that no, dude, I did not fail on that obvious phishing email, but tried to check if there’s something behind it. Did I mention this happened at a security company? I got my scolding, my mandatory online clickthrough training and learnt my lesson - I ignored these emails from now on. I learned pretty quickly that most of these emails had a special headers (which was the case for all the phishing tests I ever encountered) that I could filter on. So, I created a filter and went on with my life.

I’m gonna be honest: I created the filter at every job that I ever worked at, if they had a phishing test. For three reasons:

  1. I’m pretty constantly under the gun with one or more phishing campaigns for a while now
  2. I usually don’t read emails for quite a few years now. Once every two days, for 15 seconds max.
  3. I never fell into a real phishing campaign and it had nothing to do with the phishing tests.

But throughout the years, I silently hated these tests, always felt that they do NOTHING for security and I had no proof.

Until recently, when a friend and ex colleague of mine pointed me to two studies that basically vindicated everything I felt.

The Science Says: Phishing Tests Don’t Work (And Might Make Things Worse)

The first study, conducted by researchers at ETH Zurich, is probably one of the most comprehensive phishing studies ever done in a real organization. They ran it for 15 months with over 14,000 employees. And their findings? Well, buckle up.

Embedded training during simulated phishing exercises doesn’t make employees more resilient to phishing. In fact, it can have unexpected side effects that make employees even more susceptible to phishing.

Let me repeat that: The thing that every company does - sending fake phishing emails and then showing a training page when someone clicks - might actually be making things worse.

The researchers found that this approach can lead employees to develop a false sense of security. They start thinking “oh, this must be another training exercise” when they encounter suspicious emails. It normalizes clicking on suspicious links because there are no real consequences. Sound familiar? That’s exactly what happened to me - I learned to ignore them entirely because they were just noise.

But Wait, It Gets Worse: The Psychological Damage

The second study, from researchers at Ruhr University Bochum (published at USENIX Security 2024), looked at something nobody in the industry wants to talk about: the psychological effects of phishing tests on employees.

They studied over 400 employees immediately after they either clicked on or reported a simulated phishing email. What they found should concern every CISO:

Employees who clicked on simulated phishing emails experienced significantly higher stress levels and significantly lower self-efficacy compared to those who reported them.

Let that sink in. These tests are actively harming the psychological well-being of employees. And here’s why that matters:

  • Stress impairs learning and memory. When you’re stressed, your brain literally can’t form new memories or update existing ones as effectively. So that “training” you’re trying to deliver? It’s being delivered at the exact moment when the brain is least capable of absorbing it.

  • Low self-efficacy creates a self-fulfilling prophecy. When people feel they can’t successfully identify phishing, they become less confident in their security decisions. This can lead to either over-cautious behavior (flagging everything as suspicious, creating alert fatigue) or learned helplessness (giving up and clicking anyway).

  • Moderate stress accumulates over time. Even if each individual phishing test causes only moderate stress, these “daily stressors” accumulate and can have serious negative effects on both physical and mental health.

The researchers also found something ironic: even though the participants perceived these phishing campaigns as “positive and effective,” the actual research shows they’re neither. There’s a massive gap between what employees think these tests do and what they actually accomplish.

What Actually Works

Now here’s the interesting part. The same study found that some things do work:

Email warnings are effective. You know, those little yellow banners that say “This email is from outside your organization” or “This email looks suspicious”? Those actually help. They’re passive, they don’t require the user to do anything, and they provide information at the moment of decision.

Crowd-sourced phishing detection works. The researchers deployed a “Report Phishing” button in the email client. Employees used it. A lot. They reported thousands of suspicious emails representing hundreds of real, previously unseen phishing campaigns. The detection was fast - new campaigns could be spotted within minutes of launch. And contrary to what you might expect, the operational overhead was manageable.

Here’s what’s brilliant about this approach: it treats employees as a collective defense mechanism rather than individual points of failure. It respects their intelligence and gives them agency. Instead of “gotcha!” testing, it’s “we’re all in this together.”

The Real Problem with Phishing Tests

The fundamental issue with traditional phishing tests is that they’re based on a flawed model. They assume that:

  1. People click on phishing emails because they don’t know what phishing looks like
  2. Embarrassing people into learning will change their behavior
  3. Repeated testing creates lasting behavioral change

But the research shows none of this is really true. People click for all sorts of reasons - they’re busy, they’re distracted, the email seems legitimate, they’re used to clicking because 99.9% of emails are fine. And my experience? I investigated a suspicious email to see if it was real. That’s literally security work, not security failure.

The punishment-based approach creates a culture of fear and resentment rather than security awareness. It turns security from “something we all care about” into “something that tries to trick me.”

What Should We Do Instead?

Based on the research (and my lived experience), here’s what actually makes sense:

1. Deploy effective email warnings. Use your email system to flag external emails, suspicious patterns, and known phishing indicators. Make these warnings clear and actionable.

2. Make reporting easy and rewarding. Give employees a simple button to report suspicious emails. Thank them when they use it. Use these reports to catch real threats fast.

3. Stop the gotcha games. If you must do phishing simulations, don’t punish people who click. Use the data to improve your technical controls, not to shame individuals.

4. Focus on systemic defenses. Better authentication (like hardware tokens or passkeys), proper email authentication (SPF, DKIM, DMARC), and network segmentation will do more than any amount of training.

5. Respect people’s time and intelligence. Your employees are not stupid. They’re busy humans trying to do their jobs. Design systems that work with human nature, not against it.

The Bottom Line

For years, I thought I was just being cynical about phishing tests. Turns out, the science backs up the cynicism. These programs, as commonly implemented, don’t work and might actively harm security culture.

So to every CISO out there still sending those gotcha emails: stop it. There are better ways. The research is in. We can do better.

And to everyone who’s ever felt that pang of resentment when getting a phishing test email: you’re not alone. Your frustration is valid. And now you have science to back it up.

References: