Enhancing Trust and Decision-Making in Phishing Detection with Automated Aids

Problem:
Phishing remains a persistent cybersecurity threat, as users often struggle to identify suspicious emails on their own. Automated decision-support aids, which offer guidance on whether an email is a potential phishing attempt, could significantly reduce the risk of falling for these attacks. However, the effectiveness of these aids hinges on more than just their technical capability—it depends heavily on users' trust in the tool and their willingness to heed its advice. The challenge was to understand how different factors, like the type of aid (human, AI, or text-based), transparency, and feedback, influence users' trust and decision-making when dealing with potential phishing emails.

Solution:
To investigate these factors, we designed a study with 465 participants who were tasked with identifying phishing emails using an automated decision aid. We varied the type of aid (human, AI, text-based, or no aid), the presence of decision transparency (whether the reasoning behind the aid’s suggestion was explained), and whether feedback was given to users after each decision. Additionally, we examined how gender influenced these interactions to gain further insights into trust-building dynamics.

Outcome:
The study revealed several important findings:
- Aid Type: All aids (human, AI, and text-based) were generally effective in helping users identify phishing emails, regardless of their anthropomorphic appearance.
- Transparency: Transparency—explaining why the aid suggested an email was suspicious—was particularly effective with the human aid, increasing user trust. However, this same transparency had little to no impact on trust when using an AI aid.
- Feedback: Feedback was crucial for building trust across all aids, but it had the most significant retention effect for users of the text-based aid. It not only boosted trust but also helped participants remember and use the aid more effectively over time.
- User Experience: Participants generally responded positively to the aids, noting their helpfulness in identifying suspicious emails, regardless of the specific type of aid.

These findings offer actionable insights for improving automated phishing detection tools. By ensuring transparency in decision-making and incorporating feedback mechanisms, designers can build more effective tools that users trust and rely on. The study underscores the importance of designing with the user in mind—especially when dealing with cybersecurity, where trust is paramount.