Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features

Jean Marie Tshimula1,2,3 D’Jeff K. Nkashama1,3 Jean Tshibangu Muabila1,4 René Manassé Galekwa1,2,5
Hugues Kanda1 Maximilien V. Dialufuma1,6 Mbuyi Mukendi Didier1,2,7,8 Kalala Kalonji1,9
Serge Mundele1 Patience Kinshie Lenye1 Tighana Wenge Basele1,2,10 Aristarque Ilunga1,2
Christian N. Mayemba1 Nathanaël M. Kasoro2 Selain K. Kasereka2 Hardy Mikese11
Pierre-Martin Tardif3 Marc Frappier3 Froduald Kabanza3 Belkacem Chikhaoui12

Shengrui Wang3 Ali Mulenda Sumbu13 Xavier Ndona14 Raoul Kienge-Kienge Intudi15
1Groupe de Recherche de Prospection et Valorisation des Données (Greprovad), Global 2Department of Computer Science, University of Kinshasa, DRC 3Department of Computer Science, Université de Sherbrooke, Canada 4LISV-UVSQ, Université Paris-Saclay, France 5University of Klagenfurt, Austria 6Montreal Behavioural Medicine Centre, Centre Intégré Universitaire de Santé et Services Sociaux du Nord-de-l’Île-de-Montréal (CIUSSS-NIM), Canada 7Biomedical Research Unit, Hospital Monkole, Kinshasa, DRC 8University of Florida, USA 9School of Electrical Engineering and Computer Science, University of Ottawa, Canada 10Karlstad University, Sweden 11Institut Supérieur Pédagogique de Kikwit, DRC 12Applied Artificial Intelligence Institute, TELUQ University, Canada 13Faculty of Psychology and Education Sciences, University of Kinshasa, DRC 14Harrisburg University of Science and Technology, USA 15School of Criminology, University of Kinshasa, DRC. Correspondence email: [email protected] and [email protected]
Abstract

The increasing sophistication of cyber threats necessitates innovative approaches to cybersecurity. In this paper, we explore the potential of psychological profiling techniques, particularly focusing on the utilization of Large Language Models (LLMs) and psycholinguistic features. We investigate the intersection of psychology and cybersecurity, discussing how LLMs can be employed to analyze textual data for identifying psychological traits of threat actors. We explore the incorporation of psycholinguistic features, such as linguistic patterns and emotional cues, into cybersecurity frameworks. Our research underscores the importance of integrating psychological perspectives into cybersecurity practices to bolster defense mechanisms against evolving threats.

1 Introduction

Psychological profiling plays a crucial role in cybersecurity, particularly in understanding and identifying the traits and motives of cybercriminals. In computer science, cybersecurity aims to safeguard technology within computer systems, implementing security measures to prevent risks and threats that could harm the system. This field regulates security measures to thwart third-party invaders or intruders who engage in malicious activities such as stealing private, business, or organizational information for personal gain Weimann (2004); Li and Liu (2021); Cremer et al. (2022).

In the domain of cybercrime, understanding the identity and motives of intruders plays a key role in mitigating risks to information security McBrayer (2014); Kumar and Carley (2016); Li (2017); Ablon (2018); Bada and Nurse (2020); Hunter et al. (2021); Chng et al. (2022); Thackray et al. (2016). Psychological profiling emerges as a valuable tool for understanding the psychological traits and characteristics of cybercriminals, which strengthens strategies against potential cyber threats and assists in the identification of intruders and their motives through an examination of behavior, nature, and thought process.

Profiling in cybersecurity involves diverse criminological and criminal-law-based components, encompassing personal traits, criminal expertise, social attributes, and motivational factors. These elements help in understanding the predispositions, personality traits, demographics, socio-economic status, and motivations of cybercriminals, including those who are particularly elusive Hani et al. (2024); Holt et al. (2024).

Cybercriminals frequently exhibit a range of psychological traits that strongly shape their behaviors and actions Bada and Nurse (2020); Chng et al. (2022); Montañez et al. (2020). These individuals often possess a strong command of cyber technology, which they exploit for harmful purposes and various motives; common motives include financial gain, as seen in activities such as data theft and other forms of cyber fraud Li (2017); Holt et al. (2021). Many are driven by greed, pursuing financial rewards, while others seek power or revenge against certain groups or institutions. Some cybercriminals are thrill-seekers, relishing the risk involved in their illicit activities, or opportunists who take advantage of vulnerabilities for personal benefit Thackray et al. (2016); Saroha (2014); McBrayer (2014). There are also those who simply disregard legal and ethical standards, compromising their reputations within the cyber community. Traits of fearlessness, with little regard for potential consequences, and a lack of empathy are also prevalent. Moreover, some individuals demonstrate boldness, testing their hacking abilities against individuals and organizations. Collectively, these traits paint a complex picture of the motivations and behaviors driving cybercriminals in various scenarios Thackray et al. (2016); Li (2017); Madarie (2017); Chng et al. (2022); Maalem Lahcen et al. (2020).

Motivating factors behind cybercriminal personality traits include revenge and blackmailing. Understanding these traits can help minimize security risks and enable better analysis and resolution of cybercrimes Kipane (2019). In addition, integrating findings from Large Language Models (LLMs) and psycholinguistic tools, such as the Linguistic Inquiry and Word Count (LIWC) dictionary and the Medical Research Council (MRC) psycholinguistic database Ke et al. (2024); Boyd et al. (2022); Coltheart (1981), into psychological profiling can significantly enrich the understanding of cybercriminal behaviors and motivations. This holistic approach to psychological profiling can not only reveal the complex personalities of cybercriminals but also strengthen overall security measures, protecting both individuals and organizations from cyber threats. In this paper, we explore the intersection of psychology and cybersecurity, with a specific emphasis on the role of LLMs and psycholinguistic features in profiling cyber threats.

The remainder of this work is organized as follows. Section §2 discusses the fundamental role of psychological profiling in cybersecurity, outlining how it aids in understanding and mitigating the behaviors of cybercriminals. Section §3 explores the application of LLMs in psychological profiling, highlighting their potential to decode complex patterns of cybercriminal activity. In Section §4, we examine the incorporation of psycholinguistic features into cybersecurity strategies, demonstrating how these tools can enhance the precision of psychological profiles. Section §5 discusses different perspectives on psychological profiling in cybersecurity. Section §6 addresses the ethical considerations and privacy implications inherent in the use of psychological profiling and data analysis in cybersecurity. Finally, Section §7 discusses future directions for research in this area and Section §8 concludes the paper with reflections on the evolving landscape of cybersecurity profiling.

2 Psychological Profiling in Cybersecurity

Researchers and practitioners reveal a complex profile of cyber criminals, showcasing traits such as tech-savvy, well-networked, vengeful, goal-oriented, greedy, manipulative, risk-takers, opportunists, rule-breakers, fearless, emotionless, and daring McBrayer (2014); Palassis et al. (2021); Saroha (2014); Thackray et al. (2016); Li (2017); Yang et al. (2018); Holt et al. (2021). More specifically, Saroha (2014) identified a range of characteristics including smartness, creativity, and a need for control, shedding light on the multifaceted nature of individuals involved in cyber crimes, and uncovering motivating factors like monetary gain, thrill-seeking, and political beliefs that drive individuals towards engaging in cyber criminal activities.

In addition to profiling traits, understanding the psychological effects of cybercrime remains essential. Gross et al. (2016) indicated that exposure to cyber terrorism triggers heightened levels of stress and anxiety among individuals, akin to the psychological effects of conventional terrorism, emphasizing the pivotal role of perceived threats in sha** individuals’ attitudes towards government surveillance, regulation, and military responses in the face of cyber threats. Curtis and Oxburgh (2023) underscored the significant influence of law enforcement’s lack of cybercrime knowledge on low conviction rates and victim underreporting. The study revealed that victims often delay reporting cybercrimes due to embarrassment or a perception that they are better equipped to handle the situation themselves. This highlights the importance of training officers to increase their preparedness in dealing with cybercrime cases and engaging with victims.

In a related vein, Palassis et al. (2021) explored the psychological impacts of hacking victimization and underlined the need for support organizations to address these issues. The study underscores the importance of raising awareness about the psychological effects of cybercrime and promoting support opportunities for victims. Its findings provide valuable insights for clinicians and support organizations, informing the development of treatment guidelines and interventions to address the negative psychological impacts of hacking. Gomez and Villar (2018) investigated how limited experience and domain knowledge in cyberspace lead to the use of cognitive shortcuts and inappropriate heuristics, resulting in elevated levels of dread.

In recent investigations, building upon prior research, Geer (2023) highlighted the importance of leveraging cybercriminals’ cognitive biases to influence their behaviors during attacks. The study suggested that by using algorithms informed by cyberpsychology research, defenders can present low-risk, low-reward targets to steer hackers away from high-value assets. Studies show that attackers exhibit risk-averse behavior, preferring attacks on less secure machines to avoid the appearance of failure. Research on human subjects engaging in cybercriminal behavior revealed a strong relationship between key risk-taking and cybercriminal behaviors. Bolton (2019) indicated that participants’ exposure to fictional media, particularly crime-related television shows, can influence their attitudes towards criminal investigations and profiling techniques. The study revealed a correlation between media consumption habits and the perceived realism of investigative procedures portrayed in television episodes. Additionally, participants’ beliefs about the role of criminal profilers and the importance of intuition in investigations were influenced by their media exposure. This underscores the nuanced relationship between media consumption and perceptions of criminal behavior and profiling accuracy.

Expanding upon the evolving understanding of cybercriminal behavior, Lickiewicz (2011) highlighted the significance of intelligence, personality traits, and social skills in the effectiveness of cyber attacks. The study emphasized the role of environmental factors, such as family relationships and educational background, in sha** the behaviors of hackers. It suggested that a holistic approach, considering both individual characteristics and external influences, is crucial for develo** a comprehensive psychological profile of cyber criminals. Additionally, the study noted the need for interdisciplinary collaboration between information technology and investigative psychology to combat cybercrime.

Psychological profiling, rooted in behavioral analysis and psychological theory, aims to uncover patterns and traits indicative of malicious intent in cyber activities. This approach utilizes various aspects of human behavior, such as language use, decision-making processes, and emotional responses, to discern the psychological profiles of threat actors Thackray et al. (2016); Jiang et al. (2018); Kipane (2019); Hani et al. (2024); Budimir et al. (2021); Bada and Nurse (2021); Montañez et al. (2020); Gaia et al. (2020); Zambrano et al. (2023); Kioskli and Polemi (2022). Leveraging techniques from psychology, including personality assessment and psycholinguistic analysis, enables the identification of anomalous behaviors and potential indicators of cyber threats.

For instance, Kioskli and Polemi (2022) emphasized the importance of profiling potential attackers in cybersecurity to enhance the accuracy of vulnerability severity scores using psychological and behavioral traits. Research investigated the influence of cultural and psychological factors on cyber-security behavior, utilizing the Big Five Framework to assess personality traits and their impact on user attitudes towards privacy and self-efficacy Halevi et al. (2016); Odemis et al. (2022). More specifically, Hani et al. (2024) proposed machine learning models for psychological profiling of hackers based on the “Big Five” personality traits model (OCEAN - Openness, Conscientiousness, Extroversion, Agreeableness, Neuroticism) and their models achieved 88% accuracy in map** personality clusters with different types of hackers (White Hat, Grey Hat, etc.), identifying cyber-criminal behaviors. Gaia et al. (2020) discovered that individuals attracted to hacking exhibit high scores on Machiavellianism and Psychopathy scales, with Grey Hat hackers showing opposition to authority, Black Hat hackers scoring high on thrill-seeking, and White Hat hackers displaying tendencies towards Narcissism. The Dark Triad traits significantly predict interest in different types of hacking, while thrill-seeking emerges as a key motivator for Black Hat hackers. Perceptions of apprehension for violating privacy laws negatively impact Grey Hat and Black Hat hacking.

Table 1: Summary of LLM applications in psychological profiling in cybersecurity
Research Focus Cybersecurity applications Sources of data
Petrov et al. (2024) Simulating human psychological behaviors using LLMs Evaluating psychometric properties for profiling potential threats Standardized personality constructs
Pellert et al. (2023) Repurposing psychometric inventories for LLMs Profiling values, morality, and beliefs to detect radicalization Standard psychometric inventories
Sorokovikova et al. (2024) Fine-tuning LLMs on Big Five traits Profiling based on language to identify potential threats Psychometric test items
Safdari et al. (2023) Administering personality tests on LLMs Mimicking specific human personality profiles for threat detection Personality tests
Huang et al. (2023) PsychoBench framework for evaluating LLM personalities Understanding complex psychological profiles for enhanced cybersecurity Personality traits, interpersonal relationships, motivational tests, emotional abilities
Frisch and Giulianelli (2024) Conditioning LLM agents on personality profiles Mimicking human traits for improved phishing and social engineering detection Persona conditioning data
Yamin et al. (2021) Weaponized use of LLMs in cyber attacks Generating malicious code, automated hacking, phishing Training data on malware and exploits
Motlagh et al. (2024) Generating malicious payloads with LLMs Creating new strains of malware Relevant cybersecurity data
Beckerich et al. (2023) Using LLMs for automated hacking Vulnerability scanning and develo** exploits Hacking toolkits
Schmitt and Flechais (2023) Social engineering and phishing Mimicking human language for cyber attacks Historical phishing data
Zhang et al. (2024) PsySafe for framework understanding and mitigating risks arising from dark psychological states Identifying vulnerabilities, evaluating safety, and implementing defense mechanisms Psychological assessments, behavioral evaluations

Moreover, Kipane (2019) revealed that cybercriminals exhibit a range of behaviors and traits that deviate from societal norms, influenced by factors such as heredity, education, culture, and socio-economic status. Profiling methods focus on identifying key psychological features, modus operandi, and criminal motivations to aid in early detection and investigation of cybercrimes. The study emphasizes the significance of expert knowledge and advanced technologies in enhancing law enforcement efforts to combat cybercrime. Overall, the research underscores the evolving nature of criminal profiling in the digital era and the critical role it plays in addressing the growing threat of cybercriminal activities. In response to the escalating threat posed by cybercrimes, Thackray et al. (2016) highlighted the diverse motivations of hackers, including recreation, prestige, revenge, profit, and ideology, which influence their engagement in cyber activities. The study underscores the importance of not only teaching coding skills but also educating individuals about the risks and consequences of online actions to prevent cyber-crime involvement. Additionally, the research emphasizes the need to identify at-risk groups and individuals to target awareness campaigns and promote informed online behavior for future generations. Lastly, the study suggests that understanding social psychological theories can enhance communication with hacker communities and individuals, ultimately contributing to more effective cybersecurity practices.

3 LLMs in Psychological Profiling

Large Language Models (LLMs), such as OpenAI’s GPT series of models, Google’s PaLM and Gemini, and Meta’s LLaMA family of open-source models, have demonstrated remarkable capabilities in natural language understanding and generation tasks Minaee et al. (2024). As these models continue to evolve and become more sophisticated, researchers and practitioners are exploring their potential applications beyond language tasks, venturing into the realm of psychological profiling (see Table 1). These models are utilized to profile individuals based on their language use patterns and communication styles, facilitating the early detection of potential threats Ke et al. (2024).

The potential applications of LLM-based psychological profiling are vast and diverse Abdurahman et al. (2024); Ke et al. (2024); Hani et al. (2024); Pellert et al. (2023); Petrov et al. (2024); Huang et al. (2023). In mental health settings, these techniques aid in the early detection of psychological disorders and the development of personalized treatment plans Lai et al. (2023); Chung et al. (2023); Hagendorff (2023). In human-AI interaction, understanding the perceived personalities of LLMs improves user engagement and trust, leading to more natural and effective interactions Sharma et al. (2024).

However, the application of LLMs to psychological profiling is not without challenges and ethical considerations. Existing personality models and assessment methods have been developed primarily for human subjects, and their suitability for evaluating artificial intelligence systems is questionable. Additionally, the fluid and context-dependent nature of LLM “personalities” raises concerns about the reliability and validity of traditional personality assessment techniques when applied to these models Sorokovikova et al. (2024). As researchers delve deeper into this emerging field, they must grapple with the complexities of transferring human-centric concepts like personality to artificial intelligence systems. LLMs are explored for psychological profiling tasks, such as detecting personality traits, values, and other non-cognitive characteristics Hani et al. (2024); Frisch and Giulianelli (2024); Pellert et al. (2023); Petrov et al. (2024); Huang et al. (2023); Song et al. (2024); Safdari et al. (2023); Hani et al. (2024); Sorokovikova et al. (2024); Zhang et al. (2024).

In exploring the multifaceted landscape of psychological profiling with LLMs, researchers have embarked on various avenues to understand their potential applications. For instance, Petrov et al. (2024) focused on investigating the ability of LLMs to simulate human psychological behaviors using prompts to adopt different personas and respond to standardized measures of personality constructs to assess their psychometric properties. Pellert et al. (2023) repurposed standard psychometric inventories originally designed for assessing human psychological characteristics, such as personality traits, values, morality, and beliefs, to evaluate analogous traits in LLMs. Sorokovikova et al. (2024) fine-tuned LLMs on psychometric test items related to the Big Five personality traits for evaluating personalities based on language. Safdari et al. (2023) introduced a method for administering personality tests on LLMs and sha** their generated text to mimic specific human personality profiles.

Furthermore, Huang et al. (2023) proposed PsychoBench, a framework for evaluating personality traits, interpersonal relationships, motivational tests, and emotional abilities to uncover complex psychological profiles within LLMs and their potential integration into human society as empathetic and personalized AI-driven solutions. Frisch and Giulianelli (2024) demonstrated that LLM agents conditioned on personality profiles can mimic human traits, with creative personas displaying more consistent behavior in both interactive and non-interactive conditions; the research highlights the importance of robust persona conditioning in sha** LLM behavior and emphasizes the asymmetry in linguistic alignment between different persona groups during interactions.

Zhang et al. (2024) presented PsySafe, a framework designed to evaluate and improve the safety of multi-agent systems (MAS) by addressing the psychological aspects of agent behavior. PsySafe incorporates dark personality traits to assess and mitigate potential risks associated with agent behaviors in MAS; in addition, it includes identifying vulnerabilities, evaluating safety from psychological and behavioral perspectives, and implementing effective defense strategies. The findings yielded by PsySafe reveal several phenomena, including collective dangerous behaviors among agents, their self-reflection on engaging in such behaviors, and the correlation between psychological assessments and behavioral safety.

While LLMs offer promising applications in psychological profiling, their language generation capabilities also raise concerns about potential misuse for cyber attacks and malicious activities Yamin et al. (2021); Motlagh et al. (2024); Gupta et al. (2023); Yao et al. (2024). Attack payloads and malware creation involve LLMs generating malicious code or new strains of malware through training on relevant data Beckerich et al. (2023); Wu et al. (2023). Automated hacking and vulnerability scanning tasks can be performed by LLMs, including generating code for automated hacking attacks, scanning software for vulnerabilities, or develo** exploits Wu et al. (2023); Xu et al. (2024).

In addition, LLMs can be used for social engineering and phishing purposes, leveraging their ability to mimic human language patterns to create convincing social engineering attacks, phishing emails, or disinformation campaigns Schmitt and Flechais (2023). Adversaries could potentially manipulate LLM outputs for malicious purposes using prompt injection techniques Liu et al. (2023); Piet et al. (2023). LLMs can generate highly personalized and persuasive phishing emails tailored to specific individuals within an organization, bypassing traditional detection systems. Studies show these AI-crafted attacks can be strikingly effective, with around 10% of recipients entering credentials on fake login portals Bethany et al. (2024). The ability of LLMs to mimic human language patterns and adapt to different contexts makes them a powerful tool for deception and manipulation Prome et al. (2024).

The 2023 Report of Voice of SecOps provides a comprehensive analysis of threats and stressors posed by LLMs, revealing that 51% of security professionals are likely to leave their job within 2024.111Generative AI and Cybersecurity: Bright Future or Business Battleground? Deep Instinct. (2023). Voice of SecOps Reports. Retrieved from https://www.deepinstinct.com/voice-of-secops-reports. Accessed on May 12, 2024. The study surveyed over 650 senior security operations professionals in the U.S. to assess LLMs’ impact on the cybersecurity industry. Findings indicate a 75% surge in attacks in 2022, with 85% attributing this increase to bad actors leveraging LLMs. Furthermore, 70% of respondents believe LLMs positively influence employee productivity and collaboration, while 63% perceive an enhancement in employee morale. Ransomware emerges as the greatest threat to organizational data security, with 46% of respondents acknowledging its severity and 62% indicating it as the top C-suite concern, a notable increase from 44% in 2022; the pressure to combat ransomware has prompted organizations to revise their data security strategies, with 47% now possessing a policy to pay the ransom, compared to 34% in the previous year. Moreover, the report reveals a 55% increase in stress levels among security professionals, primarily attributed to staffing and resource constraints, cited by 42% of respondents.

4 Psycholinguistic Features

Psycholinguistic features encompass a wide range of linguistic attributes and psychological constructs that reflect cognitive and emotional aspects of language use. Integrating psycholinguistic features into cybersecurity frameworks enhances the granularity of threat profiling techniques and enables a deeper understanding of cybercriminals’ mental states and feelings Jiang et al. (2018); Deb et al. (2018); Uyheng et al. (2022); Krylova-Grek (2019); Xu and Rajivan (2023). Psycholinguistic features include sentiment analysis, linguistic complexity measures, lexical diversity metrics, and stylistic characteristics. Through advanced text analysis algorithms and machine learning algorithms, these features can be leveraged to identify anomalous patterns indicative of malicious intent.

One of the powerful tools in psycholinguistic analysis is the Linguistic Inquiry and Word Count (LIWC) dictionary Boyd et al. (2022). In the context of cyber attacks, LIWC has been used to detect deception in phishing emails by analyzing the psycholinguistic features that attackers employ to deceive end-users Xu and Rajivan (2023). Research shows that phishers often use language conveying certainty (e.g. always, never), time pressure and work-related words to increase vulnerability of targets. Conversely, reward-related words like money or cash tend to decrease vulnerability as they are associated with scams. Beyond phishing, LIWC has been applied to study online predator behavior, analyze developer personalities, model social media rumors, and understand user reactions in crowdsourcing Rogers et al. (2006); Tausczik and Pennebaker (2010); Shappie et al. (2020); Kranenbarg et al. (2023); Budimir et al. (2021).

Building on the potential of LIWC for psycholinguistic analysis in cybersecurity, researchers explore its applications to understand attacker behavior and victim vulnerabilities. More precisely, Guo et al. (2023) focused on analyzing the vulnerability factors of potential victims to cybergrooming using LIWC to quantify and understand the social-psychological traits that may make individuals more susceptible to online grooming; they reveal significant correlations between specific vulnerability dimensions and the likelihood of being targeted as a victim of cybergrooming. Interestingly, the research observed negative correlations between victims and certain family and community-related traits, challenging conventional beliefs about the key factors contributing to vulnerability in online contexts. Tan et al. (2019) utilize LIWC and demonstrate that malicious insiders exhibit specific linguistic patterns in their written communications, including increased use of self-focused words, negative language, and cognitive process-related words compared to other team members; as insiders become more detached from the team, language similarity decreases over time.

In a different angle, psycholinguistic features were utilized to examine the manipulative aspects of cybercrimes. More specifically, Krylova-Grek (2019) investigated the psycholinguistic dimensions of social engineering within cybersecurity, employing activity theory to dissect the methods and techniques utilized by malicious actors. This research reveals the sophisticated tactics employed by social engineers to manipulate emotions, impede critical thinking, and exploit moral values to influence user behavior and extract sensitive information. Parapar et al. (2014) proposed a machine learning model for detecting sexual predation in chatrooms using psycholinguistic, content-based, and chat-based features, and show distinct characteristics that differentiate predators from non-predators. Particularly, Rogers et al. (2006) investigated the psychological traits and behaviors of individuals involved in self-reported criminal computer activities, emphasizing the role of extraversion in predicting such behavior and challenging stereotypes by shedding light on the complexities of personality factors in criminal/deviant computer behavior through the use of Likert-scale questionnaires and psychometric instruments.

Furthermore, Chatterjee and Basu (2021) conducted a study on phishing influence detection using a novel computational psycholinguistic analysis approach to identify influential sentences that could potentially lead to security breaches and hacking in online transactions and social media interactions, develo** a language and domain-independent computational model based on Cialdini’s principles of persuasion.222The 6 Principles of Persuasion: Tips from the leading expert on social influence, Douglas T. Kenrick. Posted Dec. 8, 2012. Retrieved from https://www.psychologytoday.com/ca/blog/sex-murder-and-the-meaning-of-life/201212/the-6-principles-of-persuasion. Accessed May 20, 2024. Kranenbarg et al. (2023) indicated that cyber offenders displayed similarities to the community sample on certain traits but exhibited differences from offline offenders, particularly in conscientiousness and openness to experience. Notably, cyber offenders showed lower scores on honesty-humility compared to the community sample, suggesting potential implications for intervention strategies targeting specific personality traits in this population.

Budimir et al. (2021) emphasized the importance of understanding psycholinguistic features and psychology in cybersecurity to develop effective strategies and interventions. They explore the emotional responses triggered by cybersecurity breaches, focusing on the hacking of smart security cameras. The study identifies a 3-dimensional structure of emotional reactions, highlighting negative affectivity, proactive versus fight/flight action tendencies, and emotional intensity and valence. Personality characteristics, such as the Big Five traits and resilient/overcontrolled/undercontrolled types, were found to relate to these emotional dimensions.

Recently, the application of sentiment analysis techniques has paved the way for building psychological profiles and detecting and understanding cyber threats. Sapienza et al. (2017) utilized sentiment analysis to identify discussions around exploits, vulnerabilities, and attack planning on dark web forums even before these threats manifest in the real world, and to provide early warnings through the observation of changes in sentiment and semantic context. Deb et al. (2018) proposed approaches to predict cyber-events by leveraging sentiment analysis on hacker forums and social media to analyze the sentiment expressed in online discussions and detect signals that may precede cyber attacks. Jiang et al. (2018) built user psychological profiles based on the sentiment analysis of their network browsing and email content, and demonstrate that this approach can proactively and accurately detect malicious insiders with extreme or negative emotional tendencies.

Building upon recent studies and advancements, Uyheng et al. (2022) developed a machine learning model called TrollHunter and collected a dataset of online trolling messages and found that troll messages exhibit more abusive language, lower cognitive complexity, and greater targeting of named entities and identities; the model achieved an 89% accuracy rate and F1 score in identifying trolling behavior.

5 Discussion

The integration of psychological profiling into cybersecurity practices offers a multifaceted approach to understanding and mitigating cyber threats. LLMs and psycholinguistic features provide deeper understanding into the behaviors, motivations, and emotional states of cybercriminals. This discussion section explores the potential benefits, and challenges of these techniques, drawing from the research findings presented earlier.

5.1 Benefits of Psychological Profiling in Cybersecurity

Psychological profiling in cybersecurity holds significant promise. Identifying psychological traits and patterns in cybercriminal behavior enables security professionals to anticipate and preemptively counteract potential threats. For instance, understanding the personality traits and motivations of different types of hackers (e.g., White Hat, Black Hat, Grey Hat) allows for more tailored security measures and interventions Hani et al. (2024); Gaia et al. (2020). The use of LLMs enhances this profiling by analyzing large volumes of text data, identifying linguistic patterns that may indicate malicious intent.

Psycholinguistic features, such as those derived from the LIWC dictionary, provide additional granularity. These features help in detecting subtle cues in language that might indicate deception, stress, or malicious intent. For example, certain linguistic markers can distinguish phishing emails from legitimate communications, thereby improving the accuracy of threat detection systems Xu and Rajivan (2023); Rogers et al. (2006).

Moreover, the incorporation of psychological profiling can aid in the development of more personalized cybersecurity training programs. Understanding the psychological traits that make individuals more susceptible to cyber attacks allows organizations to design targeted awareness campaigns and training modules that address specific vulnerabilities.

5.2 Challenges and Limitations

Despite the promising applications, several challenges and limitations need to be addressed. One major challenge is the accuracy and reliability of psychological profiling techniques.

While LLMs and psycholinguistic tools provide valuable insights, they come with inherent limitations. Implementing and maintaining these advanced profiling systems require a workforce equipped with specialized skills in artificial intelligence, cybersecurity, and psychological analysis. There is often a shortage of professionals with the necessary expertise to develop, deploy, and refine these tools. Addressing this skill gap is crucial for the effective utilization of psychological profiling in cybersecurity.

The effectiveness of LLMs largely depends on the quality and diversity of the data they are trained on. Inaccurate models can result from poor-quality data, such as poisoned or contaminated datasets, or from non-representative data. Moreover, acquiring diverse and representative datasets is particularly challenging in the field of cybersecurity, where data sensitivity and proprietary information are significant concerns.

Additionally, the use of these tools can lead to false positives and negatives, causing either unnecessary alarms or undetected threats. Thus, ensuring the robustness and validity of these models is vital for their successful deployment in real-world scenarios Xu and Rajivan (2023); Hani et al. (2024).

Another challenge lies in the dynamic and evolving nature of cybercriminal behavior. Cybercriminals continually adapt their tactics to evade detection, which means that profiling techniques must also evolve. Continuous updates and refinements to the models and algorithms are necessary to keep pace with these changes.

The ethical implications of psychological profiling in cybersecurity cannot be overlooked. The use of personal data to create psychological profiles raises significant privacy concerns. It is essential to balance the benefits of enhanced security with the protection of individual privacy rights. Transparent policies and stringent data protection measures must be in place to ensure that the use of psychological profiling does not infringe on personal freedoms.

6 Ethical Considerations

Ethical considerations are paramount when employing psychological profiling in cybersecurity. The potential for misuse of these technologies for surveillance, manipulation, or discrimination is a serious concern. For example, the ability of LLMs to generate persuasive phishing emails tailored to specific individuals poses a significant threat if used maliciously Liyanage and Ranaweera (2023).

To mitigate these risks, it is crucial to establish ethical guidelines and regulatory frameworks that govern the use of psychological profiling tools. These guidelines should emphasize the importance of informed consent, data minimization, and transparency in the use of personal data. Additionally, there should be mechanisms for accountability and oversight to ensure that these technologies are used responsibly and ethically McStay (2020); Fleming (2021).

7 Future Directions

Future research should focus on improving the robustness of psychological profiling techniques. This includes develo** more sophisticated models that can adapt to the evolving tactics of cybercriminals and integrating multimodal data sources (e.g., text, behavioral data, biometric data) to create more comprehensive profiles.

Another promising direction is the exploration of collaborative approaches that combine human expertise with machine intelligence. Human analysts and AI systems can collaborate to achieve more effective and nuanced threat detection and mitigation strategies.

Finally, ongoing efforts to address the ethical and privacy concerns associated with psychological profiling are essential. This includes develo** new methods for anonymizing and protecting personal data while still enabling meaningful analysis, as well as fostering a culture of ethical awareness and responsibility among cybersecurity professionals.

8 Conclusion

The integration of psychological profiling, LLMs, and psycholinguistic features into cybersecurity practices represents a significant advancement in the field. These techniques offer the potential to enhance threat detection and mitigation strategies by providing deeper understanding into the behaviors and motivations of cybercriminals. However, realizing this potential requires addressing the challenges and ethical considerations associated with these technologies. By doing so, we can create more robust and responsible cybersecurity frameworks that protect both organizations and individuals from evolving cyber threats.

Acknowledgments

The authors thank all Greprovad members for helpful discussions and comments on early drafts.

References

  • Abdurahman et al. (2024) S. Abdurahman, A.S. Ziabari, A. Moore, D. Bartels, and M. Dehghani. 2024. Evaluating large language models in psychological research: A guide for reviewers.
  • Ablon (2018) L. Ablon. 2018. Data thieves: The motivations of cyber threat actors and their use and monetization of stolen data. RAND Corporation Santa Monica, CA, USA.
  • Bada and Nurse (2020) M. Bada and J.R.C Nurse. 2020. The social and psychological impact of cyberattacks. In Emerging cyber threats and cognitive vulnerabilities, pages 73–92.
  • Bada and Nurse (2021) M. Bada and J.R.C. Nurse. 2021. Profiling the cybercriminal: A systematic review of research. In 2021 international conference on cyber situational awareness, data analytics and assessment, pages 1–8.
  • Beckerich et al. (2023) M. Beckerich, L. Plein, and S. Coronado. 2023. Ratgpt: Turning online llms into proxies for malware attacks. arXiv preprint arXiv:2308.09183.
  • Bethany et al. (2024) M. Bethany, A. Galiopoulos, E. Bethany, M.B. Karkevandi, N. Vishwamitra, and P. Najafirad. 2024. Large language model lateral spear phishing: A comparative study in large-scale organizational settings. arXiv preprint arXiv:2401.09727.
  • Bolton (2019) A. Bolton. 2019. Media effects and criminal profiling: How fiction influences perception and profile accuracy. Ph.D. thesis, Nova Southeastern University.
  • Boyd et al. (2022) R.L. Boyd, A. Ashokkumar, S. Seraj, and J.W. Pennebaker. 2022. The development and psychometric properties of liwc-22. Austin, TX: University of Texas at Austin, pages 1–47.
  • Budimir et al. (2021) S. Budimir, J.R.J. Fontaine, N.M.A. Huijts, A. Haans, G. Loukas, and E.B. Roesch. 2021. Emotional reactions to cybersecurity breach situations: scenario-based survey study. Journal of medical Internet research, 23(5):e24879.
  • Chatterjee and Basu (2021) A. Chatterjee and S. Basu. 2021. How vulnerable are you? a novel computational psycholinguistic analysis for phishing influence detection. In Proceedings of the 18th International Conference on Natural Language Processing, pages 499–507.
  • Chng et al. (2022) S. Chng, H.Y. Lu, A. Kumar, and D. Yau. 2022. Hacker types, motivations and strategies: A comprehensive framework. Computers in Human Behavior Reports, 5:100167.
  • Chung et al. (2023) N.C. Chung, G. Dyer, and L. Brocki. 2023. Challenges of large language models for mental health counseling. arXiv preprint arXiv:2311.13857.
  • Coltheart (1981) M. Coltheart. 1981. The mrc psycholinguistic database. The Quarterly Journal of Experimental Psychology Section A, 33(4):497–505.
  • Cremer et al. (2022) F. Cremer, B. Sheehan, M. Fortmann, A.N. Kia, M. Mullins, F. Murphy, and S. Materne. 2022. Cyber risk and cybersecurity: a systematic review of data availability. The Geneva Papers on risk and insurance-Issues and practice, 47(3):698–736.
  • Curtis and Oxburgh (2023) J. Curtis and G. Oxburgh. 2023. Understanding cybercrime in ‘real world’policing and law enforcement. The Police Journal, 96(4):573–592.
  • Deb et al. (2018) A. Deb, K. Lerman, and E. Ferrara. 2018. Predicting cyber-events by leveraging hacker sentiment. Information, 9(11):280.
  • Fleming (2021) M.N. Fleming. 2021. Considerations for the ethical implementation of psychological assessment through social media via machine learning. Ethics & behavior, 31(3):181–192.
  • Frisch and Giulianelli (2024) I. Frisch and M. Giulianelli. 2024. Llm agents in interaction: Measuring personality consistency and linguistic alignment in interacting populations of large language models. arXiv preprint arXiv:2402.02896.
  • Gaia et al. (2020) J. Gaia, B. Ramamurthy, G. Sanders, S. Sanders, S. Upadhyaya, X. Wang, and C. Yoo. 2020. Psychological profiling of hacking potential. In Proceedings of the 53rd Hawaii International Conference on System Sciences.
  • Geer (2023) D. Geer. 2023. Using psychology to bolster cybersecurity. Communications of the ACM, 66(10):15–17.
  • Gomez and Villar (2018) M.A. Gomez and E.B. Villar. 2018. Fear, uncertainty, and dread: Cognitive heuristics and cyber threats. Politics and Governance, 6(2):61–72.
  • Gross et al. (2016) M.L. Gross, D. Canetti, and D.R. Vashdi. 2016. The psychological effects of cyber terrorism. Bulletin of the Atomic Scientists, 72(5):284–291.
  • Guo et al. (2023) Z. Guo, P. Wang, J.-H. Cho, and L. Huang. 2023. Text mining-based social-psychological vulnerability analysis of potential victims to cybergrooming: Insights and lessons learned. In Companion Proceedings of the ACM Web Conference 2023, pages 1381–1388.
  • Gupta et al. (2023) M. Gupta, C. Akiri, K. Aryal, E. Parker, and L. Praharaj. 2023. From chatgpt to threatgpt: Impact of generative ai in cybersecurity and privacy. IEEE Access.
  • Hagendorff (2023) T. Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. arXiv preprint arXiv:2303.13988.
  • Halevi et al. (2016) T. Halevi, N. Memon, J. Lewis, P. Kumaraguru, S. Arora, N. Dagar, F. Aloul, and J. Chen. 2016. Cultural and psychological factors in cyber-security. In Proceedings of the 18th International Conference on Information Integration and Web-based Applications and Services, pages 318–324.
  • Hani et al. (2024) U. Hani, O. Sohaib, K. Khan, A. Aleidi, and N. Islam. 2024. Psychological profiling of hackers via machine learning toward sustainable cybersecurity. Frontiers in Computer Science, 6:1381351.
  • Holt et al. (2024) T.J. Holt, S.M. Chermak, J.D. Freilich, N. Turner, and E. Greene-Colozzi. 2024. Assessing racial and ethnically motivated extremist cyberattacks using open source data. Terrorism and Political Violence, 36(1):113–126.
  • Holt et al. (2021) T.J. Holt, M. Stonhouse, J. Freilich, and S.M. Chermak. 2021. Examining ideologically motivated cyberattacks performed by far-left groups. Terrorism and political violence, 33(3):527–548.
  • Huang et al. (2023) J. Huang, W. Wang, E.J. Li, M.H. LAM, S. Ren, Y. Yuan, W. Jiao, Z. Tu, and M. Lyu. 2023. On the humanity of conversational ai: Evaluating the psychological portrayal of llms. In ICLR.
  • Hunter et al. (2021) L.Y. Hunter, C.D. Albert, and E. Garrett. 2021. Factors that motivate state-sponsored cyberattacks. The Cyber Defense Review, 6(2):111–128.
  • Jiang et al. (2018) J. Jiang, J. Chen, K.K.R. Choo, K. Liu, C. Liu, M. Yu, and P. Mohapatra. 2018. Prediction and detection of malicious insiders’ motivation based on sentiment profile on webpages and emails. In 2018 IEEE Military Communications Conference, pages 1–6. IEEE.
  • Ke et al. (2024) L. Ke, S. Tong, P. Chen, and K. Peng. 2024. Exploring the frontiers of llms in psychological applications: A comprehensive review. arXiv preprint arXiv:2401.01519.
  • Kioskli and Polemi (2022) K. Kioskli and N. Polemi. 2022. Estimating attackers’ profiles results in more realistic vulnerability severity scores. In 13th International Conference on Applied Human Factors and Ergonomics (AHFE 2022).
  • Kipane (2019) A. Kipane. 2019. Meaning of profiling of cybercriminals in the security context. In SHS Web of Conferences, volume 68, page 01009. EDP Sciences.
  • Kranenbarg et al. (2023) M.W. Kranenbarg, J.-L. Van Gelder, A.J. Barends, and R.E. de Vries. 2023. Is there a cybercriminal personality? comparing cyber offenders and offline offenders on hexaco personality domains and their underlying facets. Computers in human behavior, 140:107576.
  • Krylova-Grek (2019) Y. Krylova-Grek. 2019. Psycholinguistic aspects of humanitarian component of cybersecurity. Psycholinguistics, 26(1):199–215.
  • Kumar and Carley (2016) S. Kumar and K.M. Carley. 2016. Approaches to understanding the motivations behind cyber attacks. In 2016 IEEE Conference on Intelligence and Security Informatics (ISI), pages 307–309. IEEE.
  • Lai et al. (2023) T. Lai, Y. Shi, Z. Du, J. Wu, K. Fu, Y. Dou, and Z. Wang. 2023. Psy-llm: Scaling up global mental health psychological services with ai-based large language models. arXiv preprint arXiv:2307.11991.
  • Li (2017) X. Li. 2017. A review of motivations of illegal cyber activities. Kriminologija & socijalna integracija: časopis za kriminologiju, penologiju i poremećaje u ponašanju, 25(1):110–126.
  • Li and Liu (2021) Y. Li and Q. Liu. 2021. A comprehensive review study of cyber-attacks and cyber security; emerging trends and recent developments. energy reports, 7, 8176–8186.
  • Lickiewicz (2011) J. Lickiewicz. 2011. Cyber crime psychology-proposal of an offender psychological profile. Problems of forensic sciences, 2(3):239–252.
  • Liu et al. (2023) Y. Liu, G. Deng, Y. Li, K. Wang, T. Zhang, Y. Liu, H. Wang, Y. Zheng, and Y. Liu. 2023. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499.
  • Liyanage and Ranaweera (2023) U.P. Liyanage and N.D. Ranaweera. 2023. Ethical considerations and potential risks in the deployment of large language models in diverse societal contexts. J of Computational Social Dynamics, 8(11):15–25.
  • Maalem Lahcen et al. (2020) R.A. Maalem Lahcen, B. Caulkins, R. Mohapatra, and M. Kumar. 2020. Review and insight on the behavioral aspects of cybersecurity. Cybersecurity, 3:1–18.
  • Madarie (2017) R. Madarie. 2017. Hackers’ motivations: Testing schwartz’s theory of motivational types of values in a sample of hackers. International Journal of Cyber Criminology, 11(1).
  • McBrayer (2014) J. McBrayer. 2014. Exploiting the digital frontier: hacker typology and motivation. The University of Alabama.
  • McStay (2020) A. McStay. 2020. Emotional ai, soft biometrics and the surveillance of emotional life: An unusual consensus on privacy. Big Data & Society, 7(1):2053951720904386.
  • Minaee et al. (2024) S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao. 2024. Large language models: A survey. arXiv preprint arXiv:2402.06196.
  • Montañez et al. (2020) R. Montañez, E. Golob, and S. Xu. 2020. Human cognition through the lens of social engineering cyberattacks. Frontiers in psychology, 11:528099.
  • Motlagh et al. (2024) F.N. Motlagh, M. Hajizadeh, M. Majd, P. Najafi, F. Cheng, and C. Meinel. 2024. Large language models in cybersecurity: State-of-the-art. arXiv preprint arXiv:2402.00891.
  • Odemis et al. (2022) M. Odemis, C. Yucel, A. Koltuksuz, et al. 2022. Detecting user behavior in cyber threat intelligence: Development of honeypsy system. Security and Communication Networks, 2022.
  • Palassis et al. (2021) A. Palassis, C.P Speelman, and J.A. Pooley. 2021. An exploration of the psychological impact of hacking victimization. Sage Open, 11(4):21582440211061556.
  • Parapar et al. (2014) J. Parapar, D.E. Losada, and A. Barreiro. 2014. Combining psycho-linguistic, content-based and chat-based features to detect predation in chatrooms. J. Univers. Comput. Sci., 20(2):213–239.
  • Pellert et al. (2023) M. Pellert, C.M. Lechner, C. Wagner, B. Rammstedt, and M. Strohmaier. 2023. Ai psychometrics: Assessing the psychological profiles of large language models through psychometric inventories. Perspectives on Psychological Science, page 17456916231214460.
  • Petrov et al. (2024) N.B. Petrov, G. Serapio-García, and J. Rentfrow. 2024. Limited ability of llms to simulate human psychological behaviours: a psychometric analysis. arXiv preprint arXiv:2405.07248.
  • Piet et al. (2023) J. Piet, M. Alrashed, C. Sitawarin, S. Chen, Z. Wei, E. Sun, B. Alomair, and D. Wagner. 2023. Jatmo: Prompt injection defense by task-specific finetuning. arXiv preprint arXiv:2312.17673.
  • Prome et al. (2024) S.A. Prome, N.A. Ragavan, M.R. Islam, D. Asirvatham, and A.J. Jegathesan. 2024. Deception detection using ml and dl techniques: A systematic review. Natural Language Processing Journal, page 100057.
  • Rogers et al. (2006) M.K. Rogers, K. Seigfried, and K. Tidke. 2006. Self-reported computer criminal behavior: A psychological analysis. digital investigation, 3:116–120.
  • Safdari et al. (2023) M. Safdari, G. Serapio-García, C. Crepy, S. Fitz, P. Romero, L. Sun, M. Abdulhai, A. Faust, and M. Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184.
  • Sapienza et al. (2017) A. Sapienza, A. Bessi, S. Damodaran, P. Shakarian, K. Lerman, and E. Ferrara. 2017. Early warnings of cyber threats in online discussions. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pages 667–674. IEEE.
  • Saroha (2014) R. Saroha. 2014. Profiling a cyber criminal. International Journal of Information and Computation Technology, 4(3):253–258.
  • Schmitt and Flechais (2023) M. Schmitt and I. Flechais. 2023. Digital deception: Generative artificial intelligence in social engineering and phishing. arXiv preprint arXiv:2310.13715.
  • Shappie et al. (2020) A.T. Shappie, C.A. Dawson, and S.M. Debb. 2020. Personality as a predictor of cybersecurity behavior. Psychology of Popular Media, 9(4):475.
  • Sharma et al. (2024) A. Sharma, S. Rao, C. Brockett, A. Malhotra, N. Jojic, and W.B. Dolan. 2024. Investigating agency of llms in human-ai collaboration tasks. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1968–1987.
  • Song et al. (2024) X. Song, Y. Adachi, J. Feng, M. Lin, L. Yu, F. Li, A. Gupta, G. Anumanchipalli, and S. Kaur. 2024. Identifying multiple personalities in large language models with external evaluation. arXiv preprint arXiv:2402.14805.
  • Sorokovikova et al. (2024) A. Sorokovikova, N. Fedorova, S. Rezagholi, and I.P. Yamshchikov. 2024. Llms simulate big five personality traits: Further evidence. arXiv preprint arXiv:2402.01765.
  • Tan et al. (2019) S.-S. Tan, J. Na, and S. Duraisamy. 2019. Unified psycholinguistic framework: an unobtrusive psychological analysis approach towards insider threat prevention and detection. Journal of Information Science Theory and Practice, 7(1):52–71.
  • Tausczik and Pennebaker (2010) Y.R. Tausczik and J.W. Pennebaker. 2010. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54.
  • Thackray et al. (2016) H. Thackray, J. McAlaney, H. Dogan, J. Taylor, and C. Richardson. 2016. Social psychology: An under-used tool in cybersecurity. BCS Human Computer Interaction Conference.
  • Uyheng et al. (2022) J. Uyheng, J.D. Moffitt, and K.M. Carley. 2022. The language and targets of online trolling: A psycholinguistic approach for social cybersecurity. Information Processing & Management, 59(5):103012.
  • Weimann (2004) G. Weimann. 2004. Cyberterrorism: How real is the threat?, volume 31. United States Institute of Peace.
  • Wu et al. (2023) F. Wu, X. Liu, and C. Xiao. 2023. Deceptprompt: Exploiting llm-driven code generation via adversarial natural language instructions. arXiv preprint arXiv:2312.04730.
  • Xu et al. (2024) J. Xu, J.W. Stokes, G. McDonald, X. Bai, D. Marshall, S. Wang, A. Swaminathan, and Z. Li. 2024. Autoattacker: A large language model guided system to implement automatic cyber-attacks. arXiv preprint arXiv:2403.01038.
  • Xu and Rajivan (2023) T. Xu and P. Rajivan. 2023. Determining psycholinguistic features of deception in phishing messages. Information & Computer Security, 31(2):199–220.
  • Yamin et al. (2021) M.M. Yamin, M. Ullah, H. Ullah, and B. Katt. 2021. Weaponized ai for cyber attacks. Journal of Information Security and Applications, 57:102722.
  • Yang et al. (2018) G. Yang, L. Cai, A. Yu, J. Ma, D. Meng, and Y. Wu. 2018. Potential malicious insiders detection based on a comprehensive security psychological model. In 2018 IEEE Fourth International Conference on Big Data Computing Service and Applications (BigDataService), pages 9–16. IEEE.
  • Yao et al. (2024) Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211.
  • Zambrano et al. (2023) P. Zambrano, J. Torres, L. Tello-Oquendo, Á. Yánez, and L. Velásquez. 2023. On the modeling of cyber-attacks associated with social engineering: A parental control prototype. Journal of Information Security and Applications, 75:103501.
  • Zhang et al. (2024) Z. Zhang, Y. Zhang, L. Li, H. Gao, L. Wang, H. Lu, F. Zhao, Y. Qiao, and J. Shao. 2024. Psysafe: A comprehensive framework for psychological-based attack, defense, and evaluation of multi-agent system safety. arXiv preprint arXiv:2401.11880.