Generative AI Captures Threat Actor Attention

Generative AI applications are redefining how we engage with each other and our environments. Whereas, the threat actors are abusing these same generative AI applications that are creating new positive possibilities to manipulate them for malicious purposes.

Today, it become a boardroom discussion, discussing a lot of worst-case scenarios, which is important to talk about, but we must also look at what's actually happening.

In contrast, information operations actors of diverse motivations and capabilities have increasingly leveraged AI-generated content, particularly imagery and video, in their campaigns, likely due at least in part to the readily apparent applications of such fabrications in disinformation.

Additionally, the release of multiple generative AI tools in the last year has led to a renewed interest in the impact of these capabilities. We anticipate that generative AI tools will accelerate threat actor incorporation of AI into information operations and intrusion activity.

AI has been around for a while, but this is the inflection point where the general public has taken notice. Like any technological innovation, we expect adversaries are going to find applications for these tools. However, there is far greater promise for defenders who have the ability to direct the development of it. We still own the technology. There are going to be people who will use AI for ill intent, but that shouldn't stop us from leapfrogging ahead to out innovate the adversaries.

As per Mandiant ,it anticipates that information operations actors' adoption of generative AI will vary by media form – text, images, audio, and video – due to factors including the availability and capabilities of publicly available tools and the effectiveness of each media form to invoke an emotional response (Figure 1). We believe that AI-generated images and videos are most likely to be employed in the near term; and while we have not yet observed operations using LLMs, we anticipate that their potential applications could lead to their rapid adoption.

Expert predicts that, while we expect the adversary to make use of generative AI, and there are already adversaries doing so, adoption is still limited and primarily focused on social engineering. There’s no doubt that criminals and state actors will find value in this technology, but many estimates of how this tool will be used are speculative and not grounded in observation.

Going forward, malicious actors will continue to capitalize on the general public’s inability to differentiate between what is authentic and what is counterfeit. There is need to conduct end-to-end risk assessments related to how organizations will deploy AI can help inform decisions.

Dr. Deepak Kumar Sahu, President & CEO, VARINDIA