New Use Cases, Key Players, and the Technologies

Artificial intelligence has become the central organizing challenge for the security industry. The old security playbook, with firewalls at the perimeter, antivirus on the endpoint, and periodic access reviews, was never built for systems that read untrusted data and act on their own.

Cybersecurity in the age of AI is evolving in two opposing ways. Malicious actors are weaponizing AI through automated phishing, deepfake scams, offensive ML tools that craft malware or crack passwords, and more. This has led to an 89% increase in AI-enabled attacks in 2025, according to the CrowdStrike 2026 Global Threat Report.

At the same time, defenders are deploying AI-driven tools (anomaly detection, intelligent SIEM/XDR, adaptive authentication) to scale threat hunting and response. This dual-use dynamic demands new security approaches, including embedding AI-safe design and governance across systems, adopting MLSecOps for ML pipelines, and strengthening fundamentals such as zero-trust identity.

Simply put, security in the age of AI is two stories unfolding at once. AI as a new thing that must be defended, and AI as a new weapon that changes how everything else is defended. This article walks through both, along with the platforms, technologies, and companies shaping the space right now.

AI-Driven Threats (Offensive Use Cases)

Modern attackers are adopting AI as a force multiplier. CrowdStrike found that over 90 organizations had legitimate AI tools exploited to generate malicious commands and extract sensitive data.

AI-Powered Social Engineering

The first thing cybercriminals started doing and still do is using generative AI to craft personalized phishing and BEC campaigns. This is happening by jailbreaking mainstream tools like GPT or Gemini, but there are now also purpose-built models (WormGPT and FraudGPT) that can be sold to churn out phishing emails and malicious code.

The study found that control-group phishing emails had a 12% click-through rate, while fully AI-generated phishing emails achieved 54%. A human-in-the-loop AI approach reached 56%.

Voice cloning is the other half of this. The 2024 deepfake CFO scam CNN reported on, in which a cloned executive’s voice convinced an employee to wire $25 million, wasn’t a one-off. It has become a template. Now that there are enough scraps of public data (social media, even someone’s Spotify habits), attackers can predict who on a target list is likely to fall for a given pitch.

Intelligent Reconnaissance and Profiling

AI is doing the homework before an attack even starts, improving target discovery. Attackers use ML to mine open-source intelligence (OSINT), which aggregates social data and profiles users from innocuous inputs (such as Spotify or game profiles).

Acoustic side-channel attacks can infer keystrokes from VoIP calls using ML. Even model-driven tools (Maltego with AI plugins) automate sentiment analysis and persona-building. The result is a more nuanced attack surface, where models can predict who is vulnerable to phishing or expose personal traits that aid social engineering.

Automated Malware Creation and Evasion

AI is being used to make code harder to catch. GAN-based tools like MalGAN produce binaries designed to slip past antivirus software. A proof-of-concept called BlackMamba takes it further, using an LLM at runtime to rewrite its own malicious code on the fly, so it never has quite the same signature twice.

The same logic applies to infrastructure. DeepDGA generates constantly shifting command-and-control domains that are hard to blocklist. On the offensive-testing side, tools like DeepExploit can automatically scan for and exploit vulnerabilities, which is handy for red teams and attackers, since the line between the two has largely blurred.

Password cracking has leveled up, too. PassGAN and other generative models learn real-world password patterns and are better at guessing than older brute-force approaches.

AI-Assisted Lateral Movement and Exfiltration

AI is also helping plan the next moves when attackers are inside the network. ChainReactor uses AI planning to map multi-step privilege-escalation paths through a network.

At the data-exfiltration stage, AI can disguise that traffic as ordinary, so it doesn’t trigger the alarms meant to catch large transfers.

Simply put, offensive AI spans the entire kill chain, including reconnaissance, weaponization, delivery, exploitation, installation, and action on objectives, with ML techniques enhancing each stage.

Adversarial AI (Attacking Models)

There is a separate category of attacks aimed at the models themselves. This includes poisoning training data or extracting sensitive information a model memorized during training. It also involves crafting adversarial inputs that cause ML classifiers (such as image or malware detectors) to make errors.

For example, accessible target models (such as open-source detectors) can be probed to build proxy models, which in turn guide the creation of adversarial samples. These “adversarial attacks” are often difficult to detect.

An interesting fact is that the same techniques used to break a model are also used to test and harden it, which is a major reason red-teaming has become such a core part of AI security work.

AI-Enhanced Security (Defensive Use Cases)

Defenders aren’t sitting still either. AI and machine learning are now baked into most modern security stacks.

Anomaly Detection and SIEM/XDR

Modern SIEM and XDR platforms use ML to sift through logs and network data at scale. For example, Microsoft Sentinel and Security Copilot use AI agents to process billions of signals and surface anomalies in real time.

CrowdStrike’s Falcon platform employs threat-graph ML models to detect unusual endpoint behaviors.

Darktrace built its reputation on self-learning AI that models “normal” network and user behavior and flags deviations.

Similarly, Vectra’s “Attack Signal Intelligence” uses ML to highlight in-progress attacks in network/identity traffic. These AI engines enable defenders to spot stealthy breaches (file-less malware, insider misuse) that signature-based systems miss.

Automated Incident Response

AI assistants like Charlotte in CrowdStrike or Microsoft Security Copilot can summarize incidents and suggest triage steps. They can also automatically take remediation actions under guidance. This “agentic SOC” approach chains multiple actions without human intervention to accelerate response.

Similarly, AI-driven playbooks in SOAR platforms execute routine tasks (such as isolating endpoints and blocking malicious IPs) faster than manual response. This frees up limited security staff to focus on complex decisions.

Smarter Identity Checks

AI has also found its way into identity risk analysis. Unusual login patterns, an unfamiliar device, or odd geolocation can now trigger adaptive authentication. For example, Thales and Microsoft emphasize contextual MFA and continuous adaptive controls, with AI continuously evaluating login risk (biometrics, geolocation, device fingerprinting) to flag anomalies.

It matters because Microsoft reports that over 97% of identity attacks are still brute-force, which phishing-resistant MFA already blocks effectively. AI’s contribution here is mostly making detection faster and more precise.

Vulnerability and Code Analysis

AI accelerates vulnerability management. For example, GitHub CodeQL (powered by ML) automatically detects code flaws. Similarly, Amazon’s CodeWhisperer and GitHub Copilot (LLM code assistants) can suggest secure coding fixes or detect vulnerabilities in code.

On the network side, AI models analyze traffic to detect misconfigurations or zero-day vulnerabilities by identifying anomalous protocol patterns. Some teams use AI for “red teaming” their own infrastructure. For example, IBM’s DeepExploit (an ML pentesting tool) automates vulnerability discovery, while others augment fuzzing with generative models. The OpenSSF’s MLSecOps framework (and Cisco/Ericsson guides) emphasize integrating these AI-based tests into development pipelines.

Adversarial Training and Robust Models

Defenders are also working to harden the models themselves. Adversarial training (augmenting training sets with worst-case perturbations) and certified defenses aim to make models robust.

Open-source projects like IBM’s Adversarial Robustness Toolbox (ART) provide tools for researchers and developers to evaluate and harden models against evasion, poisoning, and model-extraction attacks. For example, security teams can use ART to generate adversarial examples and retrain their models to withstand them.

Data privacy techniques (differential privacy, homomorphic encryption) protect sensitive training data from extraction. This includes scanning ML code for known vulnerabilities (such as insecure TensorFlow/Torch versions) and applying DevSecOps practices during model training.

The Bigger Picture

AI is turning rule-based defenses into something dynamic and far more automated. Some in the industry call it autonomous cyberdefense because it uses ML to anticipate threats and automate responses. It’s showing up everywhere, whether it’s Microsoft Defender’s ML classifiers, AWS GuardDuty’s anomaly detection, or Palo Alto’s Cortex XSIAM.

The important point is that these systems need monitoring, too. An AI model that’s misconfigured can become a liability, which is really the whole argument for pairing “AI for good” with a real commitment to securing the AI itself.

Technologies & Platforms Driving AI Security

It’s easy to talk about “AI in security” as a single abstract trend, but it is a stack of specific technologies, frameworks, cloud platforms, and open-source tools that make it possible.

ML Frameworks & Libraries

TensorFlow, PyTorch, Scikit-Learn, JAX, and similar frameworks are the workhorses behind most security tooling. They power anomaly detectors, NLP classifiers (for email scanning), and image-based sensors (for biometrics).

Note that these frameworks aren’t immune to vulnerabilities (both TensorFlow and PyTorch have had CVEs over the years), so they need the same patching and sandboxing discipline as any other dependency.

Many vendors wrap these frameworks into their products. For example, CrowdStrike uses PyTorch for on-device inference, and Darktrace’s Immune System is based on proprietary ML. This is a reminder that most household-name platforms are built on the same open frameworks that anyone else can use. They are simply tuned and hardened for their specific product.

MLOps / MLSecOps

Just as DevOps transformed software delivery, MLOps automates the ML lifecycle (data collection, training, and deployment) with platforms such as Kubeflow, MLflow, AWS SageMaker, and Azure ML.

The security-focused version of this, often called MLSecOps, integrates security checks directly into the ML pipeline. This means security checks are embedded at every stage of the ML pipeline. Teams can scan training data for poisoning, check models and dependencies for vulnerabilities, control access to sensitive data, and monitor deployed models for attacks or unusual behavior.

Model Hubs and APIs

Public model stores (Hugging Face, TensorFlow Hub) and LLM APIs (OpenAI, Azure OpenAI, Google Vertex AI) have made building with AI much easier. However, each of those API endpoints is also a new attack surface, vulnerable to model extraction or prompt injection attacks.

Enterprises must secure these channels through enforced authentication and content filtering of model inputs and outputs. Several vendors now offer “LLM security” features (such as watermarking or content classifiers to detect deepfake generation) to mitigate misuse.

Cloud and Platform Services

Major cloud providers integrate AI into their security suites.

• AWS offers GuardDuty (ML-based threat detection), Macie (PII discovery), and CodeGuru (ML code review), among others.
• Azure includes Sentinel (SIEM with AI analytics), Defender (endpoint AI), and Entra ID Protection (identity risk ML).
• Google Cloud provides Chronicle (security analytics) and AI Threat Detection APIs. It also integrates Vertex AI for detection.

These platforms also provide secure ML building blocks (TPUs, ML System Security guidelines). Hybrid SaaS platforms (Microsoft 365 Copilot, Google Workspace AI) are integrating security features into productivity tools to simplify IT and AI governance.

Open-Source Security Tools

The security community has developed ML-aware open-source tools. For example, IBM’s Adversarial Robustness Toolbox (ART) helps teams test and harden models against evasion and poisoning attacks. MITRE’s ATLAS knowledge base catalogs real-world ML attack techniques, similar in spirit to MITRE ATT&CK for traditional security.

Zeek, Suricate, and other established network tools now include ML extensions for anomaly detection. Similarly, container security tools (Falco and others) use rule-based ML for runtime threat alerts.

Efforts such as the OWASP Machine Learning Security Top 10 and the Linux Foundation’s AI & Data initiatives aim to turn all of this into shared best practices, so smaller teams don’t have to build them from scratch.

Specialized AI Security Platforms:

Several new products address AI-specific challenges. For example, Hugging Face launched Model Guardrails to filter malicious inputs, and companies such as DataRobot offer MLOps with built-in compliance checks.

Immuta and BigID apply ML to data governance, automatically classifying sensitive data for privacy and regulatory purposes. In addition, synthetic-data startups like Hazy and MOSTLY AI use generative models to produce realistic training data that doesn’t expose real personal information, which is useful for training security models without handling live PII directly.

Security Tools Enhanced with AI

Many traditional security tools now advertise “AI-driven” features:

• Email gateways use NLP to flag phishing content
• Cloud workload protection platforms (CWPP/CNAPP) leverage ML to prioritize risks
• UEBA/Insider Threat products use ML to score user anomalies.

Endpoint and network vendors (Palo Alto, Fortinet, Zscaler, Check Point, etc.) have all embedded AI to enable faster detection. Even vulnerability scanners (Qualys, Tenable) are experimenting with ML to predict which CVEs will be exploited.

Dominating Players in AI-Security Market

The AI-security market comprises established vendors that bolt AI onto existing platforms and newer players built around AI from day one. Let’s take a quick look at some of the key players in the AI-security market.

Endpoint and XDR

CrowdStrike, SentinelOne, Microsoft, Palo Alto Networks, VMware Carbon Black, and Cisco lead this space. They all use ML analytics for detection, automated response, threat hunting, etc.

This is probably the most mature corner of AI security, as these platforms have been layering ML into endpoint telemetry for years now.

Network and Cloud Security

Palo Alto (through Prisma Cloud and Cortex), Fortinet, Check Point’s CloudGuard, and Zscaler all embed ML in firewalls and secure web gateways.

On the cloud-native side, Wiz and Orca Security have built their entire pitch around ML correlating risk across cloud environments. Wiz in particular has grown into one of the category’s most talked-about names and managed to close a high-profile acquisition by Google in 2025.

Email and Identity

Proofpoint and Mimecast use AI to filter phishing and malware from email.

On the identity side, Okta, Ping Identity, and Microsoft Entra all use ML-based risk engines for adaptive MFA and fraud detection.

Incident Response and SOAR

Splunk, IBM Security (QRadar and Resilient), ServiceNow SecurityOps, and LogRhythm have all integrated AI into triage and case management.

Rapid7 and Exabeam lean specifically on user behavior analytics (UBA) to spot anomalies that rule-based systems tend to miss.

Threat Intelligence

Recorded Future uses NLP to process and prioritize threat intelligence. A smaller set of specialized services also monitor social media and open channels for emerging threats or coordinated disinformation campaigns.

AI-Specific Security and Governance

A few vendors focus on securing AI and ML assets. Oracle’s AI Governance Tools, IBM’s Trustworthy AI suite, and startups (OneTrust for AI risk, Mona.ai for model security) help monitor model drift, bias, and compliance.

IBM’s Adversarial Robustness Toolbox (ART), now maintained under the Linux Foundation’s AI umbrella, remains one of the more widely referenced open-source projects for adversarial ML defense research.

Regulations/Standards and Best Practices Related to AI Security

Legislators and standards bodies worldwide are codifying AI security requirements. The current regulatory trend is to ensure that AI is transparent, auditable, and resilient.

The EU AI Act

The EU AI Act will enter into force in August 2026. This regulation classifies many cybersecurity-relevant AI systems as “high-risk” (such as biometric identification and critical infrastructure control).

Article 15 mandates that such AI be designed to meet an “appropriate level of accuracy, robustness and cybersecurity” throughout its lifecycle. In other words, models must be resilient to attacks and operate reliably under adverse conditions.

Providers must document performance metrics and maintain logs, and regulators can require post-market monitoring. Non-compliance carries heavy fines. This means any organization deploying high-stakes AI (such as AI-based threat detection and identity verification) must implement secure development practices (code review, pen-testing), data governance, and incident-response plans tailored to AI.

OECD AI Principles

The OECD (adopted by G20, 42+ countries) states that “AI systems should be robust, secure and safe throughout their entire lifecycle.” This principle calls for safeguards against foreseeable misuse or attacks on AI.

It also recommends impact assessment and human oversight. While non-binding, these principles influence national policies (Japan and Canada have AI guidelines echoing them).

NIST and International Standards

In the US, NIST’s AI Risk Management Framework and forthcoming guidelines emphasize AI resiliency. NIST SP800-series may extend to “Cybersecurity for AI” (similar to zero-trust frameworks).

On the international side, ISO/IEC 42001 now covers AI management systems including security and risk clauses, and ISO/IEC JTC1 SC42 handles broader AI standardization. Neither is mandatory the way the EU Act is, but both are what auditors expect to see referenced.

Data and Privacy Laws

GDPR and CCPA still apply to AI. This means training on personal data demands consent/legitimate interest and data minimization. Financial regulators in the US/EU are requiring AI risk management, such as CECL models under Fed guidance.

Cybersecurity Regulations

Existing cybersecurity laws are also evolving. The EU Cybersecurity Act, the UK’s CSR bill, and the EU CRA (Cyber Resilience Act) enforce security best practices on software, including AI components.

Organisations covered by these laws will need to ensure their AI tools meet software security standards (patching, vulnerability disclosure, secure defaults). Similarly, US agencies (OMB, CISA) are issuing AI security advisories.

Future Trends and Emerging Risks

After all the discussion, the next question is where the future trends and emerging risks lie for the security world in the AI era. Here’s what that seems to be shaping up:

Autonomous AI Agents on Both Sides

Attackers are heading toward running full campaigns on autopilot. Find a weakness, break in, move around, and cover their tracks, all without scripting each step.

Defenders are building the same kind of autonomous agents to fight back. The “AI vs. AI” arms race people talk about is starting to look like it.

Adversarial Arms Races

Every time defenders harden a model, attackers find smarter workarounds, such as sneakier data poisoning or prompt injection, and even attempt to slip backdoors into model weights shared on open hubs.

That’s pushing certification and benchmarking to the forefront of the conversation, because it’s no longer enough for a vendor to claim a model is robust. They need a way to prove it.

Supply Chain and Open-Source Risk

Hardly anyone trains a model from scratch anymore. Most teams build on models like Llama or Stable Diffusion. That’s efficient, but it also means a single bad weight or a hidden flaw in a widely used model can ripple across an entire industry at once.

This is why tools like SLSA, Sigstore, and AI-specific SBOMs are gaining traction. They provide a paper trail for where a model came from and what’s inside it.

Multimodal Attack Surface

As AI systems process audio, video, text, and even code together, attack surfaces widen. Deepfake audio and video will continue to improve (making authentication harder), while language models may be tricked into generating harmful outputs via clever prompts (known as “jailbreaks” or “prompt injections”). Defenders will need multimodal detection (AI that can spot manipulated images, audio, and text).

Human-AI Interaction Risks

The last mile of security is human trust. AI tools can also amplify human error. For example, reliance on LLM responses could lead to unchecked misconfigurations. Phishing attacks may exploit public AI content (malware disguised as AI tutorials). Building awareness will be as critical as the technology.

Regulatory Catch-Up and Ethics

Rules keep being written in response to what’s already happened. Expect more pressure toward explainability and ethical design going forward, especially in fields like healthcare, where an AI’s decision needs to withstand scrutiny after the fact.

Better Defenses Are Coming Too

It’s not all bad news. Privacy-preserving techniques such as federated learning and encrypted inference are moving from research labs into everyday use. And groups like IEEE and ISO are expected to continue rolling out AI-specific security standards to give the industry a more shared playbook than it has right now.

Wrapping Up: Recommendations for Security Teams

This is a lot of ground to cover, but the takeaway is simple. Know what AI is actually running in your environment, treat it like any other system that needs hardening and monitoring, and don’t wait for a regulation deadline to start documenting how it’s controlled. Here’s where to start.

• Make AI a security priority and invest real resources in AI-powered detection tools. Let AI handle the repetitive work (alert triage, malware analysis) since most teams are already short-staffed.
• Build security from the start. Check your training data, stress-test your models, sign them before deployment, and maintain tight control over who can access or modify them.
• Get the basics of Zero Trust right. Identity remains the area where most attacks succeed. Phishing-resistant MFA and least-privilege access go a long way on their own.
• Stay on top of what’s actually happening out there. Keep tabs on AI-specific threats and consider using tools like honeypots that can flag an attacker’s interest before real damage occurs.
• Don’t overlook your people. Teach them to spot deepfakes and question content that looks AI-generated. And get your security and data science teams to actually talk to each other.
• Put someone in charge. AI governance works better with a clear owner (ideally your CISO and Chief Data Officer) conducting regular risk checks.
• Security in the age of AI isn’t a destination you arrive at once and check off. It’s a posture you keep adjusting as the ground shifts under it.