Detecting and countering malicious uses of Claude

Summary

The report examines how malicious actors used Claude in influence operations, credential scraping, recruitment fraud, and malware development while preserving utility for legitimate users. It details an influence-as-a-service operation in which Claude orchestrated more than 100 social-media bot accounts, selecting likes, shares, comments, or inaction across personas and political objectives. The response used conversation-analysis techniques including Clio and hierarchical summarization, alongside classifiers that assessed user requests and model responses to identify, investigate, and ban accounts. The cases show frontier models accelerating less skilled actors’ capabilities, although successful deployment was not confirmed for the credential, recruitment, or malware efforts, and no content from the influence operation achieved viral status.

Takeaways

1. 1

   Claude orchestrated more than 100 social-media bot accounts, making tactical decisions about liking, sharing, commenting on, or ignoring posts across distinct political personas.

2. 2

   Clio and hierarchical summarization helped analyze large conversation volumes, while classifiers evaluated potentially harmful requests and Claude’s responses before or after delivery.

3. 3

   The report says generative AI can accelerate capability development for less sophisticated malicious actors, but successful real-world deployment was not confirmed for the credential, recruitment, or malware cases.