--- title: "AI Security" description: "50 posts about AI Security, summarised, each linking to the original." --- # AI Security > 50 posts about AI Security, summarised, each linking to the original. ## Articles ### [Expanding Project Glasswing](https://yomu.fyi/post/expanding-project-glasswing.md) - Company: [Anthropic](https://yomu.fyi/company/anthropic.md) - Published: Jun 2, 2026 - Type: Announcement Project Glasswing is expanding its collaboration from roughly 50 initial partners to approximately 150 new organizations across more than 15 countries. These partners must meet security requirements, and many operate critical infrastructure or maintain software used widely, with a major attack estimated to affect more than 100 million people for most. The program follows partners’ use of Claude Mythos Preview to scan codebases, finding more than 10,000 high- or critical-severity flaws, while some now use it to write patches and run pre-release checks. Its broader purpose is to help defenders adapt to powerful cyber models by providing tools and infrastructure, then shifting support toward disclosure, fixing, and deployment of patched software. General access will require safeguards against misuse that have yet to be developed. ### [How to safeguard AI workloads with Unity AI Gateway Guardrails](https://yomu.fyi/post/how-to-safeguard-ai-workloads-with-unity-ai-gateway-guardrails.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: Tim Lortz - Published: May 19, 2026 - Type: Tutorial Unity AI Gateway’s beta LLM Guardrails extend earlier Gateway guardrails with LLM-powered evaluation, pre-built protections, and tunable custom rules for AI security, compliance, and sensitive-data protection. An Acme marketing scenario maps policies to PII redaction on inputs, jailbreak and prompt-injection blocking, unsafe-content blocking on outputs, and a custom rule that blocks competitor references. Teams configure these guardrails on an endpoint, optionally use Log mode, select evaluator endpoints, and test representative requests while tracking activity in inference tables. Built-in controls behave as expected in the examples, while the custom rule initially performs unreliably until a more specific prompt and gpt-5-4-mini evaluator improve triggering without degrading other tests. Inference tables record request outcomes, evaluator calls, latency, statuses, token counts, and shared request\_id values for auditing and refinement. ### [Responsible AI Governance: A Practical Framework for Business Leaders](https://yomu.fyi/post/responsible-ai-governance-a-practical-framework-for-business-leaders.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: Databricks Staff - Published: May 6, 2026 - Type: Explainer Responsible AI governance is presented as an operational framework for leaders overseeing systems that can produce biased outputs, expose sensitive data, and create regulatory, financial, or reputational harm. It draws on the NIST AI RMF and OECD AI principles, maps to EU AI Act requirements, and uses human dignity, fairness, privacy, accountability, transparency, and security as governance values. The program starts with a living inventory recording purpose, ownership, training-data sources, affected populations, review dates, model lineage, and third-party status, followed by risk classification and assessments based on potential impact. It calls for lifecycle controls including bias mitigation, security testing, human review, drift monitoring, audits, incident exercises, and confidential concern reporting. The roadmap recommends piloting governance on a highest-risk product line, scaling controls across business units, and reviewing the framework annually or after major incidents, regulatory updates, or portfolio changes. ### [A Modern AI Risk Management Framework](https://yomu.fyi/post/a-modern-ai-risk-management-framework.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: Databricks Staff - Published: Apr 22, 2026 - Type: Explainer AI systems introduce probabilistic behavior, model drift, adversarial manipulation, bias, and auditability challenges that traditional IT risk practices may not address. The framework described combines NIST AI RMF, EU AI Act, and ISO/IEC 23894:2023, using Govern, Map, Measure, and Manage as its operational structure while recognizing regulatory and societal context. It recommends cross-functional governance, clear risk ownership, an AI-BOM covering systems, data flows, dependencies, and accountability, plus continuous measurement of fairness, explainability, data quality, security vulnerabilities, and harm likelihood and severity. Across the AI lifecycle, the text identifies risks including data poisoning, model drift, malicious library injection, prompt injection, hallucinations, and platform vulnerabilities, and points to the AI RMF Playbook for checklists, timelines, and governance artifacts. ### [Dario Amodei on the Department of War discussions](https://yomu.fyi/post/dario-amodei-on-the-department-of-war-discussions.md) - Company: [Anthropic](https://yomu.fyi/company/anthropic.md) - Published: Feb 26, 2026 - Type: Announcement Dario Amodei says Anthropic considers AI deployment for US and allied democratic defense existentially important and has deployed Claude across the Department of War and other national security agencies, including classified networks and National Laboratories. The company says it supports intelligence analysis, modeling and simulation, operational planning, and cyber operations, while restricting Claude’s use by firms linked to the Chinese Communist Party and advocating chip export controls. Anthropic maintains two safeguards: mass domestic surveillance is incompatible with democratic values, and current frontier systems are not reliable enough for fully autonomous weapons without proper oversight. The Department has stated it demands “any lawful use” and threatens offboarding, a “supply chain risk” designation, and Defense Production Act action; Anthropic says it will not remove the safeguards, while offering continued service or transition support. ### [Detecting and preventing distillation attacks](https://yomu.fyi/post/detecting-and-preventing-distillation-attacks.md) - Company: [Anthropic](https://yomu.fyi/company/anthropic.md) - Published: Feb 23, 2026 - Type: Explainer Anthropic reports identifying industrial-scale distillation campaigns by DeepSeek, Moonshot, and MiniMax that generated more than 16 million Claude exchanges through approximately 24,000 fraudulent accounts. The campaigns used proxy services and coordinated account networks to extract agentic reasoning, tool use, coding, computer-use, and other capabilities, including chain-of-thought data and reinforcement-learning tasks. Anthropic attributed the operations with high confidence using IP correlation, request metadata, infrastructure indicators, and partner corroboration, while describing hydra cluster architectures that replace banned accounts and obscure patterns. The response combines classifiers, behavioral fingerprinting, coordinated-activity detection, intelligence sharing, stronger account verification, and product, API, and model-level countermeasures. The report argues illicitly distilled models may strip safeguards and undermine export controls, and says coordinated action across AI labs, cloud providers, and policymakers is required. ### [Strengthening safeguards with US CAISI and UK AISI](https://yomu.fyi/post/strengthening-safeguards-with-us-caisi-and-uk-aisi.md) - Company: [Anthropic](https://yomu.fyi/company/anthropic.md) - Published: Sep 12, 2025 - Type: Explainer Anthropic describes an ongoing partnership with the US Center for AI Standards and Innovation (CAISI) and UK AI Security Institute (AISI) to test and improve AI safeguards during model development. The organizations evaluated multiple iterations of Constitutional Classifiers on models including Claude Opus 4 and 4.1, using pre-deployment prototypes, varied system configurations, documentation, and classifier scores for red-teaming. Their testing exposed prompt-injection bypasses, universal jailbreaks, cipher-based and input/output obfuscation attacks, and automated attack refinement that produced an effective universal jailbreak. Anthropic patched specific vulnerabilities and restructured safeguard architecture, improved detection and filtering, and used findings to strengthen evaluation, deployment monitoring, and response practices. The collaboration also showed that sustained access, real-time data, daily technical communication, and complementary government, expert, and bug-bounty testing can reveal both common exploits and subtle attack vectors. ### [Developing nuclear safeguards for AI through public-private partnership](https://yomu.fyi/post/developing-nuclear-safeguards-for-ai-through-public-private-partnershi.md) - Company: [Anthropic](https://yomu.fyi/company/anthropic.md) - Published: Aug 21, 2025 - Type: Problem & solution AI models are increasingly capable, raising concern that they could provide dangerous nuclear-technical knowledge that threatens national security. Nuclear information is especially difficult for a private company to evaluate alone, motivating a partnership with the U.S. Department of Energy’s National Nuclear Security Administration (NNSA). Together with NNSA and DOE national laboratories, the company co-developed a classifier that distinguishes concerning from benign nuclear-related conversations, achieving 96% accuracy in preliminary testing. The classifier has been deployed on Claude traffic within a broader system for identifying model misuse, and early deployment data suggests it works well on real conversations; the approach will be shared with the Frontier Model Forum as a potential blueprint for similar safeguards developed with NNSA. ### [Find the intersection of security, AI, IAM, and fun at Oktane](https://yomu.fyi/post/find-the-intersection-of-security-ai-iam-and-fun-at-oktane.md) - Company: [Okta](https://yomu.fyi/company/okta.md) - Author: Michael Palermo - Published: Aug 13, 2025 - Type: Announcement Oktane 2025 will focus on security, AI, and identity, with programming for protecting software systems that use both human and non-human identities. The event is scheduled for September 24–26, 2025, at Caesars Forum in Las Vegas, and will include the Oktane Developer Lounge, Expo Hall activities, lightning talks, documentation feedback games, and Developer Support consultations. Hands-on labs cover securing enterprise AI with the OAuth Cross App Access (XAA) extension and Model Context Protocol (MCP), automating Okta with Terraform, replacing standing administrative access with time-bound requests through Okta Identity Governance and Workflows, and using OAuth 2.0 Demonstrating Proof of Possession (DPoP) tokens in a single-page application. An Okta Workflows community meetup will add networking, drinks, and appetizers, while the broader event offers learning, expert conversations, and application-focused discussions. ### [Charting a path to AI accountability](https://yomu.fyi/post/charting-a-path-to-ai-accountability.md) - Company: [Anthropic](https://yomu.fyi/company/anthropic.md) - Published: Jun 13, 2023 - Type: Explainer Anthropic’s response to the NTIA’s Request for Comment on AI Accountability proposes processes and infrastructure for evaluating highly capable and general-purpose AI models. It recommends funding rigorous evaluations, requiring near-term disclosure of evaluation methods and results, and developing longer-term industry standards with agencies such as NIST. The framework would use capability-based risk thresholds, pre-registration of large training runs, confidential registries, technically literate and security-conscious third-party auditors, and external red teaming before advanced model release. It also calls for more interpretability research, acknowledges that regulations requiring interpretable models are currently infeasible, and seeks antitrust guidance for industry safety collaboration. Anthropic says these measures require collaboration among researchers, AI labs, regulators, auditors, and other stakeholders to build accountability and auditing mechanisms that support safe development and deployment. [Newer posts](https://yomu.fyi/topic/ai-security/page/2.md)