Democratizing AI Safety with RiskRubric.ai

Summary

Cloud Security Alliance and Noma Security introduced RiskRubric.ai to provide standardized, transparent risk assessments across the open AI model ecosystem. The framework evaluates AI models across six pillars—transparency, reliability, security, privacy, safety, and reputation—using over 1,000 reliability tests, 200 adversarial security probes, automated code scanning, and harmful content evaluations. Each model receives 0–100 scores and A–F letter grades, supplemented by specific vulnerability findings and recommended mitigation strategies to assist deployment filtering. Initial benchmark results across models showed composite scores ranging from 47 to 94 with a median of 81, revealing polarized safety distributions and indicating that security hardening directly correlates with reduced safety risks.

Takeaways

1. 1

   Model risk scores ranged from 47 to 94 with a median of 81, showing polarization where 54 percent reached A or B levels while a long tail clustered in the medium-to-low protection C and D range.

2. 2

   Safety pillar scores exhibited the widest variation across evaluated models but tracked closely with security posture, indicating that prompt injection defenses and policy enforcement directly mitigate harmful outputs.

3. 3

   Stricter guardrails frequently decrease user-perceived transparency through opaque refusals, which can be mitigated by combining safeguards with explanatory refusals and provenance signals.