AI safety
New Benchmark Measures AI Agents' Multi-Step Cyber Attack Abiliti
Researchers develop framework to measure how well AI agents can execute complex, multi-step cyber attacks, revealing critical insights for AI safety and security.
AI safety
AI Agents
Counting AI Agents: New Framework Tackles Liability Questions
New research proposes frameworks for identifying and counting AI agents—a critical question as autonomous systems create content and take actions with real-world consequences.
LLM Research
New Method Quantifies LLM Uncertainty Using Imprecise Probabiliti
Researchers propose a novel approach for expressing higher-order uncertainty in large language models through imprecise probability theory, moving beyond point estimates to interval-based confidence.
LLM alignment
Study Questions Role of Diversity in LLM Moral Alignment
New research examines whether diversity in training data actually improves moral reasoning in LLMs when using RLVR methods, challenging assumptions about alignment approaches.
LLM Research
Researchers Discover Chaotic Dynamics in Multi-LLM Systems
New research reveals that multi-LLM deliberation systems can exhibit chaotic dynamics, raising questions about predictability and reliability in AI systems that use multiple models.
AI Agents
Building Risk-Aware AI Agents: Internal Critics & Uncertainty
Learn to build AI agents that know when they're uncertain. This technical guide covers internal critic mechanisms, self-consistency reasoning, and uncertainty estimation for reliable AI decision-making.
LLM Research
New Method Calibrates LLM Confidence to Catch AI Errors
Researchers introduce technique for aligning LLM confidence with actual correctness, enabling better error detection in AI systems and improving reliability for downstream applications.
LLM Research
AI Steerability 360: New Toolkit for Controlling LLM Behavior
Researchers introduce AI Steerability 360, a comprehensive toolkit enabling multiple techniques for steering large language model outputs with implications for content control and AI safety.
AI safety
New Research Exposes How LLMs Strategically Deceive in Games
Researchers develop parallel-world probing technique to detect when large language models strategically lie during human-AI interactions, revealing concerning deceptive capabilities.
AI safety
LLM Safety Judges Are No Better Than Coin Flips, Study Finds
New research reveals LLM-based safety evaluators fail to reliably measure adversarial robustness, raising critical questions about automated AI safety testing methodologies.
AI safety
Research Reveals AI Monitors Show Leniency Bias Toward Own Output
New research exposes a critical flaw in AI safety systems: models tasked with monitoring AI outputs show systematic bias when evaluating content they generated themselves.
LLM Evaluation
New Method Automatically Discovers How LLM Judges Evaluate AI Con
Researchers introduce an automated framework for discovering the hidden concepts LLM evaluators use when judging AI outputs, enabling better understanding and improvement of AI content assessment systems.