LLM research
New research reveals how benchmark data contamination undermines the reliability of LLM-based recommendation systems, raising critical questions about AI evaluation integrity.
LLM research
Researchers assess how well large language models handle questions about recent events, revealing critical limitations in temporal knowledge that affect AI system reliability.
Agentic AI
A comprehensive guide to evaluating AI agents covering benchmarks, testing frameworks, and metrics for measuring autonomous system performance in real-world applications.
AI Agents
New research proposes a comprehensive framework for empirically evaluating LLM-based agentic AI systems in healthcare, establishing seven key dimensions for systematic assessment.
LLM research
New research introduces DAJ, a data-reweighting approach for LLM judges that improves test-time scaling in code generation by better identifying correct solutions.
LLM research
New research challenges how we measure edit locality in LLM model editing, revealing potential blind spots in current evaluation methods that could impact knowledge modification reliability.
AI Agents
New survey paper comprehensively examines AI agent system architectures, their applications across domains, and frameworks for evaluating autonomous AI behavior and capabilities.
LLM-as-a-Judge
Researchers present a mathematically tractable model for understanding how LLM-as-a-Judge systems scale during inference, offering insights into AI evaluation mechanisms.
AI Agents
New benchmark reveals AI agents fail 63% of complex tasks. Patronus AI's dynamic simulation environments aim to fix reliability crisis plaguing autonomous systems.
AI Evaluation
New research explores using generative AI agents as reliable proxies for human evaluation of AI-generated content, potentially transforming how we assess synthetic media quality at scale.