Research

GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations

NeurIPS 2026 workshop

NeurIPS 2026 Workshop

Nicholas Saban, Charvi Bannur, Mariya I. Vasileva, Darshan Deshpande

October 15, 2026

Workshop

globebench

https://arxiv.org/abs/2610.22708

NeurIPS 2026 Workshop

“Byrd Glacier”, Transantarctic Mountains, Antarctica

SpeedrunBench: Challenging LLM Agents with Video Game Speedrunning

arXiv 2610.08076

Yoshinari Fujinuma, Keisuke Kamahori, Ryuto Koike, Abdelrahman Madkour, Varun Prashant Gangal, Monty Bichouna, Martyna Markiewicz, Shivani Jain, Duncan Curtis, Rebecca Qian, Anand Kannappan

October 6, 2026

speedrunbench

https://arxiv.org/abs/2610.08076

“Melted Colors”, Great Sandy Desert, Western Australia

FigmaTrace: Capturing Creative Nuances in Human Figma Design Workflows

arXiv 2608.21460

Darshan Deshpande, Yoshinari Fujinuma, Martyna Markiewicz, Devanshu Bansal, Shivani Jain, Nicholas Saban, Chirag Maheshwari, Anand Kannappan

August 20, 2026

figmatrace

https://arxiv.org/abs/2608.21460

“Mezen Mixing”, Mezen River, northern Russia

Defenses & Enablers For Skill Injection Attacks on Terminal Based Agents

arXiv 2606.01567

Yoshinari Fujinuma, Varun Gangal, Traian Rebedea, Makesh Narsimhan Sreedhar, Prasoon Varshney, Rebecca Qian, Anand Kannappan

June 1, 2026

skill-injection-attacks

https://arxiv.org/abs/2606.01567

“Serene Expressions”, off the Louisiana coast

HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models

NeurIPS 2026

Emmy Liu, Varun Gangal, Michael Yu, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng

May 19, 2026

halluworld

https://arxiv.org/abs/2605.19341

NeurIPS 2026

Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

NeurIPS 2026

Darshan Deshpande

May 7, 2026

world-modeling-for-agentic-rl

https://arxiv.org/abs/2607.16204

NeurIPS 2026

“Luminescence”, Suwannee River plume, Gulf of Mexico

Unlocking Prompt Infilling Capability for Diffusion Language Models

COLM 2026 workshop

COLM 2026 Workshop

Yoshinari Fujinuma, Keisuke Sakaguchi

April 4, 2026

Workshop

prompt-infilling-for-diffusion-lms

https://arxiv.org/abs/2604.03677

COLM 2026 Workshop

Adaptive Environments For LLM Agents Need to Jointly Scale Task, Scaffolding and Reward Design

Preprint

Darshan Deshpande, Varun Gangal, Li Siyan, Anand Kannappan, Rebecca Qian

February 18, 2026

generative-simulators

DETOUR: An Interactive Benchmark for Dual-Agent Search and Reasoning

ACL 2026 workshop

GEM Workshop at ACL 2026

Li Siyan, Darshan Deshpande, Anand Kannappan, Rebecca Qian

January 30, 2026

Workshop

detour

https://arxiv.org/abs/2602.00352

GEM Workshop at ACL 2026

“Alluvial Fan”, Taklimakan Desert, China

Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysis

ICML 2026

Darshan Deshpande, Anand Kannappan, Rebecca Qian

January 27, 2026

trace

https://arxiv.org/abs/2601.20103

ICML 2026

Position: We Need A Unified Definition of Hallucination (It’s The World Model, Stupid!)

ICML 2026

Emmy Liu, Varun Gangal, Chelsea Zou, Michael Yu, Xiaoqi Huang, Alex Chang, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng

December 25, 2025

unified-hallucination-framework

https://arxiv.org/abs/2512.21577

ICML 2026

“Copper and Blue”, Northwest Territories, Canada

Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge

Findings of ACL 2026

Yoshinari Fujinuma

October 21, 2025

contrastive-decoding-for-llm-judges

https://arxiv.org/abs/2510.18196

Findings of ACL 2026

MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments

NeurIPS 2025 workshop

NeurIPS 2025 Workshop

Darshan Deshpande, Varun Gangal, Hersh Mehta, Anand Kannappan, Rebecca Qian, Peng Wang

October 1, 2025

Workshop

memtrack

https://arxiv.org/abs/2510.01353

NeurIPS 2025 Workshop

“Capillaries”, Meta River, Colombia–Venezuela border

TRAIL: Trace Reasoning and Agentic Issue Localization

arXiv 2505.08638

Darshan Deshpande, Varun Gangal, Hersh Mehta, Jitin Krishnan, Anand Kannappan, Rebecca Qian

May 13, 2025

trail

https://arxiv.org/abs/2505.08638

“Icy Vortex”, Foxe Basin, northern Canada

Browsing Lost Unformed Recollections: A Benchmark for Tip-of-the-Tongue Search and Reasoning

ACL 2025

Sky CH-Wang, Darshan Deshpande, Smaranda Muresan, Anand Kannappan, Rebecca Qian

March 24, 2025

blur

https://arxiv.org/abs/2503.19193

ACL 2025

“Himalayas”, Eastern Himalaya, southwestern China

Humanity’s Last Exam

Nature 2026

Nature

Center for AI Safety, Scale AI and HLE Contributors Consortium

January 24, 2025

humanitys-last-exam

https://arxiv.org/abs/2501.14249

Nature

GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking

arXiv 2412.14140

Darshan Deshpande, Selvan Sunitha Ravi, Sky CH-Wang, Bartosz Mielczarek, Anand Kannappan, Rebecca Qian

December 18, 2024

glider-sota-slm-judge

https://arxiv.org/abs/2412.14140

Lynx: An Open Source Hallucination Evaluation Model

arXiv 2407.08488

Selvan Sunitha Ravi, Bartosz Mielczarek, Anand Kannappan, Douwe Kiela, Rebecca Qian

July 11, 2024

lynx-hallucination-detection-model

https://arxiv.org/abs/2407.08488

FinanceBench: A New Benchmark for Financial Question Answering

arXiv 2311.11944

Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, Bertie Vidgen

November 20, 2023

financebench

https://arxiv.org/abs/2311.11944

SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models

arXiv 2311.08370

Bertie Vidgen, Nino Scherrer, Hannah Rose Kirk, Rebecca Qian, Anand Kannappan, Scott A. Hale, Paul Röttger

November 14, 2023

simplesafetytests

https://arxiv.org/abs/2311.08370

No items found.