No items found.

Research

Featured
“Ganges River Delta”, Bay of Bengal“Ganges River Delta”, Bay of Bengal
NeurIPS 2026 Workshop
meta

GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations

October 15, 2026
NeurIPS 2026 workshop
Workshop
globebench
https://arxiv.org/abs/2610.22708
NeurIPS 2026 Workshop
“Byrd Glacier”, Transantarctic Mountains, Antarctica“Byrd Glacier”, Transantarctic Mountains, Antarctica
meta

SpeedrunBench: Challenging LLM Agents with Video Game Speedrunning

October 6, 2026
arXiv 2610.08076
speedrunbench
https://arxiv.org/abs/2610.08076
“Melted Colors”, Great Sandy Desert, Western Australia“Melted Colors”, Great Sandy Desert, Western Australia
meta

FigmaTrace: Capturing Creative Nuances in Human Figma Design Workflows

August 20, 2026
arXiv 2608.21460
figmatrace
https://arxiv.org/abs/2608.21460
All papers
“Ganges River Delta”, Bay of Bengal“Ganges River Delta”, Bay of Bengal

GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations

Nicholas Saban, Charvi Bannur, Mariya Vasileva, Darshan Deshpande
NeurIPS 2026 workshop
NeurIPS 2026 Workshop
October 15, 2026
Workshop
globebench
https://arxiv.org/abs/2610.22708
NeurIPS 2026 Workshop
“Byrd Glacier”, Transantarctic Mountains, Antarctica“Byrd Glacier”, Transantarctic Mountains, Antarctica

SpeedrunBench: Challenging LLM Agents with Video Game Speedrunning

Yoshinari Fujinuma, Keisuke Kamahori, Ryuto Koike, Abdelrahman Madkour, Varun Prashant Gangal, Monty Bichouna, Martyna Markiewicz, Shivani Jain, Duncan Curtis, Rebecca Qian, Anand Kannappan
arXiv 2610.08076
October 6, 2026
speedrunbench
https://arxiv.org/abs/2610.08076
“Melted Colors”, Great Sandy Desert, Western Australia“Melted Colors”, Great Sandy Desert, Western Australia

FigmaTrace: Capturing Creative Nuances in Human Figma Design Workflows

Darshan Deshpande, Yoshinari Fujinuma, Martyna Markiewicz, Devanshu Bansal, Shivani Jain, Nicholas Saban, Chirag Maheshwari, Anand Kannappan
arXiv 2608.21460
August 20, 2026
figmatrace
https://arxiv.org/abs/2608.21460
“Serene Expressions”, off the Louisiana coast“Serene Expressions”, off the Louisiana coast

HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models

Emmy Liu, Varun Gangal, Michael Yu, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng
NeurIPS 2026
NeurIPS 2026
May 19, 2026
Main
halluworld
https://arxiv.org/abs/2605.19341
NeurIPS 2026
“Earth’s Aquarium”, Bering Sea“Earth’s Aquarium”, Bering Sea

Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

Darshan Deshpande
NeurIPS 2026
NeurIPS 2026
May 7, 2026
Main
world-modeling-for-agentic-rl
https://arxiv.org/abs/2607.16204
NeurIPS 2026
“Luminescence”, Suwannee River plume, Gulf of Mexico“Luminescence”, Suwannee River plume, Gulf of Mexico

Unlocking Prompt Infilling Capability for Diffusion Language Models

Yoshinari Fujinuma, Keisuke Sakaguchi
COLM 2026 workshop
COLM 2026 Workshop
April 4, 2026
Workshop
prompt-infilling-for-diffusion-lms
https://arxiv.org/abs/2604.03677
COLM 2026 Workshop
“A Study in Color”, Kenya“A Study in Color”, Kenya

Adaptive Environments For LLM Agents Need to Jointly Scale Task, Scaffolding and Reward Design

Darshan Deshpande, Varun Gangal, Li Siyan, Anand Kannappan, Rebecca Qian
Preprint
February 18, 2026
generative-simulators
“Konari, Iran”, Zagros Mountains“Konari, Iran”, Zagros Mountains

DETOUR: An Interactive Benchmark for Dual-Agent Search and Reasoning

Li Siyan, Darshan Deshpande, Anand Kannappan, Rebecca Qian
ACL 2026 workshop
GEM Workshop at ACL 2026
January 30, 2026
Workshop
detour
https://arxiv.org/abs/2602.00352
GEM Workshop at ACL 2026
“Alluvial Fan”, Taklimakan Desert, China“Alluvial Fan”, Taklimakan Desert, China

Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysis

Darshan Deshpande, Anand Kannappan, Rebecca Qian
ICML 2026
ICML 2026
January 27, 2026
Main
trace
https://arxiv.org/abs/2601.20103
ICML 2026
“Meighen Island”, Canadian Arctic“Meighen Island”, Canadian Arctic

Position: We Need A Unified Definition of Hallucination (It’s The World Model, Stupid!)

Emmy Liu, Varun Gangal, Chelsea Zou, Michael Yu, Xiaoqi Huang, Alex Chang, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng
ICML 2026
ICML 2026
December 25, 2025
Main
unified-hallucination-framework
https://arxiv.org/abs/2512.21577
ICML 2026
“Copper and Blue”, Northwest Territories, Canada“Copper and Blue”, Northwest Territories, Canada

Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge

Yoshinari Fujinuma
Findings of ACL 2026
Findings of ACL 2026
October 21, 2025
Main
contrastive-decoding-for-llm-judges
https://arxiv.org/abs/2510.18196
Findings of ACL 2026
“Desert Ribbons”, Morocco“Desert Ribbons”, Morocco

MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments

Darshan Deshpande, Varun Gangal, Hersh Mehta, Anand Kannappan, Rebecca Qian, Peng Wang
NeurIPS 2025 workshop
NeurIPS 2025 Workshop
October 1, 2025
Workshop
memtrack
https://arxiv.org/abs/2510.01353
NeurIPS 2025 Workshop
“Capillaries”, Meta River, Colombia–Venezuela border“Capillaries”, Meta River, Colombia–Venezuela border

TRAIL: Trace Reasoning and Agentic Issue Localization

Darshan Deshpande, Varun Gangal, Hersh Mehta, Jitin Krishnan, Anand Kannappan, Rebecca Qian
arXiv 2505.08638
May 13, 2025
trail
https://arxiv.org/abs/2505.08638
“Icy Vortex”, Foxe Basin, northern Canada“Icy Vortex”, Foxe Basin, northern Canada

Browsing Lost Unformed Recollections: A Benchmark for Tip-of-the-Tongue Search and Reasoning

Sky CH-Wang, Darshan Deshpande, Smaranda Muresan, Anand Kannappan, Rebecca Qian
ACL 2025
ACL 2025
March 24, 2025
Main
blur
https://arxiv.org/abs/2503.19193
ACL 2025
“Himalayas”, Eastern Himalaya, southwestern China“Himalayas”, Eastern Himalaya, southwestern China

Humanity’s Last Exam

Center for AI Safety, Scale AI and HLE Contributors Consortium
Nature 2026
Nature
January 24, 2025
Main
humanitys-last-exam
https://arxiv.org/abs/2501.14249
Nature
“Slessor Glacier”, Antarctica“Slessor Glacier”, Antarctica

GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking

Darshan Deshpande, Selvan Sunitha Ravi, Sky CH-Wang, Bartosz Mielczarek, Anand Kannappan, Rebecca Qian
arXiv 2412.14140
December 18, 2024
glider-sota-slm-judge
https://arxiv.org/abs/2412.14140
“Re-entry”, Jebel Kissu, Sudan“Re-entry”, Jebel Kissu, Sudan

Lynx: An Open Source Hallucination Evaluation Model

Selvan Sunitha Ravi, Bartosz Mielczarek, Anand Kannappan, Douwe Kiela, Rebecca Qian
arXiv 2407.08488
July 11, 2024
lynx-hallucination-detection-model
https://arxiv.org/abs/2407.08488
“Desert Patterns”, Erg Chech, Algeria“Desert Patterns”, Erg Chech, Algeria

FinanceBench: A New Benchmark for Financial Question Answering

Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, Bertie Vidgen
arXiv 2311.11944
November 20, 2023
financebench
https://arxiv.org/abs/2311.11944
“Guinea-Bissau”, West Africa“Guinea-Bissau”, West Africa

SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models

Bertie Vidgen, Nino Scherrer, Hannah Rose Kirk, Rebecca Qian, Anand Kannappan, Scott A. Hale, Paul Röttger
arXiv 2311.08370
November 14, 2023
simplesafetytests
https://arxiv.org/abs/2311.08370
No items found.