Fundamental Research Labs

Fundamental Research Labs isn't a typical software company. They're an a16z and Prosus-backed AI research lab whose team comes from MIT EECS, Stanford NLP Group, Google X, and Citadel. Their flagship product, Shortcut.ai, is an AI-powered analyst trusted by firms like KKR, Microsoft, OpenAI, and Anthropic. Their Fairies platform powers autonomous AI agents, originally built for Minecraft simulations under their earlier name LyfeGame. Their research program, Project Sid, demonstrated thousands of AI agents developing specialized roles, democratic governance systems, and cultural propagation.

The work they do requires a constant cycle of training model checkpoints, deploying them for inference, testing quality, and evaluating performance against standardized benchmarks. That cycle needs to be fast, reliable, and repeatable. Manual deployment means wasted researcher time. Inconsistent evaluation means unreliable results. No environment isolation means one experiment can break another.

They needed infrastructure purpose-built for ML research workflows: a platform that understands model checkpoints, GPU requirements, inference APIs, and evaluation pipelines as first-class concepts. Not a generic dashboard bolted onto a deployment script. A system where a researcher can go from checkpoint to benchmark results without context-switching between different tools.

That's what we built.

Deploying an AI model for inference sounds straightforward until you account for the variables. A 7B parameter model needs different infrastructure than a 70B parameter model. Different model architectures use different tool call formats. Some checkpoints live on HuggingFace, others in cloud storage. GPU selection depends on model size and throughput requirements. Memory utilization needs to be maximized to justify GPU costs without crashing.

The deployment platform handles this automatically. It detects parameter count from the checkpoint and routes to the appropriate inference provider: Modal with vLLM for smaller models that benefit from fast cold starts, Baseten with Truss for larger models that need multi-GPU setups. Tool call format detection runs at deploy time so the inference API handles function calling correctly regardless of architecture. GPU tier is configurable per deployment with sensible defaults, and auto-scaling brings idle deployments down to zero cost.

The inference layer is OpenAI-compatible, so any tool that works with GPT works with these endpoints. A LiteLLM-based API gateway handles authentication with per-key rate limiting (requests per minute and tokens per minute), key expiration, and endpoint caching for fast routing.

Evaluation is equally complex. The service integrates EleutherAI's lm-evaluation-harness for 60+ academic benchmarks, the Berkeley Function Call Leaderboard for tool-use evaluation across multi-turn and language-specific variants, and SWE-Gym for evaluating models on real GitHub issues. SWE-Gym runs in Docker containers for complete environment isolation, with distributed execution on Modal scaling to thousands of concurrent tasks. Airflow DAGs orchestrate different evaluation modes for local models, API-hosted models, and distributed SWE-Gym workloads.

A professional CLI wraps the entire pipeline. Researchers run evaluations, list available benchmarks, test configurations, and manage settings from the terminal. The CLI supports local models, remote API endpoints, quantization options, automatic device detection, and configurable batch sizes.