Running directly from Hugging Face, embedded live below — no screenshots, no static mockups.
This Space is the visible layer of a bigger constraint: development happens on a MacBook and an AMD desktop — neither has an NVIDIA GPU — so CUDA and lower-level C/C++ kernel work only exists here because of free-tier cloud compute. The goal is to mimic production-grade ML infrastructure end-to-end, at zero out-of-pocket cost, using biochemical data as the working domain (methodology drawn from N. Flynn's Build AI Drug Discovery Pipelines (Manning, 2025)).
The Job Board's Neon Postgres backend already models versioned records; the next step extends that same pattern into a training/inference registry — dataset, model, and code versions tracked together — feeding a transformer serving path that, longer-term, adopts paged-attention/paged-state serving optimizations.
The compute grid below is that plan made concrete: five free tiers spanning CPU, GPU, and TPU, each with real constraints rather than idealized specs. ZeroGPU is live today; Colab's T4 is next for kernel debugging outside ZeroGPU's metered queue; Colab's TPU v5e-1 is the eventual target for GPU-vs-TPU inference benchmarking. None of it requires a credit card — that's deliberate, so the same path is repeatable by anyone else without out-of-pocket cost.
Specs and limits reflect each provider's published free tier as of now and change without notice — treat them as directional, not contractual.
vLLM popularized PagedAttention, but it isn't the only path to it — and it's a different axis from FlashAttention entirely. FlashAttention is a compute-time optimization: a fused, IO-aware kernel that avoids ever materializing the full attention matrix. PagedAttention is a serving-time one: it manages the KV cache in non-contiguous blocks the way an OS pages virtual memory, so concurrent requests can share GPU memory efficiently. FlashInfer, TGI, SGLang, MLC-LLM, and TensorRT-LLM (with its in-flight batching) all implement or wrap that same block-table mechanic independently of vLLM's serving stack — for a custom backend, dropping in FlashInfer's kernels directly is usually lighter than inheriting a full framework.
It's also no longer Transformer-only. Mamba/SSM layers carry a fixed-size recurrent state rather than a growing KV cache, and hybrid architectures like Jamba mix both — which is why vLLM is refactoring its own allocator from "paged attention" into "paged state management," a block-table abstraction general enough for recurrent state and KV cache alike.
For a custom architecture, the practical path is to adapt FlashInfer's block-sparse and ragged-tensor abstractions where the state resembles a KV cache; where it doesn't, it's paged state management from scratch — a pre-allocated GPU memory pool, a block table mapping logical sequence IDs to physical blocks, and a hand-written CUDA/Triton kernel that fetches non-contiguous blocks against it.
The broader design principle carries over here: decouple training and compilation from serving — compile once, ahead of time, and serve from that artifact — then, within serving, lean on existing frameworks where they fit and go manual where the goal is understanding, not just throughput. The Colab T4 tier above is reserved for that manual layer: CUDA/Triton first, with an eventual move to Rust and Zig — Rust for compiler-enforced memory safety and more tractable assembly-level debugging, Zig for explicit allocator control — treating the rewrite as a potential improvement on the existing frameworks, not just an exercise.
Dataset + model iteration
Colab CPU
One-time cost, ahead of serving
decoupled from serving
48GB shared slice, app-level only
HF ZeroGPU
Next: ZeroGPU only exposes application-level execution — no hardware-level profiling access. Long-term kernel benchmarking (assembly/PTX-level, register-level) moves to Colab T4 GPU / TPU v5e-1, where the instance is fully addressable.