About

I like systems where the interesting part is below the API surface: schedulers, memory managers, serving control planes, data pipelines, and the benchmarks that make performance claims honest. My current focus is AI infrastructure for LLM inference.

Recently I built mini-vLLM, a small LLM serving engine simulator for learning and testing paged KV cache, continuous batching, preemption, and prefix caching. At ASML I am designing the runtime agent infrastructure for the Tachyon SQA process. Incoming SDE intern at Amazon AI Studio (Sep–Nov 2026).

  • LLM servingKV cache management, request scheduling, batching, preemption, prefix reuse, inference metrics.
  • Agent runtimesDurable workflows, human-approval gates, async job execution, result reconciliation.
  • AI systemsReliable extraction, evaluation, schema fidelity, long-context pipelines, production constraints.

News

  • 2026.09Wrote Two Simple Loops, on human-governed skill evolution and stage checkpoints in SQA agent runtime.
  • 2026.09Incoming SDE intern at Amazon AI Studio.
  • 2026.07Published The Shell Is a Protocol Now, on agent-native CLIs as execution and governance boundaries for enterprise workflows.
  • 2026.06Started as a Software Engineering in Test intern at ASML on Tachyon SQA: daily regression and async test-plan lifecycle with QA approval gates.
  • 2026.05Published Infra Learning Path, a textbook-style AI infrastructure path anchored on vLLM and mini-vLLM.
  • 2026.05Published Cloud Computing, from the machine up, a Chinese learning site for OS concepts, cloud architecture, Kubernetes, serverless, and GPU inference.
  • 2026.05Published Data Systems Illustrated, a CS186-style visual companion for databases from SQL down to storage and recovery.
  • 2026.05Released mini-vLLM: a reproducible, CPU-only simulator of modern LLM serving internals.
  • 2026.05Updated benchmark suite for continuous batching, prefix caching, preemption, and serving latency tradeoffs.
  • 2025.08Started UCSF research on LLM-based clinical note structure extraction.

Selected Systems

Tachyon SQA runtime

ASML, Software Engineering in Test intern

Runtime agent infrastructure for the Tachyon SQA process. The control plane drafts a test plan, pauses for QA, runs jobs asynchronously, and writes results back — then evolves the harness itself under human audit, instead of silently accumulating prompt residue.

  • Designed the runtime agent infra for Tachyon SQA: a workspace harness plus an execution layer that owns test-plan state across draft, approval, job run, and result writeback.
  • Built a daily regression tracking harness that materializes review sections from regression inputs, reads QA gates from Confluence, and drives follow-up work through durable runtime state.
  • Encoded the test-plan lifecycle as a durable workflow: draft test plan → capture job execution details → wait for QA approval → agent executes jobs asynchronously → update job results and inspection results when each job finishes.
  • Built a human-governed self-evolution loop: while a human uses the harness, the agent extracts reusable memory from the session into a temporary shared-memory layer; humans periodically audit and edit that layer; after approval, the agent persists the changes into the harness internals.
  • Used Temporal-style durable-execution primitives: deterministic workflow orchestration, activities for side-effecting adapters, signals for QA approval, task-queue job claiming, retries/idempotent writes, event-history replay, and local sandbox recovery.
next stage LLM Tool stage outcome Human Approval
Human-gated stage loop
Agent loop Shared memory Canonical Knowledge Skill/Harness Discover reusable skills Human Review Correct Agent Update Retrieve
Human-governed skill evolution
Durable execution Temporal Workflows / Activities Signals Task queues Shared memory Human audit Python

Ludus

Cloud agent infrastructure

A Kubernetes-native control plane for isolated, stateful agent workspaces: provision from runtime templates, suspend and resume, then garbage-collect on TTL.

  • Designed the sandbox control plane with CRDs and controllers, warm-pool provisioning, and lifecycle reconciliation.
  • Hardened untrusted-code execution with gVisor/Kata RuntimeClasses, namespace-scoped RBAC, default-deny NetworkPolicies, and resource quotas.
  • Standardized reproducible agent environments with immutable OCI images, short-lived secret projection, PVC-backed workspaces, and OpenTelemetry traces.
Kubernetes CRDs gVisor OCI OpenTelemetry

mini-vLLM

LLM serving control plane, from scratch

A compact implementation of the control-plane ideas behind high-throughput LLM inference. It keeps compute simulated on purpose so memory management and scheduling behavior are easy to inspect, test, and benchmark on a laptop.

  • Implemented paged KV-cache allocation, block reuse, preemption, prefix caching, and continuous batching.
  • Added deterministic workload benchmarks and unit tests for scheduler, cache, engine, and prefix behavior.
  • Used the project as a concrete way to study vLLM-style serving internals instead of only reading papers.
7x more active sequences 1.53x sustained throughput 54x lower p99 TTFT
Python PagedAttention KV cache Scheduler Benchmarking

Clinical note structure extraction

UCSF research, clinical NLP and LLM inference

A pipeline for turning long, unstructured clinical records into standardized structured outputs under privacy and schema constraints.

  • Decomposed monolithic extraction into smaller schema-specific subtasks to reduce context waste and improve debuggability.
  • Built domain datasets and fine-tuning experiments with Qwen, QLoRA, self-consistency checks, and schema validation.
  • Optimized long-context inference with vLLM/PagedAttention and quantization to improve throughput and memory use.
vLLM PyTorch QLoRA MedSpacy Structured extraction

GradeSync / GradeView

UC Berkeley course infrastructure

A distributed course-management system for normalizing student score data across Gradescope, PrairieLearn, iClicker, and internal course workflows.

  • Migrated fragmented course data into a normalized PostgreSQL-backed service with role-aware access control.
  • Added Redis-backed caching and pre-rendering paths to reduce repeated query load.
  • Maintained production-facing course tooling used by students and staff.
PostgreSQL Redis React Node.js RBAC

Learning

Infra Learning Path

AI infrastructure textbook anchored on vLLM

A six-month path for learning LLM serving systems through vLLM: request lifecycle, PagedAttention, scheduler design, kernels, mini-vLLM, and frontier serving systems.

  • Connects operating systems concepts to real LLM serving internals instead of teaching them as isolated theory.
  • Includes diagrams, source-reading anchors, self-checks, paper reading order, and a mini-vLLM project arc.
  • Adds a frontier serving chapter on prefill/decode disaggregation, KV transfer, prefix-aware routing, speculative decoding, and FP8/KV quantization.
vLLM PagedAttention KV cache LLM serving Systems

Cloud Computing, from the machine up

Cloud / OS illustrated companion in Chinese

A learning site that follows one cloud request down through CPU scheduling, virtual memory, I/O, virtualization, containers, networking, Kubernetes, serverless, and GPU inference.

  • Organizes 14 chapters around the path from a single machine to cloud-scale infrastructure.
  • Connects CS162-style operating systems concepts to virtual machines, containers, orchestration, and AI serving.
  • Published as a static learning artifact under the site’s Learning section.
Cloud Operating systems Kubernetes Serverless GPU inference

Data Systems Illustrated

CS186-style visual companion for database systems

A local-first illustrated textbook that walks from SQL and relational algebra down to buffer pools, B+ trees, query execution, locking, recovery, distributed commit, and modern data systems.

  • Organizes 12 database systems chapters around diagrams, invariants, and I/O cost intuition.
  • Includes a B+ tree sandbox plus static visual chapters for sorting, joins, optimization, recovery, and distributed transactions.
  • Built as a personal learning artifact rather than a copy of course notes.
Databases CS186 Storage Transactions Recovery

Writing

Experience

  • Sep - Nov 2026
    Incoming Software Development Engineer Intern, Amazon AI Studio

    Upcoming internship on Amazon AI Studio.

  • 2026 - present
    Software Engineering in Test Intern, ASML

    Designed runtime agent infrastructure for the Tachyon SQA process: daily regression tracking, a test-plan lifecycle with QA approval gates, asynchronous agent job execution with result / inspection writeback, and a human-governed self-evolution loop (session → temporary shared memory → human audit → persist into the harness). Built on Temporal-style durable workflows.

  • 2026 - present
    Co-Founder & Software Engineer, Ludus

    Cloud agent infrastructure: Kubernetes-native sandbox control plane, isolated workspaces, and untrusted-code execution.

  • 2025 - present
    Undergraduate Research Assistant, UCSF

    Clinical NLP, LLM fine-tuning, long-context inference, and schema-constrained extraction.

  • 2025 - present
    Lead Systems Architect, UC Berkeley

    Course infrastructure for GradeSync and GradeView under Prof. Dan Garcia.