back to work

[ ai ]auto-mlsha 1168651measured-in-repo

It drives thewhole lifecycle,under guard.

Upload, EDA, natural-language SQL, preprocessing, training, experiments, deployment. Each phase is an agent that proposes Python, runs it under human approval, and repairs its own failures. It never executes model-generated code in-process.

~137k
LOC TypeScript
~2,208
automated tests
23
SQL migrations
382
React components

every figure counted from the source tree at the pinned commit

[ ai ]auto-mlsha 1168651measured-in-repo

The path from a messy CSV to a deployed model is hand-stitched.

Getting a raw, messy dataset to a model in production normally means wiring together pandas, scikit-learn, a pile of notebooks, an experiment tracker, and serving code by hand. Every handoff is manual, every step is a place to lose state, and nothing keeps a record of why a given transform happened.

The question I asked is narrow and concrete. Can an LLM agent drive that entire lifecycle, from exploration through NL querying, preprocessing, training, experiments, and deployment, while a human stays in the loop at every commit and no model-generated Python ever runs in-process? That last constraint is the hard part. An agent that writes and runs arbitrary code is a remote-code-execution surface by construction, so the whole design has to treat the model's output as hostile.

[ ai ]auto-mlsha 1168651measured-in-repo

One agent per phase. It proposes, you approve, the sandbox runs.

Every ML stage is an LLM agent that proposes an action, generates Python into a notebook cell, and validates the result through MCP tool calls, gated on operator approval. The agent never touches a Python process directly. It writes code into a cell, then a hardened Docker container with a persistent Jupyter kernel executes it. Kernel state survives across cells, so a scaler fitted early is still in memory many cells later, the same way a human's notebook session works.

When a cell fails, the agent does not silently emit bad output. The failure feeds a bounded auto-repair loop: the error context goes back into code generation, the agent rewrites the cell, and it re-runs, up to a hard cap, after which control returns to the operator. Validation is its own state in the graph, not an afterthought, so "the code ran" and "the code did the right thing" get checked separately.

The control surface is the Model Context Protocol. 12 tools (get_dataset_profile, run_cell, edit_cell, search_documents and the rest) are exposed to the model over the official MCP SDK with an InMemoryTransport, after schema sanitization strips internal fields the model has no business seeing.

[ ai ]auto-mlsha 1168651transcribed-from-source

The preprocessing engine is a guarded state machine.

The LangGraph runtime walks a guarded path: context, plan, generate, execute, validate, approve, commit. Failure routes back to code generation. Success routes through human approval before anything commits. This is the verified transition function in preprocessingRuntime.ts.

control plane · React 19 SPA + Express 5 + OpenAI SDKpreprocessing runtime · resolvePreprocessingTransition()execution plane · isolated, untrusted by defaultLLM agentOpenAI SDKMCP registry12 tools · InMemoryapproval gatehuman in the loopapprove / rejectPostgres + files23 migrations · pgvectorWebSocketlive cell streamcontextcontext_readyplanplan_stepgenerategenerate_codeexecuteexecute_codevalidatevalidate_outcomeapproveawait_approvalcommitcommit_or_reviseexecute / validate fails -> bounded auto-repair -> generate_codedocker sandbox--network none · --read-only rootfs · --user sandbox (non-root)mem / cpu caps · nosuid tmpfs · datasets mounted :ro · SSRF blocked via add-hostJupyter kernelpersistent statenotebook cellsmodel-generated Pythongenerated Python -> sandboxed cell
State machine transcribed from backend/src/services/llm/langgraph/preprocessingRuntime.ts:91-161 (resolvePreprocessingTransition); sandbox flags from services/container/dockerBuilder.ts, both covered by tests. Solid green is the success path; amber dashed is the bounded auto-repair loop back to generate_code.

[ ai ]auto-mlsha 1168651design-notes

Two decisions I kept, two boundaries I did not cross.

The boundaries I drew on purpose, and the two I never reached.

kept

Split persistence on purpose

File-backed storage for project JSON, dataset bytes, and model artifacts. Postgres for auth, embeddings, notebooks, and workflows. Big binary blobs do not belong in a relational store, and metadata does not belong in loose files.

CLAUDE.md · Architecture.md

kept

Sandbox is untrusted by default

Every execution container ships with --network none, a read-only root filesystem, a non-root user, memory and CPU caps, and an add-host rule that blackholes SSRF to the host even if networking is later turned on. The model's code is treated as hostile from the first byte.

dockerBuilder.ts · verified by tests

loss

The LangGraph engine is still a scaffold

The preprocessing state machine names its own compiled graph preprocessing-langgraph-scaffold. Production preprocessing runs through services/workflows. That seam has not yet migrated.

preprocessingRuntime.ts:272

loss

No unified benchmark runner exists yet

The repo defines benchmark suites against public datasets (Titanic, Ames Housing, Credit Card Fraud, Adult Income) but commits no measured results. The design notes pressure-test the benchmark concept, reject a naive Jaccard agreement metric, and conclude the runner is unbuilt.

docs/expo-benchmark-design-notes.md

[ ai ]auto-mlsha 1168651counted-at-sha

What is actually measured in the repo.

No model-quality numbers are committed. The receipts below are all counted from the source tree at the pinned SHA.

~2,208

test cases

1,229 backend + 908 frontend + 71 landing, across ~246 *.test.ts(x) files

~137k

lines of app code

~54k backend TS + ~83k frontend TS/TSX, excluding tests

382

React components

plus 39 Zustand stores and 39 custom hooks

23

SQL migrations

001_init through 021, sequential schema evolution

12

MCP tools

official SDK, InMemoryTransport, schema-sanitized for the model

4

NL-to-SQL phases

schema context, plan, generate, read-only validate

The sandbox, flag by flag

backend/src/services/container/dockerBuilder.ts · every flag in one auditable function, covered by dockerBuilder.test.ts

--network noneno egress by default, model code cannot phone home
--read-onlyimmutable root filesystem, writes only to scoped tmpfs
--user sandboxnon-root execution, dropped privileges
--memory / --cpushard resource caps per container
--tmpfs /tmp:nosuidwritable scratch that cannot escalate
-v datasets:/datasets:rodata mounted read-only, code cannot corrupt the source
--add-host host.docker.internal:0.0.0.0SSRF to the host blackholed even if networking is later overridden

the unearned number

There is no committed "7x faster" benchmark.

The resume credits this project with deploying models 7x faster than manual Jupyter. I believe that from building it, but the repo does not prove it: the benchmark quality gate is literally set to 'tbd' in expo-public-p0.v1.json and no run artifacts are committed. What the tree actually holds: the test count, the LOC, the migrations, the MCP tools, the exact sandbox flags.

The system is real and the engineering is auditable. The headline speed number is not yet earned.

[ ai ]no hosted demo

Run the agent loop from the repo.

The guarded loop above (propose, execute in the sandbox, auto-repair, approve) is the real agent runtime, not a staged replay. Clone the project to run it against your own dataset.

run it from the repo

[ ai ]real product captures

Six phases, one workspace.

read the write-up

auto-ml · agentic AutoML platform · pinned at sha 1168651 on main · GPL-3.0
all figures and receipts are counted from the repository at that SHA.