From Unity scenes to model serving — a decade-long arc

People who hear “game developer turned AI engineer” often expect a reinvention story—leave one career behind, learn a new vocabulary, and begin again. From inside the work, it felt more continuous.

The technologies changed: Unity scenes became distributed game services, then browser runtimes, model artifacts, and inference APIs. The enduring task was still to turn a complex system into a responsive, explainable product under real constraints.

A player does not care that a frame hitch came from garbage collection instead of rendering. A shopper does not care that a delayed result came from preprocessing instead of inference. The user experiences one budget and one failure. Learning to reason across that boundary is what transferred.

Timeline

The capability-transfer arc

1. 2015–18
   Mobile games

   Gameplay state, frame budgets, memory, device lifecycle, store releases, and live tuning.

   Learned to debug from user-visible behavior into the engine.
2. 2018–21
   Real-time and transactional backends

   Authoritative state, retries, reconnects, ledgers, fairness, and auditability.

   Learned that network failure is a normal state and identity is the basis of recovery.
3. 2021–23
   Shared game and immersive platforms

   Reusable contracts, WebGL, Three.js, WebXR, and frameworks consumed by multiple teams.

   Learned to optimize for adopters and migrations, not only implementation elegance.
4. 2023–now
   AI and model-serving systems

   Data and artifact versioning, evaluation, quality gates, routing, browser inference, and product semantics.

   Applied deterministic systems discipline around probabilistic components.

The first transferable skill: budgets

Game development made performance concrete. A feature either fit inside the frame budget on the target device or it damaged the experience. Average FPS was not enough; a periodic 100 ms spike was visible even when the average looked healthy.

That mental model maps directly to AI product latency. Model inference is only one part of the path:

user-perceived latency =
  capture + upload + quality checks + preprocessing + queueing + inference
  + postprocessing + downstream lookup + rendering

Optimizing a model from 90 ms to 70 ms is valuable only if it is the limiting term. If image decode, network transfer, or recommendation lookup dominates the tail, the model benchmark is locally impressive and globally irrelevant.

Games also taught me to measure on representative hardware. Editor performance on a development machine says little about a mid-range phone under thermal pressure. The same is true for browser inference: artifact download, warm-up, memory pressure, runtime capability, and sustained execution belong in the benchmark.

State moved from scenes to servers

Early game code often lets scene objects become the state of the world. It works until persistence, replay, testing, or multiplayer requires a stable domain model. Separating domain state from presentation created the first important architectural shift in my career.

In multiplayer systems, that separation became non-negotiable. The client submits intent; the authoritative service validates it against a versioned state and publishes a transition. A reconnecting client receives a snapshot plus newer events rather than trying to merge arbitrary scene memory.

// Illustrative state-transition contract.
type Command = { playerId: string; expectedVersion: number; action: Action };
type Transition = { version: number; accepted: boolean; events: DomainEvent[] };

function apply(state: MatchState, command: Command): Transition {
  if (command.expectedVersion !== state.version) return reject("STALE_STATE");
  if (!rules.allow(state, command)) return reject("INVALID_ACTION");
  return accept(evolve(state, command.action));
}

Model-serving systems need the same explicitness, even though the core output is probabilistic. Requests carry model and schema context. Results distinguish usable output from a user-correctable quality failure and an infrastructure failure. Downstream consumers should not infer meaning from an undocumented tensor position or empty object.

Retries taught me to look for identity

Mobile networks disappear. WebSockets reconnect. Workers repeat messages. The naïve question is “How do I make this request happen exactly once?” The more useful question is “What business action is this, and how will every participant recognize that it already happened?”

In transactional systems, the answer is a stable idempotency key and an atomic record of the effect. In model pipelines, it may be a request identity tied to input, artifact version, and operation so a retry can reuse or safely reproduce the result. In both cases, infrastructure delivery semantics do not replace a domain identity.

System architecture

One production reasoning model

1. 01 User invariant What must remain trustworthy in the experience? Outcome
2. 02 State contract What facts and versions make the operation unambiguous? Correctness
3. 03 Execution budget Where can time, memory, compute, and attention be spent? Performance
4. 04 Operational evidence How is one result explained across components? Observability
5. 05 Recovery How does retry, reconnect, fallback, or rollback preserve the invariant? Resilience

Platform work changed the definition of “done”

When I built a feature for one game, the user was the player. When I worked on shared multiplayer and immersive systems, another engineer became a user too. A framework is not complete when its happy-path API looks clean; it is complete when adopters can migrate, diagnose failures, and operate it without its author beside them.

That changed how I evaluated abstractions:

• Does the default path encode the safe behavior?
• Are escape hatches explicit and observable?
• Can a consumer test the contract without running the entire platform?
• What happens when producer and consumer versions differ?
• Is the migration cost smaller than the duplicated problem?

The same questions now apply to model platforms. A training pipeline that can produce one strong artifact is useful. A platform must also reproduce it, compare it, register it, promote it, serve it, observe it, and roll it back while preserving the consumer contract.

Probabilistic components require stronger deterministic boundaries

The largest conceptual change in AI engineering was accepting that correct code can produce an uncertain answer. The response is not to make the entire system vague. It is to make the surrounding contracts more explicit.

A production vision flow can deterministically validate file shape, locate a usable region, record artifact lineage, enforce schema, and route low-confidence outcomes—even though the model score is probabilistic. An LLM system can constrain tools, preserve evidence, validate output shape, and enforce termination—even though generation varies.

The model should be probabilistic where the problem requires it. Permissions, versions, financial effects, retries, and audit records should not be.

Tradeoff matrix

What carried forward and what changed

Option	Strengths	Costs	Decision
Frame loop → inference pathChosen	End-to-end budgets, tail latency, allocation awareness	AI adds artifact warm-up and quality variance	Directly transferable
Authoritative state → model contract	Versioned inputs, outputs, and ownership	Model output is not fully deterministic	Transfer the boundary, adapt the semantics
Game telemetry → AI evaluation	Instrument behavior and regressions	Ground truth and slices need deliberate design	Transfer the discipline, expand the method
Engine abstraction → AI framework	Potential reuse and paved paths	Premature abstraction can freeze exploration	Wait for repeated evidence

The decisions that compounded

The most valuable decisions were not tied to one tool:

1. Learning to profile before optimizing. It prevents local benchmark work from substituting for product improvement.
2. Separating state from presentation. It enables tests, replay, migration, and multiple consumers.
3. Treating failure as a state. Retry, reconnect, low confidence, and degraded service become designed behavior.
4. Versioning contracts, not just code. Artifacts, schemas, preprocessing, and events evolve independently.
5. Writing for the next operator. Correlation, diagnostics, and runbooks turn incidents into bounded investigations.
6. Building abstractions after repetition. Real usage reveals the stable contract better than speculation.

The choices that did not compound were usually technology-specific cleverness without a durable problem model. Framework expertise expires. A clear understanding of state, time, ownership, and evidence travels.

Advice for engineers crossing domains

Do not hide your prior domain to look like a beginner in the new one. Translate it precisely.

• Replace “I know engine X” with “I can profile and stabilize a real-time execution path.”
• Replace “I built multiplayer” with “I understand authoritative state, retries, reconnects, and versioned contracts.”
• Replace “I shipped games” with “I have operated user-facing software across device, network, release, and live-ops constraints.”

Then identify what does not transfer. Moving into AI required learning evaluation design, data lineage, model behavior, serving runtimes, and the limits of probabilistic outputs. Prior systems experience accelerated that learning; it did not eliminate it.

Build one vertical slice in the new domain and take it all the way to an operable result. A small, versioned, evaluated pipeline teaches more production truth than a wide collection of disconnected tutorials.

Bounded proof

The compounded engineering toolkit

Real time

Budgets

Reason from the user's complete path, including tail behavior and target hardware.

Distributed

Identity + state

Make retries, reconnects, versions, and authoritative transitions explicit.

Platform

Adoption

Design defaults, diagnostics, and migrations for engineers who consume the system.

AI

Evaluation

Wrap probabilistic behavior in reproducible artifacts and deterministic contracts.

The through-line is not Unity, Node.js, Three.js, or a model runtime. It is learning to find the user-visible invariant, trace it through the whole system, and create boundaries that let a team preserve it as the technology changes.