VGMLABS
Start a conversation
Start a conversation
Approach

The work between a demo and a system.

A prototype answers whether something is possible. Everything after that is engineering: the surrounding logic, the evaluation, the permissions, the failure paths and the handover. This page describes how that work is actually run.

Stages
05 phases / 13 disciplines
Deliverables
Documentation, evaluations and runbooks ship with the system
Includes
The conclusion "do not build this"
01

Understand

Define the problem, operating environment, data and success criteria.

02

Prototype

Validate the highest-risk assumptions using focused technical experiments.

03

Engineer

Build the complete application, orchestration, evaluation and integration layers.

04

Deploy

Optimise the system for its actual cloud, edge, on-premises or air-gapped environment.

05

Improve

Measure behaviour, evaluate failure modes and continuously refine the system.

Disciplines

How each decision gets made.

Problem discovery

Engagements begin with the workflow, not the technology. We map who does the work today, what they decide, where the process stalls, and what an acceptable outcome looks like in their words. The output is a problem statement narrow enough to be falsified — and occasionally the conclusion that the problem does not need a model at all.

Questions we answer

  • Who performs this work today, and what does it cost them?
  • What decision is being made, and against which criteria?
  • What does an unacceptable outcome look like, and who absorbs it?
  • Would a deterministic system solve this more cheaply?

Technical feasibility

Feasibility is assessed against the specific operating envelope rather than in the abstract. A capability that works in a hosted environment with a two-second budget may be impossible in a disconnected one with two hundred milliseconds. We identify the assumption most likely to break the design and test that first.

Questions we answer

  • Which single assumption, if false, invalidates the whole approach?
  • What is the latency, memory and cost budget per interaction?
  • Is the required capability demonstrated in current models, or hoped for?
  • What is the fallback when the model is unavailable or wrong?

Dataset assessment

We audit what data exists, who may access it, how current it is, and whether it can legally and practically leave its current boundary. This includes the unglamorous questions — duplication, inconsistent identifiers, undocumented conventions — that determine whether retrieval or training is viable.

Questions we answer

  • Is the data accessible, current and attributable to a source?
  • How are the same entities identified across systems?
  • What permissions govern each collection, and how are they enforced?
  • Does an evaluation set exist, and if not, who can write one?

Architecture selection

Architecture follows the constraints. Retrieval, agentic orchestration, symbolic components, fine-tuning and local inference are options with different costs and failure characteristics, and we choose between them explicitly. Decisions are recorded with the alternatives that were rejected and why.

Questions we answer

  • What is the simplest architecture that could meet the criteria?
  • Which parts must be deterministic, and which can be probabilistic?
  • Where does the system need to be inspectable or auditable?
  • What will this look like at ten times the current volume?

Model selection

Models are selected against the task, the envelope and the deployment boundary — measured on a project-specific evaluation set, not on public leaderboards. Most systems use more than one model, with the largest reserved for the reasoning that genuinely requires it.

Questions we answer

  • Which model tier does each step of the task actually require?
  • Must weights remain inside the operator’s boundary?
  • What happens when a hosted model is updated or deprecated?
  • Is a small specialised model sufficient for the highest-volume step?

Build versus buy

We prefer existing components for anything that is not differentiating: authentication, queues, vector stores, tracing, document conversion. Custom engineering is reserved for the reasoning, orchestration and domain logic that constitute the actual product. We will say when a commercial tool is the better answer.

Questions we answer

  • Is this component differentiating, or infrastructure?
  • What is the total cost of owning it, including operation?
  • Does buying create a dependency the deployment environment cannot accept?
  • Can we replace this later without rewriting the application?

Prototyping

Prototypes exist to retire risk, not to impress. Each one targets a named assumption, runs against real inputs, and produces a comparable result. A prototype that cannot fail was not testing anything.

Questions we answer

  • What specific claim does this prototype test?
  • What result would cause us to abandon the approach?
  • Are the inputs representative of production, including the awkward cases?
  • How long should this take before it stops being worth continuing?

Evaluations

Evaluation is built with the feature and run in continuous integration. Sets are written from real inputs, scored against rubrics agreed with domain users, and extended every time production surprises us. Without this, changes to prompts, retrieval or models are indistinguishable from guesswork.

Questions we answer

  • What does correct mean for this task, in reviewable terms?
  • Who owns the rubric, and how often is it revisited?
  • Does the suite include known failure modes and adversarial inputs?
  • Can two candidate configurations be compared on evidence?

Security

Model-driven systems introduce failure modes conventional threat models do not cover: instructions arriving inside retrieved content, tool permissions escalating through chained calls, and sensitive data leaking through context or traces. We threat-model the agent loop and the tool surface specifically, and enforce least privilege per tool.

Questions we answer

  • What is the maximum damage a compromised prompt could cause?
  • Which tools can write, and what authorises each call?
  • Can untrusted content reach a privileged action path?
  • What sensitive data appears in prompts, logs and traces?

Integration

Intelligent components have to operate inside systems that already exist. We integrate against identity providers, systems of record and internal services with typed contracts, idempotent writes and explicit failure semantics — so a model-side failure degrades rather than corrupts.

Questions we answer

  • Which system is authoritative for each piece of data?
  • Are writes idempotent, and are they reversible?
  • How does the surrounding system behave when the model fails?
  • Which team owns each boundary after handover?

Deployment

The deployment target is a design input from day one. Cloud, private cloud, on-premises, edge and air-gapped environments differ in latency, available accelerators, update path and permitted dependencies. We keep the serving interface stable across targets so the application does not fork per environment.

Questions we answer

  • Where must inference physically run, and why?
  • What is the update and rollback path in that environment?
  • Which dependencies are unavailable at run time?
  • Who operates this once it is live?

Observability

Production behaviour is reconstructed from recorded evidence: prompts, retrieved context, tool calls, model versions, latency and token spend, joined to outcomes. Traces are redacted at capture so observability does not become its own data-protection problem.

Questions we answer

  • Can a single response be fully explained after the fact?
  • Which quality signals are monitored, not just uptime?
  • How is cost attributed to features and to tenants?
  • What triggers an alert, and who receives it?

Maintenance

Intelligent systems drift: inputs change, hosted models are updated, source documents move, and policies are revised. Maintenance means a scheduled review of live behaviour against the original criteria, a regression suite that grows with every incident, and a versioned path for changing prompts and policy without redeploying the application.

Questions we answer

  • How is quality drift detected before users report it?
  • Who reviews and updates the evaluation set?
  • How are prompt and policy changes versioned and rolled back?
  • What documentation does the operating team need to run this alone?
Production discipline

AI systems fail outside the demo.

A demonstration runs on curated inputs, with its author present, on a good connection, with no permission model and no cost ceiling. Production has none of those properties. The gap is predictable, which means it can be engineered for.

  1. Deterministic surrounding systems

    The probabilistic component should be as small as possible. Validation, permissions, arithmetic, routing and persistence belong in ordinary code where behaviour is exact and testable.

  2. Failure handling

    Every model call, tool call and retrieval can fail, time out or return something unusable. Each needs a defined response: retry with different parameters, degrade to a simpler path, or stop and escalate.

  3. Evaluation datasets

    A held-out set of real inputs with agreed expectations is what makes improvement measurable. Without one, every change is a matter of opinion and every regression is discovered by users.

  4. Permission boundaries

    Tools and retrieval run with the caller’s authority, not the system’s. Each capability is granted individually, and the blast radius of any single compromised instruction stays bounded.

  5. Auditability

    For decisions that affect people, money or compliance, the system must be able to show its inputs, its retrieved evidence, its tool calls and who authorised the outcome.

  6. Cost controls

    Token spend scales with usage, retries and context length — often non-linearly. Budgets per task, per tenant and per feature, plus attribution in traces, keep spend predictable.

  7. Latency controls

    Interaction has a latency budget, and a chain of model and tool calls consumes it quickly. Timeouts, parallelism, caching, smaller models for high-volume steps and streaming all buy headroom.

  8. Fallback behaviour

    When the primary model, connection or index is unavailable, the system should still do something useful: serve a cached result, use a smaller local model, queue for later, or say clearly that it cannot proceed.

  9. Human intervention paths

    Someone must be able to inspect a run, correct it, approve it or stop it. That path is a designed part of the product with its own interface and its own tests.

None of this is exotic. It is the same discipline applied to any system with real consequences — made explicit because model-driven components fail in less familiar ways.

If you already know which of these is missing from your system, that is the conversation to start with.

Start a conversation