VGMLABS
Start a conversation
Start a conversation
Capabilities

Six layers, one system.

An intelligent product is not a model with an interface. It is six layers that have to agree with each other — and most production difficulty lives at the boundaries between them.

Layers
06
Covering
Purpose, technologies, architecture, failure modes
Written for
Engineering and technical decision-makers

Experience Layer

Interfaces, copilots, multimodal interaction and human oversight.

The experience layer decides what a person can see, verify, correct and authorise. In a system whose outputs are probabilistic, the interface is a control surface rather than decoration: it sets expectations, exposes provenance, and makes the difference between a suggestion and a committed action unmistakable.

Architecture considerations

  1. Distinguish drafting from committing. Reversible suggestions and irreversible effects must not share an affordance.
  2. Stream partial output where it reduces perceived latency, but never stream a value the user might act on before it is final.
  3. Attach provenance to every generated claim — source, timestamp, and the retrieval or tool call that produced it.
  4. Design the uncertain and empty states first. They occur far more often than the ideal path.
  5. Keep an escape hatch: the operator must be able to stop, edit or take over at any step.

Failure modes

  • Confident presentation of unverified output, which transfers the model’s uncertainty onto the user unnoticed.
  • Interfaces that hide tool calls, so a failed retrieval is indistinguishable from an empty result.
  • Latency with no feedback, which reads as a broken system and drives duplicate submissions.
  • Approval fatigue: so many confirmations that operators stop reading them.

How we approach reliability

We specify the interface’s state machine before building it — idle, generating, partial, verified, failed, superseded — and treat each state as a designed screen with its own copy. Provenance and confidence are data carried through the API, not styling applied at the end.

Agent Layer

Planning, memory, orchestration, tools and multi-agent collaboration.

The agent layer converts an objective into a bounded sequence of actions. It decides what to attempt next, which tool to use, what to remember, when to ask a human, and when to stop. Most production difficulty in agentic systems lives here — not in the model, but in the control loop wrapped around it.

Architecture considerations

  1. Bound the loop explicitly: step limits, wall-clock limits, token budgets and a defined terminal state for every task.
  2. Make each tool a narrow, typed, individually authorised capability. Broad tools become the system’s widest attack surface.
  3. Persist the plan and its intermediate state outside the model context so a task can be resumed, inspected and replayed.
  4. Prefer a small number of specialised agents with clear contracts over a large pool of general ones.
  5. Treat memory as a product decision. What is retained, for how long, and who can see it are policy questions before they are technical ones.

Failure modes

  • Loop non-termination: retrying a failing step indefinitely, or oscillating between two plans.
  • Compounding error, where a wrong intermediate result is treated as established fact by later steps.
  • Tool misuse — correct call, wrong arguments, right shape — which passes schema validation and still corrupts data.
  • Prompt injection reaching a privileged tool through retrieved or user-supplied content.
  • Cost and latency growth that is invisible until the monthly bill or an SLA breach.

How we approach reliability

Every agent runs under a budget and a permission set, and every tool call is traced with its arguments and result. Writes are idempotent and, where feasible, reversible. Escalation to a human is a designed path with its own interface, not an exception handler.

Intelligence Layer

Language models, vision models, specialised models and symbolic components.

The intelligence layer is the set of components that produce judgements: general and domain language models, vision and audio models, classical models, and the symbolic machinery — rules, solvers, search, verifiers — that constrains and checks them. Choosing the right mix is an engineering decision, not a preference.

Architecture considerations

  1. Route by task difficulty. A large model for open-ended reasoning and a small one for classification is usually better than one model for both.
  2. Use symbolic components where correctness is checkable — arithmetic, scheduling, eligibility, unit conversion, policy constraints — and let the model handle interpretation.
  3. Keep the model boundary abstract enough to swap providers, but do not abstract away the differences that matter, such as tool-calling behaviour and context handling.
  4. Verify before committing. A generated action that fails a deterministic check should never reach a system of record.
  5. Decide build-versus-buy on evidence: measure prompting and retrieval first, and only then consider adaptation or training.

Failure modes

  • Fluent, well-formatted output that is factually or arithmetically wrong.
  • Silent behaviour change when a hosted model is updated underneath a stable API.
  • Over-fitting to a small evaluation set that does not represent live inputs.
  • Fine-tuning applied to a problem that was actually a retrieval or prompt-context failure.
  • Symbolic rules that drift out of sync with the policy they were written from.

How we approach reliability

Model choice is recorded with the evidence behind it, and pinned by version. Anything checkable is checked by code rather than trusted from generation. Rules are stored as data with an owner and a review date, so a policy change is a data change.

Knowledge Layer

RAG, search, knowledge graphs, structured data and document understanding.

The knowledge layer determines what the system can know at the moment it answers. It converts an organisation’s documents, records and streams into material a model can ground on — with the permissions, freshness and attribution that make grounding trustworthy.

Architecture considerations

  1. Retrieval quality is set by chunking and metadata long before it is set by the embedding model.
  2. Filter by permission at query time using the caller’s identity. Post-filtering a result set leaks existence and ranking information.
  3. Enforce citation. If a claim cannot be attributed to retrieved material, the system should decline rather than compose.
  4. Model freshness explicitly: index latency, change detection and deletion propagation are functional requirements.
  5. Use graph structure where questions are relational, and text search where questions are descriptive. Most real corpora need both.

Failure modes

  • Retrieving plausible but irrelevant passages, which the model then dutifully summarises.
  • Permission bypass through a stale index or a shared cache.
  • Deleted or superseded source documents that remain answerable.
  • Chunk boundaries that split a table from its header or a clause from its condition.
  • Evaluation on questions written from the documents, which measures recall of phrasing rather than usefulness.

How we approach reliability

Retrieval is evaluated as its own component, with a question set written by domain users rather than derived from the corpus. Access control is applied at query construction. Index freshness and deletion propagation are monitored, and unattributable answers are treated as failures.

Operations Layer

Evaluation, monitoring, guardrails, security and continuous improvement.

The operations layer is how a system stays correct after launch. It defines what "working" means, measures it continuously, constrains what the system may do, and turns observed failures into changes that are verified before they ship.

Architecture considerations

  1. Write the evaluation set before the feature. It forces a definition of correct that survives contact with production.
  2. Trace end-to-end. A response you cannot reconstruct — inputs, context, tool calls, model version — cannot be debugged.
  3. Separate guardrails by intent: correctness checks, safety filters and permission enforcement have different owners and failure responses.
  4. Attribute cost and latency per feature, not per deployment, so a regression is attributable.
  5. Route every production incident into the evaluation set. Regression suites should be a record of everything that has gone wrong once.

Failure modes

  • Monitoring uptime and latency while quality degrades unmeasured.
  • Evaluation sets that stop being representative, giving false confidence.
  • Logs containing sensitive input, turning observability into a data-protection problem.
  • Guardrails so broad they block legitimate use, which pushes users to work around the system.
  • No rollback path for prompt and configuration changes, only for code.

How we approach reliability

Evaluation runs in CI on every change to prompts, retrieval configuration or model version, and results are compared against a recorded baseline. Traces are redacted at capture. Rollback is a configuration change, and it is tested before it is needed.

Infrastructure Layer

Cloud, private environments, local inference, edge systems and model optimisation.

The infrastructure layer is where the system actually runs, and the constraints of that environment propagate upward through every other layer. Latency budget, available accelerators, connectivity and data-residency rules determine which architectures are even possible.

Architecture considerations

  1. Establish the operating envelope first — latency, memory, power, connectivity, residency — and select models to fit it.
  2. Measure quality change whenever a model is quantized or distilled. Compression is a trade, and the trade must be quantified on your own task.
  3. Design for degraded connectivity rather than treating it as an error: local queueing, deferred sync, and a smaller local model as fallback.
  4. Keep the serving interface identical across environments so the application layer does not fork per deployment target.
  5. Plan the update path before the first install. Air-gapped and edge fleets are defined by how hard they are to change later.

Failure modes

  • Benchmarking on a workstation and deploying to a device with a fraction of the memory bandwidth.
  • Quantization applied without task-level evaluation, degrading exactly the reasoning the feature depended on.
  • Thermal throttling that turns acceptable latency into unacceptable latency under sustained load.
  • Hidden network dependencies — telemetry, licence checks, model downloads — that fail in an isolated environment.
  • An update mechanism that requires physical access, so the fleet is effectively frozen at v1.

How we approach reliability

Target hardware is profiled with the real workload, not a synthetic one, and compression is accepted only against measured task quality. Offline behaviour is tested by removing the network, not by mocking it. Every deployment target gets a documented update and rollback procedure.

Most engagements do not need work in all six layers. The useful first conversation is about which layer the actual constraint sits in.

Start a conversation