Skip to content
Agile Defense

Curriculum / AI Engineer — AIP Builder

CAPSTONEUnit 8unit-08

Capstone: integrated, evaluated, governed decision-support release

not started~360 min
The end-to-end Disruption Copilot: published function + eval suite + chatbot + governed Action, demonstrated on a seeded disruption with an auditable recommendation-to-edit trace.

Introduction

Scenario: A live supply-chain disruption hits the seeded region, and leadership wants the integrated Disruption Copilot to take one event from raw signal to a governed, auditable response. As the AIP engineer, you must demonstrate the whole capability end-to-end — recommendation, conversation, governed edit, and proof it was evaluated — on a single seeded case.

This capstone stitches together everything the course built into one deployable decision-support capability and proves it on a live disruption. Across the prior units you authored a recommendation function in AIP Logic with a grounded Use LLM block, added a code-authored TypeScript classifier as a Call function tool, backed the reasoning with a registered (BYOM) model, evaluated the function with an AIP Evals suite, exposed it conversationally through an AIP Chatbot built in AIP Chatbot Studio with least-privilege tools, and wrapped the resulting Ontology edit in a governed, staged-for-review Action. Today you run all of it together against the seeded DISR-CAP-1 disruption and produce an auditable recommendation-to-edit trace.

The point of the capstone is integration and defensibility, not new features. AIP connects AI to the Ontology and to operations, and its three builder surfaces — AIP Logic, AIP Chatbot Studio, and AIP Evals — must compose into a single workflow whose every consequential step is governed and traceable. The published recommendation function (recommendDisruptionResponse) is the decision core; the AIP Chatbot is the conversational surface that calls it as a Function tool under native tool calling; the governed Action is the only path by which a recommendation becomes a durable Ontology edit, so a human reviews before anything is committed. Responsible-AI governance here is concrete: granular permissions, audit logs of model interactions and decisions, and human-in-the-loop checkpoints via checks.

A recurring theme — and the reason this unit is honest about what can and cannot be machine-verified — is that much of an AIP capability is reviewable but not readable through an API. A published Logic or TypeScript function is confirmed only by execution: Execute Query runs the latest published version by apiName and you inspect the output shape. The AIP Chatbot is confirmed through the AIP Agents v2 Get Agent endpoint, which is in preview. The governed edit is proven by the resulting DisruptionEvent object state. But AIP Evals suites and the live chatbot demo leave no read-accessible artifact, so those are self-attested. Knowing which claims rest on execution, which on object state, and which on your own attestation is the difference between a demonstrable capability and a story about one.

Capability focus: End-to-end governed AIP capability: Logic + TS function + evals + chatbot + governed Action over one Ontology. · Artifact: The integrated Disruption Copilot capability demonstrated on a seeded disruption.

Key concepts

  • AIP and its builder surfaces: AIP connects AI with data and operations over the Ontology. The three builder tools are AIP Logic (no-code LLM-powered functions), AIP Chatbot Studio (interactive assistants called AIP Chatbots), and AIP Evals (a testing environment for functions). The capstone is the composition of all three into one governed capability.
  • Published functions confirmed by execution: A published AIP Logic or TypeScript function is invoked via the Execute Query API (POST /api/v2/functions/queries/{queryApiName}/execute, preview=true), which runs the latest published version by apiName with a parameters map and returns the result as JSON. Confirmation is by EXECUTION and output shape — internals such as Logic blocks, the prompt, and the output schema are never API-inspectable. Pre-publish functions have no queryApiName and are not addressable by Execute Query at all.
  • AIP Chatbots as a conversational surface: An AIP Chatbot (built in AIP Chatbot Studio) is powered by an LLM plus the Ontology, documents, and custom tools, and enforces least privilege so the model accesses only what is necessary. Native tool calling supports Action, Object query, Function, and Update application variable tools; the Function tool lets the chatbot call the published recommendation function. The chatbot is confirmed via AIP Agents v2 Get Agent (preview=true), which returns the agent's rid, version, metadata, and parameters.
  • Governed Ontology edits via Actions: The only durable path from a recommendation to the Ontology is a governed, function-backed Action that stages the edit for review (submission criteria / human-in-the-loop) rather than auto-applying it. AIP governance provides granular permissions and Markings, audit logs of model interactions and deployment decisions, and structured approval workflows via checks. The edit is proven after the fact by the DisruptionEvent OBJECT STATE (owner/status set), because the staged-review/submission-criteria configuration itself is not enumerable through a read API.
  • AIP Evals — reviewable, not readable: AIP Evals tests AIP Logic functions, AIP Chatbot functions, or code-authored functions using test cases (inputs + expected outputs), evaluation functions, evaluation suites, and metrics comparable in aggregate or per test case. A suite and its objectives are demonstrated in the product, but suites, test cases, evaluators, and metrics are NOT retrievable through a read API, so evaluation is self-attested in this course.
  • Registered models (BYOM) and the defense caveat: Bring-your-own-model (registered models) connects a customer LLM to AIP, registered once in Control Panel (AIP settings > Registered models) and usable across AIP Logic, AIP Chatbot Studio, and TypeScript functions, with rate limits and usage observability in Resource Management. The registration config and rate limits are not exposed by any decision-support read API, and registered (BYOM) models do NOT support Markings — a material caveat for defense deployments.

Companion video

Build with Us | Deep Dive: Speedrun Your First AIP Workflow · open on YouTube

Hands-on activity

each step validates · the unit completes when all steps pass
  1. 1

    Composite: recommendation function executes end-to-end

    Start with the decision core. Run the published recommendation function recommendDisruptionResponse against the seeded capstone disruption (disruptionId DISR-CAP-1) using the Execute Query API (POST /api/v2/functions/queries/{queryApiName}/execute with preview=true), which executes the latest published version by apiName and returns the result as JSON. The check confirms the structured recommendation came back with its expected keys present — action, rationale, and confidence — so a single live disruption produces a well-formed, machine-readable recommendation. Note honestly what this does and does not prove: it proves the function executes and returns the agreed output shape, but it cannot inspect the Use LLM block, the prompt, the registered model behind it, or the output schema, because those internals are not API-inspectable. Execution and output shape are the only confirmable evidence here.

    not startedinstance check

    Executes the published recommendation function on the seeded capstone disruption and confirms a structured result.

  2. 2

    Composite: chatbot exists

    Confirm the conversational surface exists. The Disruption Copilot is an AIP Chatbot built in AIP Chatbot Studio that calls recommendDisruptionResponse as a Function tool under native tool calling, and it is confirmed via the AIP Agents v2 Get Agent endpoint (preview=true), which returns the agent's rid, version, metadata, and parameters for the published chatbot. The check looks up the chatbot by its rid and verifies it exists. Be explicit about the preview honesty rule: because Get Agent is a preview API, when that endpoint is absent the check returns 'blocked' rather than silently passing — a blocked result means 'could not verify,' never 'verified.' This step confirms only that the published chatbot is real; the chatbot's tool wiring, system prompt, and least-privilege scoping are configured and reviewed in AIP Chatbot Studio but are not enumerable through this endpoint.

    not startedinstance check

    Confirms the published Disruption Copilot chatbot exists (preview → blocked, never silent pass).

  3. 3

    Composite: governed edit produced expected object state

    Prove the recommendation reached a governed, auditable Ontology edit. A recommendation only becomes durable through the governed, function-backed Action that stages the edit for human review rather than auto-applying it, so the proof is the resulting DisruptionEvent object state, not the Action's configuration. The check reads governed DisruptionEvent objects and confirms at least one has its owner field set (non-empty) and a status of either 'resolved' or 'escalated' — exactly the fields the governed Action writes once a reviewer approves. This demonstrates the full recommendation-to-edit trace landed in the Ontology under governance. State plainly why object state is the evidence: the staged-review and submission-criteria configuration is not enumerable through a read API, so the only confirmable artifact is the edit's effect on the object, which the audit log can corroborate alongside it.

    not startedinstance check

    Confirms the recommendation reached an auditable, governed Ontology edit (DisruptionEvent owner + status).

  4. 4

    Composite: evaluated and demonstrated live

    Close the loop with the two things only you can attest. First, show that an AIP Evals suite over recommendDisruptionResponse ran and passed its objectives — demonstrate the suite's test cases, evaluators, and aggregated metrics directly in AIP Evals, because suites, test cases, evaluators, and metrics are NOT retrievable through any read API (R-EVAL1), making evaluation inherently self-attested. Second, demonstrate the chatbot live: ask the Disruption Copilot a question about the seeded disruption and show it invoking the governed recommendation end-to-end, since a live conversation leaves no read-accessible artifact. This manual step is honest by design — neither the eval results nor the live demo can be machine-verified, so your demonstration and a reviewer's eyes are the evidence. Tie it together by narrating the single trace: live disruption to executed recommendation to governed, reviewed Ontology edit, all under AIP's permissions and audit logging.

    not startedself-attested

    Self-attested: an AIP Evals suite passed its objectives and the chatbot was demonstrated live.