Skip to content
Agile Defense

Curriculum / Data Scientist on Foundry

CAPSTONEUnit 8unit-08

Capstone: operationalized disruption-risk forecast in the Ontology

not started~360 min
An end-to-end governed capability: Ontology objects carry a model-derived DisruptionRisk, a published model Function powers on-demand scoring, with documented lineage and monitoring.

Introduction

Scenario: Operations leadership has approved the disruption-risk model for production and asks you to stand up the full capability end-to-end: every Supplier in the Ontology must carry a model-derived DisruptionRisk, analysts must be able to score a supplier on demand, and a handoff package must let another team run and trust it after you leave.

This capstone assembles everything from the prior units into one governed, operationalized disruption-risk forecast. You will not train a new model or invent a new artifact; instead you verify that the pieces you built — a published model asset and version, a Modeling Objective that is the system of record for evaluation, a batch-inference pipeline that writes risk onto Ontology objects, and a published model Function for on-demand scoring — actually cohere into a capability operators can use. The disruption-risk forecast is only as trustworthy as the lineage and evaluation behind it, so the deliverable is both the live capability and the documentation that makes it defensible.

Two facts shape how this unit is checked, and both are honesty points you should internalize. First, a Modeling Objective is a project for context, data, metadata, and successive model versions centered on one operational problem — it is the system of record for evaluation and operationalization — but there is no documented public read API that exposes an objective's submissions, MetricSet metrics, releases, or deployments. So the objective's internals and the evaluation numbers are confirmed by self-attestation in the handoff, not by an API call. Second, models reach the Ontology two distinct ways here: batch inference writes a DisruptionRisk band onto objects as a stored property (verifiable by Ontology aggregation), and a published model Function (a thin query-function wrapper over a direct model deployment) powers on-demand scoring (verifiable by executing the query). Neither of those is the live-deployment REST path from Unit 6 — that uses the separate foundry-ml-live POST transport and is out of scope for this unit's automated checks.

By the end you will have proven, through the platform's own read and execute APIs, that the model artifact exists, the model Function resolves and returns a schema-valid prediction, and the Ontology reflects the model output as a high-risk-supplier population reproducible by aggregation — and you will have written a handoff package covering training lineage, evaluation metrics, deployment and refresh cadence, and model-risk caveats. Treat the API-confirmable steps as the spine of trust and the manual handoff as the connective tissue that documents what the APIs cannot see.

Capability focus: End-to-end ML lifecycle in the Ontology: model + objective + batch inference + live deployment + model Function + monitoring. · Artifact: An operationalized disruption-risk forecast: Ontology-bound risk + a published model Function + a handoff package.

Key concepts

  • Modeling Objective (system of record): A Modeling Objective is a project for context, data sources, metadata, and successive model versions centralized around one operational problem; it is the system of record for evaluating, reviewing, and operationalizing model solutions and functions as a catalog of production-worthy versions. Its submissions, MetricSet metrics, releases, and deployments are NOT exposed by any documented public read API — existence of the FOUNDRY_ML_OBJECTIVE resource is confirmable, but its internals are not.
  • Model asset and version: A Foundry model is an inference artifact comprising a model artifact (trained weights/container) and a ModelAdapter (a Python class defining load/init, an api() input/output schema, and predict()). Publishing produces a versioned model with lineage and training-dataset provenance. The MODELS_MODEL and MODELS_MODEL_VERSION resources are confirmable by Filesystem Get By Path (existence + type), but the training code, ModelAdapter internals, and deployment binding are not read-API introspectable.
  • Models in the Ontology: Models connect to the Ontology two ways used in this capstone — batch inference (Pipeline Builder trained-model node plus Add an Ontology output) writes model predictions as object properties, and Functions on objects bind object properties to model inputs for on-demand scoring. This is how model output becomes object-state that operators consume in Actions and Workshop.
  • DisruptionRisk band (stored string property): The operationalized risk is stored on the object as a STRING band (for example 'high'/'medium'/'low'), distinct from any separate numeric risk score. Storing it as a band is deliberate: object-state filters match on the string, so an equality-count Aggregate Objects query can reproduce the high-risk-supplier population. A raw numeric score cannot be matched by an equality filter the same way.
  • Published model Function (query function): A model can be directly published as a Function — a thin wrapper that queries a direct model deployment by its stable URL and auto-upgrades with new model versions on the branch. It is imported and called as a query function for live inference in Workshop and other applications, and is confirmed by EXECUTION: Execute Query returns the prediction's output shape, which both proves existence and validates the I/O signature.
  • Aggregate Objects v2: POST /api/v2/ontologies/{ontology}/objects/{objectType}/aggregate computes count/min/avg/max/sum/approximateDistinct with optional where filters and groupBy. Filtering Supplier on disruptionRiskBand = 'high' and counting reproduces the high-risk-supplier population the capstone narrative cites, proving the model output landed in object-state.
  • live deployment vs. model Function transport: A Modeling Objective live deployment is a persistent scalable REST endpoint queried by POST to the foundry-ml-live transport (.../foundry-ml-live/api/inference/transform/ri.foundry-ml-live.<RID>/v2). That transport is SEPARATE from Ontology Execute Query: the published model Function is reached through Execute Query, while the live deployment is reached through foundry-ml-live and returns 'blocked' when unreachable. Do not conflate the two paths.

Companion video

Introduction to Machine Learning Operations · watch the full build → evaluate → deploy → serve walkthrough · open on YouTube

Hands-on activity

each step validates · the unit completes when all steps pass
  1. 1

    Composite model artifacts exist across the lifecycle

    Confirm the lifecycle's central artifact still exists and is resolvable before you assert the rest of the capability. Use Filesystem v2 Get By Path against the model's path; a successful resolve returns the resource's rid and a type of MODELS_MODEL, which is existence-and-type confirmation only — Get By Path does not introspect the training code, the ModelAdapter api()/predict() internals, or the deployment binding behind it. The published model version (MODELS_MODEL_VERSION) and the Modeling Objective (FOUNDRY_ML_OBJECTIVE) were confirmed the same way in Units 2 and 3; here you are re-anchoring that the model asset the rest of the capstone depends on is present and addressable, not validating its contents.

    not startedinstance check

    Confirms the published model asset exists (the lifecycle's model artifact; version + objective confirmed in U2/U3).

  2. 2

    Published model Function resolves

    Confirm the on-demand scoring surface is enumerable as a published query function. The model Function is a thin wrapper that queries a direct model deployment by its stable URL and auto-upgrades with new model versions on the branch; it is the path Workshop and Actions call, NOT the foundry-ml-live live-deployment transport. Confirm it resolves as a published query type (its apiName is enumerable and its parameters/output I/O signature are present). This step proves the scoring surface is published and discoverable; the next-but-one step proves it actually returns a prediction. Keep clear in your head that this resolution is via Ontology query metadata, separate from the live deployment's foundry-ml-live POST endpoint, which would return 'blocked' if unreachable.

    not startedinstance check

    Confirms the published model Function is enumerable (the on-demand scoring surface).

  3. 3

    Ontology reflects model output end-to-end

    Prove the model output was operationalized into object-state by reproducing the high-risk-supplier population through aggregation. Batch inference wrote a DisruptionRisk band onto Supplier objects as a stored STRING property; because it is a string band rather than a raw numeric score, an equality filter can match it directly. Issue an Aggregate Objects v2 count over Supplier with where disruptionRiskBand = 'high' and confirm the count matches the high-risk figure the capstone narrative cites. This is the end-to-end evidence that the model's predictions landed in the Ontology as object properties operators can filter and act on — the numeric risk score, if present, is a separate property and is not what this equality-count check reads.

    not startedinstance check

    Confirms the Ontology reflects the model output: the high-risk supplier count reproduces via aggregation.

  4. 4

    On-demand scoring works via the Function (execute)

    Prove the operational scoring path is live by executing the published model Function, not merely resolving it. Run Execute Query on the model Function for a seeded object (supplierId 'SUP-1') and confirm it returns a schema-valid, non-empty disruption-risk prediction containing the expected output keys. Execution is the strongest confirmation available for this surface: it both proves the Function exists and validates its output shape, which is why the published model Function is confirmed by EXECUTION rather than by an existence-only path resolve. This Execute Query path is the Ontology query-function transport; it is distinct from the foundry-ml-live live deployment, so a successful prediction here does not assert anything about the live deployment's REST endpoint.

    not startedinstance check

    Executes the published model Function and confirms a schema-valid, non-empty disruption-risk prediction.

  5. 5

    Handoff package documents lineage, evaluation & monitoring

    Write the handoff package, which is a manual, self-attested deliverable because no read API can confirm it. Document the training lineage (which datasets fed the model and how to reproduce it), the evaluation metrics and acceptance decision recorded in the Modeling Objective — these MetricSet numbers must be transcribed by hand because the objective's submissions, metrics, releases, and deployments are not exposed by any documented public read API — the deployment and refresh cadence (how often batch inference re-scores objects, and how the model Function auto-upgrades on the branch), and the model-risk caveats (drift, segment error, and the boundary between the stored DisruptionRisk band and any separate numeric score). State explicitly which parts of the capability are API-verifiable and which rest on this document, so the receiving team knows exactly what trust is automated and what is attested.

    not startedself-attested

    Self-attested: the handoff covers training lineage, evaluation metrics, deployment/refresh cadence, and monitoring.