Introduction

Scenario: Leadership will not green-light a disruption-risk forecast on a data scientist's word that the model 'looks good' — they need a governed system of record showing the candidate was evaluated against held-out data and cleared an agreed bar. You will stand up a Modeling Objective, submit the model you trained, and evaluate it so the supplier-disruption forecast has an auditable acceptance decision behind it.

Unit 3 moves the disruption-risk model out of a notebook and into governance. In the previous unit you published a Foundry model asset (a trained model artifact plus a ModelAdapter that declares the input/output API). That asset proves the model can run, but it says nothing about whether the model is good enough to forecast supplier disruption in production. A Modeling Objective is the application that answers that question: it is a project-like system of record centered on a single operational problem — here, 'predict which suppliers/shipments are at risk of disruption' — that catalogs successive candidate model versions, evaluates each against a held-out dataset, and records the review decision that promotes one toward operationalization.

The mechanics matter because they make evaluation reproducible rather than anecdotal. When you submit a model version to an objective, Foundry takes an immutable copy of that version so the candidate under review can never silently change. The objective's auto-generated evaluation pipeline scores that copy against an evaluation dataset and produces a MetricSet — a single model's measurement (AUC, precision, RMSE, and charts) against one evaluation dataset. Modeling-objective checks act as quality gates: thresholds and conditions a candidate must clear before anyone can package it as a Staging/Production release. The acceptance threshold you carry in from Unit 1's problem framing is the bar the chosen metric must clear here.

Be clear-eyed about what is and is not verifiable from outside the UI. The objective resource and the evaluation dataset are confirmable through public read APIs, so those steps are graded by an automated check. But the objective's internals — its submissions, the MetricSet values, releases, and review state — are NOT exposed by any documented public read API (the old 'define modeling objective API' is sunsetted and only ever covered I/O schema). That is why submitting the model and confirming the metrics meet threshold are honest, self-attested manual steps. The deliverable is a Modeling Objective holding the submitted candidate and its MetricSet over the held-out data, with a recorded accept/reject decision a reviewer can defend.

Capability focus: Modeling Objectives; candidate submission; MetricSet (AUC/precision/RMSE) + acceptance threshold. · Artifact: A Modeling Objective with the submitted model and a MetricSet, with a recorded acceptance decision.

Key concepts

Companion video

1. 1

   Modeling Objective exists

   Create the Modeling Objective in the Modeling Objectives application: choose New objective, select a storage location that contributors, stakeholders, and consumers can reach, add a description framing the operational problem ('predict supplier/shipment disruption risk'), and configure submission metadata, model evaluation, and the modeling-objective checks that encode your acceptance threshold. The automated check resolves the objective's path with Filesystem v2 Get By Path and confirms it returns type FOUNDRY_ML_OBJECTIVE with a rid. This is existence-only: Get By Path proves the objective was created at the expected path, but it cannot read the objective's submissions, MetricSet, releases, or review state — no documented public read API exposes those internals (the 'define modeling objective API' is sunsetted and only ever covered I/O schema), so later steps are manual by necessity.

   not startedinstance check

   Run check

   Confirms a Modeling Objective exists (type FOUNDRY_ML_OBJECTIVE).

2. 2

   Evaluation dataset exists with expected schema

   Make sure the held-out evaluation dataset the objective will score against actually exists and has the columns an evaluation needs. The automated check runs Get Dataset plus Get Dataset Schema (datasets v2) on the evaluation dataset path and confirms it resolves and contains both a `prediction` column and a `ground_truth` column — the model's scored output paired with the observed disruption label, which is what any AUC/precision/RMSE computation must align row-for-row. This confirms a real, schema-correct evaluation input exists; it does NOT confirm the metric values are correct or that the predictions are any good. Use the same held-out test split from Unit 1 so the evaluation reflects data the model never trained on.

   not startedinstance check

   Run check

   Confirms the evaluation dataset resolves with prediction + ground-truth columns.

3. 3

   Model submitted to the objective as a candidate

   Submit the model version you published in Unit 2 to this objective so it becomes a candidate under review. In the Modeling Objectives UI, choose the model asset and version and submit it; Foundry creates an immutable copy of that version and lists it as a candidate. This step is MANUAL and self-attested: candidate-submission state is not exposed by any documented public read API (R-ML1), so the grader cannot confirm it programmatically. Attest honestly that the specific version was submitted and now appears as a candidate, and record which version (by version number) you submitted so the acceptance decision is traceable. Do not confuse this with the model asset's existence, which Get By Path can see — submission lives inside the objective, which read APIs cannot introspect.

   not startedself-attested

   Mark complete

   Self-attested: the model version was submitted (immutable copy) as a candidate in the objective.

4. 4

   Metrics computed and meet the acceptance threshold

   Run the objective's evaluation so the candidate is scored against the held-out dataset, then confirm the resulting MetricSet clears the acceptance threshold you set in Unit 1's problem framing (for example, the chosen metric — AUC, precision, or RMSE — meets or beats the agreed bar), and record an explicit accept or reject decision with that number. This step is MANUAL and self-attested: MetricSet values are not retrievable through any documented public read API (R-ML1) and are read only in the Modeling Objectives UI, so no automated check can verify the metric or the threshold. Write down the metric, its value, the threshold, and the decision in the objective's submission metadata / review notes so a reviewer can later defend why this candidate was (or was not) advanced toward a release.

   not startedself-attested

   Mark complete

   Self-attested: the MetricSet (AUC/precision/RMSE) clears the chosen threshold.