⚙millwrightdocs
04 · deploy

Past where
scikit-learn stops.

Export to one portable ONNX artifact, serve a drift-monitored endpoint, version every model with its lineage, and — the framework pointed at itself — let AutoML search for the best deployable pipeline.

Portability

ONNX in and out.

With onnx, any model — or a whole pipeline — exports to one .onnx file. Whole-pipeline export folds leading affine scalers into the estimator's graph: raw features in, predictions out. InferenceModel::load runs any ONNX file back through tract.

let mut pipe = Pipeline::new()
    .step("scale", StandardScaler::new())
    .estimator("lr", LinearRegression::new());
pipe.fit(&train)?;
let native = pipe.predict(&probe)?;

pipe.export_onnx("pipeline.onnx")?;                 // scaler + model, one graph
let model = InferenceModel::load("pipeline.onnx")?;
let via_onnx = model.predict(&probe)?;              // matches `native`

Linear and affine graphs run through tract. Millwright's native ONNX-ML interpreter executes its exported tree ensembles too, so its own linear, pipeline, forest, and generated ensemble artifacts all round-trip locally and remain portable to external ONNX runtimes.

cargo run --example portability --features "smartcore-backend onnx"

GPU inference

The same artifact, on the GPU.

With gpu-inference, InferenceModel::load_on runs an exported ONNX model through onnxruntime, so it can use a GPU. Device::Auto picks the OS-native provider — DirectML on Windows, CoreML on macOS, CUDA on Linux (feature gpu-cuda) — and falls back to CPU, so a model loads on any system; Device::Gpu requires a GPU (errors rather than silently degrading); Device::Cpu forces CPU. The onnxruntime binary is fetched at build time — no system install.

let model = InferenceModel::load_on("pipeline.onnx", Device::Auto)?;
let preds = model.predict(&probe)?;                 // matches the CPU path

// forests have no GPU kernel in onnxruntime — re-encode them as tensor ops
rf.export_onnx_gpu("forest.onnx")?;                  // Gather / MatMul / … — GPU-runnable

// split a batch across several GPUs for throughput
let pooled = InferenceModel::load_multi("pipeline.onnx", &[0, 1])?;

GPU acceleration lands on linear / large-batch graphs; onnxruntime places each op per node, so GPU-supported ops use the GPU whenever one is active. Tree ensembles have no GPU kernel and otherwise stay on CPU — export_onnx_gpu re-encodes a forest as plain tensor operations so it runs on the GPU too. Predictions match the CPU path within tolerance.

cargo run --example gpu --features gpu-inference

Operations

Registry, drift, serving.

The registry versions the ONNX artifact (content-addressed, with a reference distribution and movable tags); monitor watches the prediction stream for PSI drift; serve exposes a validated endpoint that feeds the monitor.

let reg = Registry::local("./models");
let v1 = reg.register("demand", &model, Metadata {
    metrics: vec![("r2".into(), 1.0)],
    reference: reference.clone(),   // the distribution drift watches against
    note: "baseline".into(),
})?;
reg.tag("demand", &v1.id, "prod")?;
let reverted = reg.rollback("demand", "prod")?;   // revert in one line

// serve the prod artifact, watching for drift on every request
Server::from_onnx(reg.onnx_path("demand", "prod")?)?
    .route("/predict")
    .with_monitor(DriftMonitor::psi(&reference)?)
    .serve("0.0.0.0:8080").await?;         // POST /predict, GET /metrics

cargo run --example operations --features "onnx registry monitor serve"

Serving any model. The Server runs linear / NN ONNX graphs through tract, and evaluates ONNX-ML tree ensembles (a forest) with a small native interpreter — so a model exported by Millwright always serves in Millwright, and the artifact stays portable to any ONNX runtime.
Specialized shapes

Time series & out-of-core.

Same contract, different data shapes — each gets its own trait. These two crates pin ndarray 0.15 while the rest of the stack uses 0.16; Cargo links both and converts only inside the adapters.

// time series (feature = "timeseries")
let mut arima = AutoArima::new().max_p(3).max_q(3);
arima.fit(&series)?;                  // &[f64]
let forecast = arima.forecast(6)?;   // six steps ahead

// out-of-core (feature = "incremental") — never holds the whole set in memory
let mut model = IncrementalLinear::with_rate(0.05, 0.0);
for batch in batches {
    model.partial_fit(&batch)?;      // one batch at a time
}

cargo run --example specialized --features "timeseries incremental"

Synthesis

AutoML — the framework, pointed at itself.

Profiling, preprocessing, CV, search, and ensembling are exactly what an AutoML engine needs — so AutoML is not a bolt-on, it is the framework orchestrating its own parts. Choose ONNX-only candidates or the broadest predictive search, then get a leaderboard and a fitted winner. Minute budgets are soft: an in-flight CV trial finishes before the cutoff is observed.

let result = AutoML::classifier()      // or ::regressor()
    .budget(Budget::trials(20))         // or Budget::minutes(10)
    .metric(Metric::F1)
    .cv(StratifiedKFold::new(5))
    .deployability(Deployability::Onnx)
    .seed(0)
    .fit(&train)?;

println!("{}", result.leaderboard());
result.export_onnx("model.onnx")?;   // deployable — unlike a TPOT object

cargo run --example automl --features "automl onnx"

← prevInsight