IntroductionΒΆ
In addition to HPC, the Mantik platform, and itβs API can be used to run applications on local system, in such case, the set-up and execution is significantly simpler than the HPC runs. It follows the MLflow project conventions
In summary, two essential files are required:
Python script (or package) with CLI: To run a Python application with Mantik, you need either at least one Python script or a Python package that has a CLI.
This script - or the code that is invoked by the CLI - might, for example, be responsible for training a ML model or using a trained model for inference. In any case, this script may use MLflow methods to track training/inference parameters, model metrics, and artifacts (e.g. plots). (Details on experiment tracking are explained in Using MLflow for Tracking.)
Here, we assume that an application has a Python script that makes use of the
argparse
module to parse arguments passed to the script to modify the behavior of the code at execution time. This allows to use Mantik to its full potential.A CLI - either in a script or via a package - is optional, though, and an application might as well allow no arguments.
MLproject file: Defines the applicationβs entry points, i.e. its scripts and their arguments.
Its structure is defined by the MLflow conventions for MLprojects.
This file declares how your script can be invoked and what parameters it allows. The MLproject file is used by Mantik to allow modification of input parameters to an application.
Details are explained in Preparing Your Application.
If the these two files are provided, a project can be executed locally either via the Python library, or the CLI.
Alternatively, for a quick start, where using a jupyter notebook running locally is preferred only to track metrics, values, etc. during runs, check out tracking runs in a jupyter notebook