Credentials and Environment Variables¶
Access to the Mantik platform is granted by passing username and password. The Python library will use these credentials go generate an access token (JWT).
For security reasons, we follow the policy of handling credentials via environment variables instead of hardcoding them into code or configs.
The usage of Mantik platform requires several environment variables to be set on the client side:
MANTIK_USERNAME
MANTIK_PASSWORD
MLFLOW_TRACKING_URI
(default: https://api.cloud.mantik.ai/tracking/)MANTIK_API_URL
(default: https://api.cloud.mantik.ai/)MANTIK_COMPUTE_BUDGET_ACCOUNT
(optional only for HPC submissions)
Attention
In order for the tracking to work properly, the compute resources must be able to access the tracking URI. In collaboration with the Jülich Supercomputing Center, their compute facilities (JUWELS, JURECA, JUSUF) allow access to the Mantik platform from compute nodes.
For convenience, we suggest you put these variables into an .env
file and source it:
source .env
to set all required environment variables as described below.
Danger
To avoid committing your credentials to your git repository,
add the .env
file to your .gitignore
.
It is also possible to add it to your global gitignore (~/.config/git/ignore
)
to ignore it system-wide.
Storing external HPC credentials safely¶
You can set your external credentials that are required for accessing the respective HPC site with the following environment variables:
MANTIK_UNICORE_USERNAME
: Your common username for accessing the HPC siteMANTIK_UNICORE_PASSWORD
: The respective password
MANTIK_FIRECREST_CLIENT_ID
: The OIDC client ID of the OAuth client.MANTIK_FIRECREST_CLIENT_SECRET
: The respective client secret.
For further details on how to create an OIDC client with a secret, see Supported HPC Sites.
However, credentials can be stored as a Connection on Mantik. In this case, the above environment variables are not required.
Instead, the Connection UUID containing your credentials must be passed to the Python client,
or to the CLI commands with the --connection-id
flag.
Using the Connection is the preferred way to handle external credentials.
Optional Environment variables¶
When using the CLI to submit runs via command line or transfer files via Mantik’s Remote File Service, multiple project-related IDs are needed. To simplify this process and avoid specifying them as options each time you run a command, you can save these IDs as environment variables. These environment variables include:
MANTIK_PROJECT_ID
MANTIK_EXPERIMENT_REPOSITORY_ID
MANTIK_DATA_REPOSITORY_ID
MANTIK_CODE_REPOSITORY_ID
Example .env
file¶
# .env
# Required environment variables for tracking to Mantik the MLflow server
export MANTIK_USERNAME=<user>
export MANTIK_PASSWORD='<password>'
export MLFLOW_TRACKING_URI=https://api.cloud.mantik.ai/tracking/
export MANTIK_API_URL=https://api.cloud.mantik.ai/
# Required environment variables for remote execution on HPC via UNICORE.
export MANTIK_COMPUTE_BUDGET_ACCOUNT=<compute project>
Important
The
MLFLOW_TRACKING_URI
andMANTIK_API_URL
:requires the
https://
prefix,has to be the base URL of the Mantik platform in the form
https://<host>.<domain>
, e.g.export MLFLOW_TRACKING_URI=https://api.cloud.mantik.ai/tracking
. It’s important that the URI has to be the root path!
Set passwords into single quotes (’) to avoid it being interpreted as bash syntax.
Mantik Runtime Environment Variables¶
When submitting a run, the below environment variables are internally configured by Mantik and available in the run’s environment at runtime:
MANTIK_GIT_REF
: Branch or commit of the Code Repository used for run submissionMANTIK_WORKING_DIRECTORY
: Working directory of the run on the HPC siteMLFLOW_EXPERIMENT_ID
: MLflow Experiment ID of the experiment the run was created for (not equal to the Mantik Experiment ID)MLFLOW_RUN_ID
: MLflow run ID of the run created by Mantik (not necessarily equal to the Mantik Run ID)MLFLOW_RUN_NAME
: Name of the MLflow run created by MantikMLFLOW_TRACKING_TOKEN
: JWT used for authenticating at the Mantik and MLflow APIMLFLOW_TRACKING_URI
: MLflow API URL of Mantik
Attention
The MLFLOW_TRACKING_USERNAME
and MLFLOW_TRACKING_PASSWORD
environment variables are automatically preferred over MLFLOW_TRACKING_TOKEN
by the MLflow client when sending requests to the MLflow API.
Thus, if those environment variables are set, requests to Mantik’s MLFlow API will fail since it expects the JWT set as MLFLOW_TRACKING_TOKEN
.