Credentials and Environment Variables

Access to the Mantik platform is granted by passing username and password. The Python library will use these credentials go generate an access token (JWT).

For security reasons, we follow the policy of handling credentials via environment variables instead of hardcoding them into code or configs.

The usage of Mantik platform requires several environment variables to be set on the client side:

Attention

In order for the tracking to work properly, the compute resources must be able to access the tracking URI. In collaboration with the Jülich Supercomputing Center, their compute facilities (JUWELS, JURECA, JUSUF) allow access to the Mantik platform from compute nodes.

For convenience, we suggest you put these variables into an .env file and source it:

source .env

to set all required environment variables as described below.

Danger

To avoid committing your credentials to your git repository, add the .env file to your .gitignore.

It is also possible to add it to your global gitignore (~/.config/git/ignore) to ignore it system-wide.

Storing external HPC credentials safely

You can set your external credentials that are required for accessing the respective HPC site with the following environment variables:

  • MANTIK_UNICORE_USERNAME: Your common username for accessing the HPC site

  • MANTIK_UNICORE_PASSWORD: The respective password

  • MANTIK_FIRECREST_CLIENT_ID: The OIDC client ID of the OAuth client.

  • MANTIK_FIRECREST_CLIENT_SECRET: The respective client secret.

For further details on how to create an OIDC client with a secret, see Supported HPC Sites.

However, credentials can be stored as a Connection on Mantik. In this case, the above environment variables are not required.

Instead, the Connection UUID containing your credentials must be passed to the Python client, or to the CLI commands with the --connection-id flag.

Using the Connection is the preferred way to handle external credentials.

Optional Environment variables

When using the CLI to submit runs via command line or transfer files via Mantik’s Remote File Service, multiple project-related IDs are needed. To simplify this process and avoid specifying them as options each time you run a command, you can save these IDs as environment variables. These environment variables include:

  • MANTIK_PROJECT_ID

  • MANTIK_EXPERIMENT_REPOSITORY_ID

  • MANTIK_DATA_REPOSITORY_ID

  • MANTIK_CODE_REPOSITORY_ID

Example .env file

# .env

# Required environment variables for tracking to Mantik the MLflow server
export MANTIK_USERNAME=<user>
export MANTIK_PASSWORD='<password>'
export MLFLOW_TRACKING_URI=https://api.cloud.mantik.ai/tracking/
export MANTIK_API_URL=https://api.cloud.mantik.ai/

# Required environment variables for remote execution on HPC via UNICORE.
export MANTIK_COMPUTE_BUDGET_ACCOUNT=<compute project>

Important

  • The MLFLOW_TRACKING_URI and MANTIK_API_URL:

    • requires the https:// prefix,

    • has to be the base URL of the Mantik platform in the form https://<host>.<domain>, e.g. export MLFLOW_TRACKING_URI=https://api.cloud.mantik.ai/tracking. It’s important that the URI has to be the root path!

  • Set passwords into single quotes (’) to avoid it being interpreted as bash syntax.

Mantik Runtime Environment Variables

When submitting a run, the below environment variables are internally configured by Mantik and available in the run’s environment at runtime:

  • MANTIK_GIT_REF: Branch or commit of the Code Repository used for run submission

  • MANTIK_WORKING_DIRECTORY: Working directory of the run on the HPC site

  • MLFLOW_EXPERIMENT_ID: MLflow Experiment ID of the experiment the run was created for (not equal to the Mantik Experiment ID)

  • MLFLOW_RUN_ID: MLflow run ID of the run created by Mantik (not necessarily equal to the Mantik Run ID)

  • MLFLOW_RUN_NAME: Name of the MLflow run created by Mantik

  • MLFLOW_TRACKING_TOKEN: JWT used for authenticating at the Mantik and MLflow API

  • MLFLOW_TRACKING_URI: MLflow API URL of Mantik

Attention

The MLFLOW_TRACKING_USERNAME and MLFLOW_TRACKING_PASSWORD environment variables are automatically preferred over MLFLOW_TRACKING_TOKEN by the MLflow client when sending requests to the MLflow API. Thus, if those environment variables are set, requests to Mantik’s MLFlow API will fail since it expects the JWT set as MLFLOW_TRACKING_TOKEN.