How to contribute¶
Contributing as a user¶
Reporting bugs¶
This section guides you through submitting a bug report for mantik. Following these guidelines helps understand your report, reproduce the behavior, and find related reports.
Notes:
Before submitting a bug report, check that your issue does not already exist in the issue tracker.
Make sure your issue is really a bug, and is not a support request.
If you find a closed issue that seems like it is the same thing that you’re experiencing, open a new issue and include a link to the original issue in the body of your new one.
Submit your bugs in the issue tracker:
Use the bug issue template. This can be selected from the dropdown menu in the
Description
field.Completely fill out the template.
Save the issue.
Suggesting improvements¶
You may submit enhancement suggestion such as new features, improvements of existing functionality, or performance by submitting a ticket. Before submitting a new suggestion, check that your issue does not already exist in the issue tracker.
Code contributions¶
Development and testing¶
We use poetry (poetry>=1.2.0
) for dependency management and packaging.
Thus, an installation of poetry is required.
A guide on how to install poetry can be found here.
The repo contains several different components (libraries):
mantik
: Client for the Mantik API.tokens
: Verification of JWTs issued by the Mantik API.mantik-compute-backend
: Mantik Compute Backend and Fast API app.
The components have dependencies on one another as follows:
|----> mantik <----|
| ^ |
| | |
| tokens |
| ^ |
| | |
compute backend
Setup and unit tests¶
Install all project dependencies for development
make install
Install pre-commit hooks
poetry run pre-commit install
Run the tests
poetry run pytest tests
Updating the poetry.lock
file¶
To update the dependencies, run
make lock
in order to prevent updating of packages when locking use
make lock-no-update
Bumping the package version¶
All packages maintained in this repository are versioned equally. To simultaneously increase the version of all packages run
make bump-version version=<version>
where version should be without a v
prefix, e.g. make bump-version=1.1.0
Bumping the MLflow version¶
mantik_compute_backend
, mlflow_tracking_server
, and mantik[mlflow]
use the
same MLflow version. That version can simultaneously be bumped using
make bump-mlflow-version version=<version>
where version should be the desired MLflow version.
This commands also runs updating of the lock files.
Test MLflow project¶
A minimal project is located at tests/resources/test-project
.
It simply prints the input to stdout.
Testing the project:
Build the Docker image
docker build -t mantik-test:latest -f tests/resources/test-project/Dockerfile .
Build the Apptainer image
sudo apptainer build \ tests/resources/test-project/mantik-test.sif \ tests/resources/test-project/recipe.def
Run the project locally
poetry run mlflow run tests/resources/test-project
This should print
test
to stdout. The output can be changed by passing theprint
argumentpoetry run mlflow run tests/resources/test-project -P print="hello world"
In
tests/resources/test-project/compute-backend-config.json
set theApptainer.Path
value to the local path of the Image.Set the required environment variables (see backend config section above).
Run the project via the
UnicoreBackend
poetry run tests/resources/test-project/run.py
Note: When running the example via mlflow CLI, asynchronous execution is triggered. No job results can then be accessed.
Warning: When using a local Apptainer image, the image will be compressed and send to the backend. Here, during decompression a so far unspecified error may occur that crashes the submission (see this ticket). Please, use a remote image and remove any images from your project folder.
Compute backend¶
The compute backend API is a module located in mantik-compute-backend
and depends on mantik
.
Its endpoint is /submit/<experiment_id>
.
Mlflow projects can be submitted via this endpoint.
The endpoint expects a zipped Mlflow project directory. Unicore backend configuration
should reside inside the zipped directory, its name can be specified with the
compute_backend_config_path
http form, alongside UNICORE credentials, experiment id
and mlflow parameters as serialized json.
Docker Image¶
A Dockerfile for the compute backend service is provided. To build the image, run
sudo docker build -t compute_backend_service:latest -f docker/mantik-compute-backend.Dockerfile .
To run the image locally, use
sudo docker run --env MLFLOW_TRACKING_URI=<tracking uri> --env LOGLEVEL=<desired log-level> --env LISTEN_PORT=<listen port> -p <host port>:<listen port> compute_backend_service:latest
The environment variable MLFLOW_TRACKING_URI
must be provided, LOGLEVEL
defaults to
INFO
and LISTEN_PORT
to 80
.
Test CI locally¶
Install gitlab-runner (see this example) on your machine. Usually, you you would need to register your gitlab-ci tokens, which is however not required as we want to run everything locally.
Therefore just executing the integration test in the ci can be done with a single command
gitlab-runner exec docker "<job name>"
Integration tests with UNICORE¶
For integration testing using a real UNICORE instance, we make use of the UNICORE Docker image provided by JSC. For more details see here.
To run the tests locally:
Run UNICORE locally
make up-integration
Run the integration tests
poetry run pytest tests/integration
Note: The UNICORE Docker container takes a while to process a job, hence tests can be rather slow (15-30 seconds). Thus, be careful with test cases.
Create local runs¶
For certain e2e tests or during development it might be necessary to create pre-existing runs via the CLI. These runs can be executed on your local machine. For this you need a mantik account that has management access to a project. This project needs to have an experiment, code repository and data repository configured. You can do all this via the mantik GUI. Furthermore you need to provide the URLs for the APIs of the mantik and mlflow endpoints that you want to test.
There already exist a tutorials repository that you can use for both data and code when configuring your setup via the GUI: https://gitlab.com/mantik-ai/tutorials
Export the run-specific environment variables (find the UUIDs via the GUI)
export MANTIK_USERNAME=<ACCOUNT_NAME> export MANTIK_PASSWORD=<ACCOUNT_PASSWORD> export MANTIK_PROJECT_ID=<PROJECT_UUID> export MANTIK_EXPERIMENT_REPOSITORY_ID=<EXPERIMENT_UUID> export MANTIK_CODE_REPOSITORY_ID=<CODE_REPOSITORY_UUID> export MANTIK_DATA_REPOSITORY_ID=<DATA_REPOSITORY_UUID> # IMPORTANT change these if you want to test a backend other than production, in this case it's dev2: export MANTIK_API_URL="https://api.dev2.cloud.mantik.ai" export MLFLOW_TRACKING_URI="https://api.dev2.cloud.mantik.ai/tracking"
Initialize mantik token with your exported credentials
eval $(mantik init)
Start a local run (here we use the example wine-estimator tutorial project)
mantik runs local "./wine-quality-estimator/mlproject/MLProject" \ --name "Local: wine quality estimator" \ --project-id $MANTIK_PROJECT_ID \ --code-repository-id $MANTIK_CODE_REPOSITORY_ID \ --branch main \ --experiment-repository-id $MANTIK_EXPERIMENT_REPOSITORY_ID \ --parameter alpha=0.1
Now the run should be displayed in the GUI under your project’s runs/submissions
.
Merge requests¶
Describe your changes as accurately as possible. The merge request body should be kept up to date as it will usually form the base for the changelog entry.
Be sure that your pull request contains tests that cover the changed or added code. Tests are generally required for code be to be considered mergable, and code without passing tests will not be merged. Ideally, the test coverage should not decrease.
Ensure your merge request passes the pre-commit checks. Remember that you can run these tools locally.
If your changes warrant a documentation change, the merge request must also update the documentation.
Note: Make sure your branch is rebased against the latest base branch.
Changelog¶
The file CHANGELOG
lists changes that contribute to each version of mantik. In the
Unreleased
section, changes that will be added in the next version are collected.
Update the CHANGELOG
with every merge request.
Add or update documentation pages¶
All the documentation content is stored as .md files in the docs/source
folder. This folder contains the structured content for the site, organized into subfolders based on topics.
To add new page:¶
Page is a part of the existing navigation group:
Create a new .md file in the relevant folder based on the desired structure.
Open the
index.md
file in that folder, where the toctree is defined. Add a reference to your new file. Example:
```{toctree}
:titlesonly:
new-page.md
```
Every page should have one (and only one) first level heading otherwise the page content toc is not correctly displayed.
Page in a new navigation group:
Create new folder in the
docs/source
directory.Create
index.md
file in this folder. The file will be the main entry of the navigation group and contain sub-menu.Create a new .md file for the sub-menu.
Add a reference to your new file in the
index.md
in the current folder. Example:
// docs/source/new-topic/index.md
```{toctree}
:titlesonly:
new-page.md
```
To make new navigation group appear in the left-side navigation add new toctree entry to the
docs/source/index.md
All documents listed in toctree code block will appear as part of this menu group in the navigation.
Make sure to reference the file path correctly in the index.md toctree for it to appear in the left-side navigation. Path to the file is relative to index.md
.
The navigation link takes the first title of the file as a name.
Note: For additional directives check sphinx documentation.
Types of content¶
Sub-menu:
```{toctree}
sub-topic/menu-item1.md
sub-topic/menu-item2.md
```
Assets required in the documentation like videos or pictures will be added to the folder docs/source/_static
. The file can than be referenced via relative path as listed below. The video tag embeds the video docs/source/_static/mantik-api.mp4
from a document inside a subfolder of the docs/source
e.g. docs/source/topic/chapter.md
Video:
Image:
Code:
def greet(name):
print(f"Hello, {name}!")
Preview your changes¶
cd docs
make install // (only ones)
make html
python -m http.server --directory build/html
Open http://localhost:8000 in a browser
Contributors¶
Omar Ahmed, Elia Boscaini, Fabian Emmerich, Kristian Ehlert, Sara Grau, Britta Oberlack, Thomas Seidler, Jakub Jagielski, Nastassia Kundas, Jakob Fuhrmann, Kim Lyons