Development

Quickstart

  • Clone the repository:

    git clone https://github.com/Systems-Theory-in-Systems-Biology/EPI.git

    git clone git@github.com:Systems-Theory-in-Systems-Biology/EPI.git
    Should I choose https or ssh? You can clone the repository over https or ssh. Use https if you only want to obtain the code. Use ssh if you are a registered as developer on the repository and want to push changes to the code base. If you want to contribute to the project but are not a registered developer, create a fork of the project first. In this case, you have to clone your fork, not this repository.

  • Install uv:

    curl -LsSf https://astral.sh/uv/install.sh | sh

  • Install dependencies:

    • For amici (sbml):

      sudo apt install swig
sudo apt install libblas-dev
sudo apt install libatlas-base-dev
sudo apt install libhdf5-dev

    • For cpp:

      sudo apt install libeigen3-dev

  • Install eulerpi:

    uv sync --all-extras

  • Run the tests:

    uv run pytest

    You can add the --verbose parameter to get a more detailed report.

Maintaining the repository

Here are the most important information on how to maintain this repository.

Dependency Management with uv

We use uv to manage the project dependencies and the virtual python environment. During the Quickstart we installed all dependencies into the virtual environment, therefore:

IMPORTANT

Run all commands in the next section in the uv shell. It can be started with source .venv/bin/activate. Alternatively, you can run commands with uv run <yourcommand>.

Run uv add package_name to add the library/package with the name package_name as dependency to your project. Use uv add --group dev package_name to add package_name to your dev dependencies. You can have arbitrary group names.

For more information read the uv Documentation.

Code quality checks

We use ruff to maintain a common code style to lint the code. Please check your code install the pre-commit hook:

pre-commit install

You can also check your changes manually:

pre-commit run --all-files

Testing with pytest

pytest

You can generate a coverage report by running the following code block in your terminal. Please be aware that it might take a long time, think about lowering the number of steps in the sampling.

coverage run -m pytest -v
coverage report
coverage html

Running the tutorial (jupyter notebook)

The jupyter notebook can be run using

  • vs code: https://code.visualstudio.com/docs/datascience/jupyter-notebooks

  • shell + browser: jupyter notebook

In the first case you need to select the uv virtual env when selecting the interpreter, in the second case you need to run the command in the uv shell.

Profiling with scalene

You can profile eulerpi with scalene (or gprofile) using the commands:

python3 -m pip install -U scalene
scalene tests/profiling.py

This will create a profile.html file, which you can open using your browser. Do not rely on the OPENAI optimization proposals. They are often plain wrong in scalene.

Documentation with Sphinx

cd docs
sphinx-apidoc -e -f -o source/ ../
make html

All extensions of sphinx which are used to create this documentation and further settings are stored in the file docs/source/conf.py. If you add extensions to conf.py which are not part of sphinx, add them to the docs/source/requirement.txt file to allow github action mmaraskar/sphinx-action@master to still build the documentation.

A cheatsheet for reStructuredText with Sphinx.

Hosting with GitHub Pages

To publish the documentation on github pages you probably have to change some settings in the GitHub Repository

Settings -> Code and automation -> Pages -> Build and Deployment:
- Source: Deploy from a branch
- Branch: gh-pages && /(root)

Changelog

We use the Keep a Changelog format for the changelog. It should be updated with every pull request.

Versioning

We use Semantic Versioning. A version number is composed of three parts: major.minor.patch

  1. The major version should be incremented when you make incompatible changes.

  2. The minor version should be incremented when you add new functionality in a backward-compatible manner.

  3. The patch version should be incremented when you make backward-compatible bug fixes.

Every time a new version is tagged, a GitHub Action workflow is triggered which builds and uploads the version to pypi.

Please update the version number in the pyproject.toml file before tagging the version.

Test Deployment to TestPyPi

Build and deploy:

uv build
uv publish --index testpypi --token $TESTPYPI_TOKEN

Test this with

uvx --directory ./.. --no-project --with eulerpi --refresh-package eulerpi --index testpypi -- python -c "from importlib.metadata import version; import eulerpi; print(version('eulerpi')); print(eulerpi)"

Jax with CUDA

Jax can be run with cuda on the gpu. However, you need a recent nvidia-graphics-driver, the cuda-toolkit (cuda) and cudnn installed. Getting the versions right can cause headaches ;)

I used the following tricks to get it running:

# for cuda toolkit
export CUDA_HOME=/usr/local/cuda
export PATH="/usr/local/cuda-12.0/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda-12.0/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/usr/local/cuda-11-fake/lib64:$LD_LIBRARY_PATH"

Follow this issue to see whether you possibly need to create the cuda-11-fake folder as a copy from the 12.0 folder and “create the .so.11” libraries as symlinks using the script below.

#!/bin/bash

# Find all files in the current directory that match the pattern "lib*.so"
for file in lib*.so; do
    # Extract the base name of the file (without the ".so" extension)
    base_name="${file%.*}"

    # Construct the name of the symbolic link we want to create
    link_name="${base_name}.so.11"

    # Check if the link already exists
    if [ ! -e "$link_name" ]; then
        # Create the symbolic link if it doesn't exist
        ln -s "$file" "$link_name"
    fi
done

It can happen that old code is executed due to the generated pycache. For example, an old version of cuda or cudnn could be used. If you believe that this is happening:

pip install pyclean
py3clean .