Note: the full code for this post can be found here

I’ve been messing with python 3.10 features recently in my free time, but wanted to dig deeper into setting up something more substantial by including mypy and some automatic validation using gitlab pipelines. I’ve been a long time fan of python, and it is one of my most comfortable languages at this point so spending more time getting “up to date” sounded like fun.

Let’s assume that python3.10 is installed and go from there.

First, let’s get everything working locally (I’ll be using poetry here, but pip works fine, too. Just ignore the poetry info/steps if you decide to use pip):

> poetry add mypy
...
> poetry add pytest
...
> poetry install
...

mypy and pytest should be set up now so let’s try testing them out:

> poetry run mypy --version; poetry run pytest --version
mypy 0.961 (compiled: yes)
pytest 7.1.2

If you see something similar to the above then you’re good to go.

Now that mypy is up and working the next step is to throw together some code that we can use it to check. Feel free to put whatever you want in or copy over the code from example.py. There is also a pretty bare mypy.ini file that can be copied, but will include it below since it isn’t too large:

[mypy]
python_version = 3.10
warn_return_any = True
disallow_untyped_defs = True

The last line will cause errors to be thrown if mypy finds any untyped functions. Let’s test it to check that mypy is picking up the config file:

> mypy .
logic/example.py:12: error: Function is missing a return type annotation
Found 1 error in 1 file (checked 4 source files)

Looks good. I had removed a return type on line 12 just before executing so this is exactly what was expected.

Feel free to do the same for the test/ directory and any tests that may or may not be there. I will include a step for executing mypy on the tests as well as running the tests using gitlab’s pipelines, but you can omit those if you don’t care about that piece right now.

Now that everything looks to be glued together pretty well locally we can move on to the CI.

For the sake of keeping this brief, I’m going to use a public image that includes poetry and python already installed since installing poetry before each pipeline job is unnecessary in my opinion. For prod, create your own image or install each time.

For the pipeline, we’ll be doing everything in three stages with one job per stage:

  • build: Setup Dependencies*
    • add/install dependencies and cache the generated poetry environment for later use
  • type-check: Check Types
    • use mypy to check the project for any typing rule violations listed in mypy.ini
  • test: Test
    • use pytest to run any/all tests in the project

Note: we are skipping a deploy step since this is just an example, but there is plenty of info on getting that setup depending on your preferences.

If errors came up during any of the above I recommend checking out Gitlab’s pipeline documentation, mypy’s docs, or poetry’s docs (if applicable). Otherwise, that’s it! You should now have a small “up to date” python project set up with CI using Gitlab’s pipelines.

*If you are using poetry there is an important before_script piece that allows us to cache the poetry virtual environment so that we don’t have to re-install the dependencies each time. Poetry will default the virtualenv to a place other than the working dir which is inaccessible to Gitlab pipelines, so we must specify that we want the virtualenv to be stored in the working dir. We do this with the config variables virtualenvs.in-project and virtualenvs.path by setting them to true and .venv respectively. We then tell the pipeline that we are caching the .venv directory and reference that same folder in each of our pipelines cache section so they know where to look.