Update
I migrated to Pipenv some time ago and being pretty happy about this.
Outdated article
For every Python project I set up a virtualenv and handle my requirements with pip-tools. It is a command line tool to keep pip-based packages fresh and pinns them. It is my favorite tool to handle Pyhon requirements. Using pip-tools you basically define your requirements in a *.infile and when you run pip-compile it will create a *.txtrequirements files with all dependencies (and all underlying dependencies) pinned. And using pip-sync it will install all packages definied in a requirements file but also uninstall any other package which is not defined.
In most cases I split my requirements files into four categories: base, dev, tests and production. When developing I often have all requirements installed, in my CI only base and tests requirements and on production only base and production requirements.
So over time I approximately came to the following script, that lies as update.sh in the root of my Python projects. On a fresh virtualenv you first need to run pip install pip-tools, change the file permissions chmod 744 update.sh and then the following script can be used.
What it simply does when you run ./update.sh compile it will use pip-tools to create your pinned requirements files.
And then you can install the different combinations with ./update.sh sync-all for all packages, ./update.sh sync-tests for base and test and ./update.sh sync-dev for base, test and dev.
There are also commands which do not require pip-toolsto be installed, these use pipto install all needed dependencies. This is useful for CI environments or production where you install the dependencies only once and dont want unneded extra tools to be installed. You can run ./update sync-ciin the CI to install base and test or ./update sync-productionto install base and production requirements.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
#!/bin/bash
function help {
echo "usage: ./update arguments"
echo " compile Upgrades the pinned version of all packages."
echo " sync-all Update your virtual environment of all packages."
echo " sync-dev Update your virtual environment for base, dev and test."
echo " sync-tests Update your virtual environment for base and test."
echo " sync-ci Update your virtual environment for base and test."
echo " sync-production Update your virtual environment for base and production."
echo " upgrade Upgrades pip, setuptools and pip-tools."
}
if [ $# -eq 0 ]; then
help
exit 1
fi
if [ "$1" == "sync-production" ]; then
pip install --no-cache-dir -r requirements/production.txt
exit 0
elif [ "$1" == "sync-ci" ]; then
pip install --no-cache-dir -r requirements/tests.txt
exit 0
elif [ "$1" == "upgrade" ]; then
pip install --upgrade pip setuptools pip-tools
exit 0
fi
if ! pip list | grep "pip-tools" > /dev/null ; then
echo "This action requires pip-tools. Run 'pip install pip-tools' or './update.sh upgrade'."
help
exit 1
fi
if [ "$1" == "sync-all" ]; then
pip-sync requirements/base.txt requirements/dev.txt requirements/tests.txt requirements/production.txt
exit 0
elif [ "$1" == "sync-dev" ]; then
pip-sync requirements/base.txt requirements/dev.txt requirements/tests.txt
exit 0
elif [ "$1" == "sync-tests" ]; then
pip-sync requirements/base.txt requirements/tests.txt
exit 0
elif [ "$1" == "compile" ]; then
pip-compile --upgrade requirements/base.in -o requirements/base.txt
pip-compile --upgrade requirements/base.in requirements/tests.in requirements/dev.in -o requirements/dev.txt
pip-compile --upgrade requirements/base.in requirements/tests.in -o requirements/tests.txt
pip-compile --upgrade requirements/base.in requirements/production.in -o requirements/production.txt
else
help
exit 1
fi