Create a Modern Python Project
Last updated: May 23, 2021
This guide will walk you through everything you need to know to set up a Python project repository with the most up-to-date tools.
You can set up your project in many different ways, but this is an opinionated guide based on my own experience of what works well.
This guide is based on using a Mac, but it should also work for Linux or using Windows Subsystem for Linux although some modifications may be necessary for your particular system.
Create a Git Repository
First create a new empty repo on GitHub or a similar service. You'll want to grab the SSH URL of the new git repo and then clone it and
cd into it:
cd ~/projects git clone firstname.lastname@example.org:<username>/<project-name>.git cd <project-name>
The HTTPS URL also works, but you'll need to enter your password every time you push to the repository.
After setting up SSH you never have to enter your password to push to GitHub.
If you haven't set up SSH on GitHub yet, check out their guide.
Add the following contents to a new file named
.venv __pycache__/ *.pyc
This file tells git to ignore files, directories, or file patterns that are automatically generated and managed by Python, so we don't want to include them in our source code repository.
We will add new files to this
.gitignore as needed. If you want a more comprehensive
.gitignore, you can use GitHub`s default Python .gitignore.
asdf to manage Python versions
You probably have a version of Python pre-installed onto your computer already, as it is included with many operating systems by default.
python --versionto find the version of Python you have installed.
which pythonto find the full path of your default Python installation.
I don't recommend using the default Python installation to build your own projects, because when you collaborate on it you'll come across issues from different people having different versions of Python on their computers.
The Python core team is constantly creating awesome new features that get released in new versions, so I encourage you to use the latest version of Python available to take advantage of all of them.
I'll show you how you can use
asdf to manage installing versions of Python and to automatically keep track of which version of Python you're using in which project.
It's a little bit more work up front, but it will pay off down the line.
We'll use it here for Python, but you can also use it for Node, PHP or just about anything else.
asdf by following these steps:
- Get the code:
git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.8.0
- Activate it by adding it to your shell
echo ". $HOME/.asdf/asdf.sh" >> ~/.zshrc(or if you use
asdf works with a plugin system, where each plugin handles one specific language.
asdf plugin add python to install the Python plugin.
Now you can install the latest version of Python by running
asdf install python latest.
At the time of writing this, the latest version that gets installed is
3.9.1. Set it as your default Python version globally by running
asdf global python 3.9.1.
You can check this by running
python --version and it should print
cd into your project directory if you aren't already there and run
asdf local python 3.9.1. This will create a file in your project directory named
.tools-versions with the following contents:
This file sets a local version of Python to use for the project. When you are in that directory,
asdf will automatically use that version. If you have a project with multiple languages you can include them all there.
poetry to manage Python packages
Every programming language needs a way to manage dependencies on other projects, and Python is no different.
Python has a few built-in concepts and tools for managing dependencies:
venv (short for virtual environment). You can do everything you need with just the built-in tools for it, but it is a bit complicated and can be confusing to newcomers to Python.
I've used many of the different tools available for managing Python packages (including just Python's built-in tools) and they all work fine. If you're joining an existing project, you should learn how they do it already.
For a better developer experience I personally recommend using
poetry, which provides the most similar experience to
npm and similar modern package managers.
You can install
asdf by running the following commands:
asdf plugin add poetry asdf install poetry latest
poetry introduces a
pyproject.toml file (similar to
package.json) that is automatically updated as you add and remove dependencies.
You should also add the following to a file named
poetry.toml in your project directory:
[virtualenvs] create = true in-project = true
This file configures
poetry to include the python packages (stored in an automatically created virtual environment) in the project directory, similar to how
npm stores project dependencies in
poetry init, which will lead you through entering metadata about your package that creates your project's
pyproject.toml. You can accept the defaults and edit everything in
pyproject.toml later. When it asks if you want to define your dependencies interactively, answer "no" because we will be adding our dependencies later.
If you've followed all the steps in this guide, you should be all set up to start installing your packages and get started on whatever Python project you're working on. Congratulations, you're all set to build with the latest Python tools!
Run the following commands to save the configuration and push to GitHub.
git add -A . git commit -m "set up python project skeleton" git push