Create a Modern Python Project

Last updated: May 23, 2021

This guide will walk you through everything you need to know to set up a Python project repository with the most up-to-date tools.

You can set up your project in many different ways, but this is an opinionated guide based on my own experience of what works well.

This guide is based on using a Mac, but it should also work for Linux or using Windows Subsystem for Linux although some modifications may be necessary for your particular system.

Create a Git Repository

First create a new empty repo on GitHub or a similar service. You'll want to grab the SSH URL of the new git repo and then clone it and cd into it:

cd ~/projects
git clone git@github.com:<username>/<project-name>.git
cd <project-name>

TIP

The HTTPS URL also works, but you'll need to enter your password every time you push to the repository.

After setting up SSH you never have to enter your password to push to GitHub.

If you haven't set up SSH on GitHub yet, check out their guide.

Add a .gitignore

Add the following contents to a new file named .gitignore:

.venv
__pycache__/
*.pyc

This file tells git to ignore files, directories, or file patterns that are automatically generated and managed by Python, so we don't want to include them in our source code repository.

We will add new files to this .gitignore as needed. If you want a more comprehensive .gitignore, you can use GitHub`s default Python .gitignore.

Set up asdf to manage Python versions

You probably have a version of Python pre-installed onto your computer already, as it is included with many operating systems by default.

TIP

  • run python --version to find the version of Python you have installed.
  • run which python to find the full path of your default Python installation.

I don't recommend using the default Python installation to build your own projects, because when you collaborate on it you'll come across issues from different people having different versions of Python on their computers.

The Python core team is constantly creating awesome new features that get released in new versions, so I encourage you to use the latest version of Python available to take advantage of all of them.

I'll show you how you can use asdf to manage installing versions of Python and to automatically keep track of which version of Python you're using in which project.

It's a little bit more work up front, but it will pay off down the line.

We'll use it here for Python, but you can also use it for Node, PHP or just about anything else.

Install asdf by following these steps:

  1. Get the code: git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.8.0
  2. Activate it by adding it to your shell echo ". $HOME/.asdf/asdf.sh" >> ~/.zshrc (or if you use bash instead of zsh replace ~/.zshrc with ~./bash_profile)

asdf works with a plugin system, where each plugin handles one specific language.

Run asdf plugin add python to install the Python plugin.

Now you can install the latest version of Python by running asdf install python latest.

At the time of writing this, the latest version that gets installed is 3.9.1. Set it as your default Python version globally by running asdf global python 3.9.1.

You can check this by running python --version and it should print Python 3.9.0.

Now cd into your project directory if you aren't already there and run asdf local python 3.9.1. This will create a file in your project directory named .tools-versions with the following contents:

python 3.9.1

This file sets a local version of Python to use for the project. When you are in that directory, asdf will automatically use that version. If you have a project with multiple languages you can include them all there.

Use poetry to manage Python packages

Every programming language needs a way to manage dependencies on other projects, and Python is no different.

Python has a few built-in concepts and tools for managing dependencies: pip and venv (short for virtual environment). You can do everything you need with just the built-in tools for it, but it is a bit complicated and can be confusing to newcomers to Python.

I've used many of the different tools available for managing Python packages (including just Python's built-in tools) and they all work fine. If you're joining an existing project, you should learn how they do it already.

For a better developer experience I personally recommend using poetry, which provides the most similar experience to npm and similar modern package managers.

You can install poetry using asdf by running the following commands:

asdf plugin add poetry
asdf install poetry latest

poetry introduces a pyproject.toml file (similar to package.json) that is automatically updated as you add and remove dependencies.

You should also add the following to a file named poetry.toml in your project directory:

[virtualenvs]
create = true
in-project = true

This file configures poetry to include the python packages (stored in an automatically created virtual environment) in the project directory, similar to how npm stores project dependencies in node_modules.

Now run poetry init, which will lead you through entering metadata about your package that creates your project's pyproject.toml. You can accept the defaults and edit everything in pyproject.toml later. When it asks if you want to define your dependencies interactively, answer "no" because we will be adding our dependencies later.

If you've followed all the steps in this guide, you should be all set up to start installing your packages and get started on whatever Python project you're working on. Congratulations, you're all set to build with the latest Python tools!

Run the following commands to save the configuration and push to GitHub.

git add -A .
git commit -m "set up python project skeleton"
git push