Using virtual environments for your Python project deployments allows you to isolate your project’s dependencies in an environment that is unaffected by the system Python installation(s). This makes it possible to run multiple projects with otherwise conflicting dependencies side-by-side on the same system. It also gives you peace of mind knowing that your code will run on any system with Python installed, regardless of whether that system has any of your required Python libraries installed system-wide.
- A desktop or cloud server running Windows, OS X, Linux or any other operating system supported by Python
- A working Python installation
- (optional) If you plan to install Python libraries that have C extensions (such as NumPy) then you will need to have a C compiler and the Python development headers installed.
What Is a Virtual Environment?
Think of a virtual environment as a complete Python environment that is entirely separate and distinct from any Python environment installed at the operating system (OS) level. When we say “environment” here, we are referring to the Python standard library and header files, any third-party Python packages ( including those with C extensions), and any binary executables related to these. Having a separate environment for each project you are working on makes it much easier to manage dependencies. When you activate a Python virtual environment, your
PATH changes to prioritize the Python binary inside your virtual environment over the one(s) installed to your OS. Using the Python binary in the virtual environment also means you will be using Python libraries installed in the virtual environment rather than the ones installed on your OS.
The good news is that Python version 3.3 or later comes with a virtual environment manager (
venv) built in. If you use an earlier Python version, however, you can install the virtual environment manager (
sudo pip install virtualenv
There is little difference between
virtualenv, and this article will stick to discussing the shared features.
Managing your Python Virtual Environments
Now that you have a virtual environment manager (either
virtualenv), it’s time to start creating some isolated Python environments!
To simplify the discussion, from here on out we will refer to the environment manager as
venv, and you can assume that any commands will work equally well with
virtualenv just by replacing
virtualenv in the command. If you are using
venv your installed
venv command may have a version suffix, such as
pyvenv-3.5. Use that in place of
pyvenv in the examples given.
Now, to get started, open a command prompt to a sandbox directory and enter the command
You should now have a
venv-test directory structure that looks like this:
├── bin (called Scripts in Windows)
├── lib (called Lib in Windows)
├── lib64 -> lib
└── pyvenv.cfg (venv only)
This directory structure is very similar to what you will see in a system-wide Python install, but it stands alone.
Now let’s activate it. At the same command prompt:
# or in Windows:
You may see your command prompt change to append your virtual environment’s name (in our case
(venv-test)) to the beginning of the prompt. To see that the virtual environment is active, run
The output should indicate a path for the
pip binary within the virtual environment directory.
pip in a Virtual Environment
You can easily install packages to your new virtual environment using
pip, just as you would in your “real” Python environment. For example, let’s install the Django web framework in this virtual environment:
pip install django
Notice how administrator privileges were not required? That is because the virtual environment is completely owned by the user who created it. Let’s take a look at our packages now:
pip freeze command shows a list of all Python packages installed and the version that is installed. You should see that Django has been installed. Since we are running
pip within an activated virtual environment, the list includes only packages installed to the virtual environment, not system-wide packges.
You can use
pip in a virtual environment exactly as you would use the system-wide
pip with one crucial distinction: administrator privileges are not required, which is very convenient. You can see all the Python packages for your virtual environment under the
venv-test/lib/python?.?/site-packages/ directory (
venv-test/Lib/site-packages on Windows) which closely mirrors where packages are installed in the global Python directory structure.
For a quick refesher on
pip, check out our article on other useful
More Virtual Environment Options
As with most UNIX commands, you can add
pyvenv in a terminal window to get a complete list of options that can be used with the command. We’ll go over a few of them in more detail here.
You can use this option to clean out an existing virtual environment, so you can start from scratch. This option will remove any packages you’ve installed within the virtual environment. We could use this option on our example virtual environment like so:
pyvenv --clear venv-test
You can run the
--clear option with the virtual environment activated or not. You can verify that the virtual environment is clear by activating it and running
This option makes it so that any Python running in the virtual environment can access both packages installed to the virtual environment as well as packages installed system-wide. It can be useful if you have certain packages installed system-wide that you often use and want to use in all the projects you work on and the version of the package isn’t critical. We could use this option to create a new virtual environment like so:
pyvenv --system-site-packages venv-system-test
Now if you activate the new virtual environment and run
pip freeze, you will see all your system-wide packages as well as the packages installed only in the virtual environment.
pip is a wonderful tool that most of us Python developers rely on completely, there may be legitimate situations where one might not want to include
pip in a virtual environment. For example, if storage space were at a premium and the virtual environment is for a production deployment where you don’t need
pip, it could be useful to create a virtual environment without it. You can create a virtual environment without
pip like so:
pyvenv --without-pip venv-nopip-test
Note: the option is different in
virtualenv –no-pip venv-nopip-test
Deactivation of a Virtual Environment
When you are finished working in a virtual environment, you must deactivate it to return to your system’s global environment. Otherwise, any commands you make in the same terminal window will be in that virtual environment.
Alternatively, you could simply close the terminal window.
While virtual environments are an incredible tool in their own right, the Python community has come up with a few ways to make virtual environments even better to work with. We’ll discuss a few of them in more detail below.
One of the most popular tools used with virtual environments is virtualenvwrapper. You can install it by running
sudo pip install virtualenvwrapper
# or, on Windows:
pip install virtualenvwrapper-win
virtualenvwrapper organizes all virtual environments in one place, provides convenient wrappers for managing virtual environments, and gives you tab completion for commands that take a virtual environment as an argument.
Another popular tool that goes beyond the basics of virtual environments is
autoenv, another hit from the prolific Kenneth Reitz, creator of python-requests.
sudo pip install autoenv
autoenv, you can automatically activate virtual environments when you use
cd to enter the directory. This addition really comes in handy for local development.
If you work on Python packages that support several different Python versions or different versions of a major dependency, like Django,
tox can help manage the prospect of testing these different environments.
sudo pip install tox
When you specify all the different environments you want to support in a
tox.ini file, you can then run the
tox command to run your unit test suite in all the environments specified.
tox automatically creates virtual environments for all the different Python versions specified, installs required packages, and runs the test suite.
I hope by now you are convinced of the usefulness and necessity of Python virtual environments, in both development and production environments. Try them out, along with the additional tools provided by the community, and find the workflow that works best for you. And if you have an idea for improvement, share it. You’ll find that the Python community is very receptive to hearing new ideas.
Atlantic.Net provides leading cloud, dedicated and managed hosting services. Learn more about our reliable HIPAA-compliant cloud hosting solutions.