In this article, we’ll show how to use virtual environments to create and manage separate environments for your Python projects, each using different versions of Python for execution, as well as how Python dependencies are stored and resolved.

在本文中,我们将展示如何使用虚拟环境为Python项目创建和管理单独的环境,每个环境都使用不同版本的Python来执行,以及如何存储和解决Python依赖项。

为什么需要虚拟环境? (Why the need for virtual environments?)

Python, like most other modern programming languages, has its own unique way of downloading, storing, and resolving packages (or modules). While this has its advantages, there were some interesting decisions made about package storage and resolution, which has lead to some problems – namely how and where packages are stored.

与大多数其他现代编程语言一样,Python也有其独特的下载,存储和解析包(或模块 )的方式。 尽管这样做有其优点,但在软件包的存储和解析方面做出了一些有趣的决定,这导致了一些问题,即如何以及在何处存储软件包。

There are a few different locations where these packages can be installed on your system. For example, most system packages are stored in a child directory of the path stored in sys.prefix.

这些软件包可以在几个不同的位置安装到您的系统上。 例如,大多数系统软件包都存储在sys.prefix中存储的路径的子目录中 。

On Mac OS X, you can easily find where sys.prefix points to using the Python shell:

在Mac OS X上,您可以使用Python shell轻松找到sys.prefix指向的位置:

1
1
2
2
3
3

More relevant to the topic of this article, third party packages installed using easy_install or pip are typically placed in one of the directories pointed to by site.getsitepackages:

与本文主题更相关,使用easy_install或pip安装的第三方软件包通常放置在site.getsitepackages指向的目录之一中:

1
1
2
2
3
3
4
4
5
5
6
6

So, why do all of these little details matter?

那么,为什么所有这些小细节都很重要?

It’s important to know this because, by default, every project on your system will use these same directories to store and retrieve site packages (3rd party libraries). At first glance this may not seem like a big deal, and it isn’t really for system packages – packages part of the standard Python library – but it does matter for site packages.

了解这一点很重要,因为默认情况下,系统上的每个项目都将使用这些相同的目录来存储和检索站点包(第三方库)。 乍一看,这似乎没什么大不了的,对于系统软件包(标准Python库的一部分软件包)来说,这并不是真的,但对于站点软件包确实很重要。

Consider the following scenario where you have two projects – ProjectA and ProjectB, both of which have a dependency on the same library, ProjectC. The problem becomes apparent when we start requiring different versions of ProjectC. Maybe ProjectA needs v1.0.0, while ProjectB requires the newer v2.0.0, for example.

考虑以下场景,其中有两个项目– ProjectA和ProjectB,这两个项目都依赖于同一库ProjectC。 当我们开始需要不同版本的ProjectC时,问题变得很明显。 例如,也许ProjectA需要v1.0.0,而ProjectB需要较新的v2.0.0。

This is a real problem for Python since it can’t differentiate between versions in the “site-packages” directory. So both v1.0.0 and v2.0.0 would reside in the same directory with the same name:

对于Python来说,这是一个真正的问题,因为它无法区分“ site-packages”目录中的版本。 因此,v1.0.0和v2.0.0都将位于相同名称的同一目录中:

1
1

And since projects are stored according to just their name there is no differentiation between versions. Thus, both projects, ProjectA and ProjectB, would be required to use the same version, which is unacceptable in many cases.

并且由于仅根据名称存储项目,因此版本之间没有区别。 因此,两个项目ProjectA和ProjectB都必须使用相同的版本,这在许多情况下是不可接受的。

This is where the concept of virtual environments (and the virtualenv/pyvenv tools) comes into play…

这就是虚拟环境(和virtualenv / pyvenv工具)概念发挥作用的地方…

什么是虚拟环境? (What is a virtual environment?)

At its core, the main purpose of Python virtual environments is to create an isolated environment for Python projects. This means that each project can have its own dependencies, regardless of what dependencies every other project has.

从本质上讲,Python虚拟环境的主要目的是为Python项目创建一个隔离的环境。 这意味着每个项目都可以具有自己的依赖关系,而不管每个其他项目都具有什么依赖关系。

So, in our little example above, we’d just need to create a separate virtual environment for both ProjectA and ProjectB and we’d be good to go. Each environment, in turn, would be able to depend on whatever version of ProjectC they choose, independent of the other.

因此,在上面的小示例中,我们只需要为ProjectA和ProjectB创建一个单独的虚拟环境,就可以了。 反过来,每个环境都将能够依赖于他们选择的任何版本的ProjectC,彼此独立。

The great thing about this is that there are no limits to the number of environments you can have since they’re just directories containing a few scripts. Plus, they’re easily created using the virtualenv or pyenv command line tools.

这样做的好处是,您可以拥有的环境数量没有限制,因为它们只是包含一些脚本的目录。 另外,可以使用virtualenvpyenv命令行工具轻松创建它们。

使用虚拟环境 (Using virtual environments)

To get started, if you’re not using Python 3, you’ll want to install the virtualenv tool with pip:

首先,如果您不使用Python 3,则需要使用pip安装virtualenv工具:

1
1

If you are using Python 3 then you should already have pyvenv installed. This is a file that uses the venv library underneath.

如果您使用的是Python 3,则应该已经安装了pyvenv 。 这是一个使用下面的venv库的文件。

From here on out we’ll assume you’re using the newer pyvenv tool, since there are few differences between it and virtualenv with regard to the actual commands. In reality, though, they are very different tools.

从现在开始,我们假设您正在使用较新的pyvenv工具,因为在实际命令方面,它与virtualenv之间几乎没有区别。 但实际上,它们是非常不同的工具。

Start by making a new directory to work with:

首先创建一个新目录以供使用:

1
1

Create a new virtual environment inside the directory:

在目录内创建一个新的虚拟环境:

1
1

By default this will NOT include any of your existing site packages.

默认情况下,这将不包括任何现有站点包。

In the above example, this command creates a directory called “env”, which contains a directory structure similar to this:

在上面的示例中,此命令创建一个名为“ env”的目录,其中包含类似于以下内容的目录结构:

1
1
2
2
3
3
4
4
5
5
6
6
7
7
8
8
9
9
10
10
11
11
12
12
13
13
14
14
15
15
16
16
17
17

What does each folder contain?

每个文件夹包含什么?

  • “bin” – files that interact with the virtual environment
  • “include” – C headers that compile the Python packages
  • “lib” – a copy of the Python version along with a “site-packages” folder where each dependency is installed
  • “ bin” –与虚拟环境交互的文件
  • “ include” –编译Python包的C头文件
  • “ lib” – Python版本的副本,以及安装了每个依赖项的“ site-packages”文件夹

Further, there are copies of, or symlinks to, a few different Python tools and to the Python executables themselves. These files are used to ensure all Python code and commands are executed within the context of the current environment, which is how the isolation from the global environment is achieved. We’ll explain this in more detail in the next section.

此外,还有一些不同的Python工具和Python可执行文件本身的副本或符号链接 。 这些文件用于确保所有Python代码和命令均在当前环境的上下文中执行,这是实现与全局环境隔离的方式。 我们将在下一节中更详细地说明这一点。

More interestingly are the activate scripts in the “bin” directory. These scripts are used to set up your shell to use the environment’s Python executable and its site-packages by default.

更有趣的是“ bin”目录中的激活脚本。 这些脚本用于将您的外壳设置为默认使用环境的Python可执行文件及其站点包。

In order to use this environment’s packages/resources in isolation, you need to “activate” it. To do this, just run:

为了隔离使用此环境的程序包/资源,您需要“激活”它。 为此,只需运行:

1
1
2
2

Notice how your prompt is now prefixed with the name of your environment (env, in our case). This is the indicator that env is currently active, which means the python executable will only use this environment’s packages and settings.

请注意,您的提示现在是如何以环境名称为前缀的(在本例中为env )。 这表明env当前处于活动状态,这意味着python可执行文件将仅使用此环境的程序包和设置。

To show the package isolation in action, we can use the bcrypt module as an example. Let’s say we have bcrypt installed system-wide, but not in our virtual environment.

为了显示实际的程序包隔离,我们可以使用bcrypt模块作为示例。 假设我们已经在系统范围内安装了bcrypt ,但没有在虚拟环境中安装。

Before we test this, we need to go back to the “system” context by executing deactivate:

在测试之前,我们需要通过执行deactivate返回“ system”上下文:

1
1
2
2

Now your shell session is back to normal, and the python command refers to the global Python install. Remember to do this whenever you’re done using a specific virtual environment.

现在您的shell会话恢复正常,并且python命令引用了全局Python安装。 使用特定的虚拟环境时,请记住要这样做。

Now, install bcyrpt and use it to hash a password:

现在,安装bcyrpt并将其用于哈希密码:

1
1
2
2
3
3

What happens if we try the same command when the virtual environment is activated?

如果在激活虚拟环境时尝试相同的命令会发生什么?

1
1
2
2
3
3
4
4
5
5

As you can see, the behavior of the python -c "import bcrypt..." command changes after the source env/bin/activate call.

如您所见,在source env/bin/activate调用之后, python -c "import bcrypt..."命令的行为发生了变化。

In one instance we have bcrypt available to us, and in the next we don’t. This is the kind of separation we’re looking to achieve with virtual environments, which is now easily achieved.

在一种情况下,我们可以使用bcrypt ,而在bcrypt情况下,则没有。 这是我们希望通过虚拟环境实现的分离,这种分离现在很容易实现。

虚拟环境如何工作? (How does a virtual environment work?)

So what exactly does it mean to “activate” an environment? Knowing what’s going on under the hood can be pretty important for a developer, especially when you need to understand execution environments, dependency resolution, etc.

那么“激活”环境到底意味着什么? 对于开发人员而言,了解幕后的情况可能非常重要,尤其是在您需要了解执行环境,依赖关系解决方案等时。

To explain how this works, let’s first check out the locations of the different python executables. With the environment “deactivated”, run:

为了解释它是如何工作的,让我们首先检查不同python可执行文件的位置。 在“停用”环境的情况下,运行:

1
1
2
2

Now activate it and run the command again:

现在激活它并再次运行命令:

1
1
2
2
3
3

After activating the environment we’re now getting a different path for the python executable because in an active environment the $PATH environment variable is slightly modified.

激活环境后,我们现在获得了python可执行文件的其他路径,因为在活动环境中,对$PATH环境变量进行了一些修改。

Notice the difference between the first path in $PATH before and after the activation:

注意激活前后$PATH第一个路径之间的区别:

1
1
2
2
3
3
4
4
5
5
6
6

In the latter example, our virtual environment’s “bin” directory is now at the beginning of the path. That means it’s the first directory searched when running an executable on the command line. Thus, the shell uses our virtual environment’s instance of Python instead of the system-wide version.

在后一个示例中,虚拟环境的“ bin”目录现在位于路径的开头。 这意味着它是在命令行上运行可执行文件时搜索的第一个目录。 因此,shell使用我们虚拟环境的Python实例而不是系统范围的版本。

Other packages that bundle Python, like Anaconda, also tend to manipulate your path when you activate them. Just be aware of this in case you run into problems with your other environments. This can become a problem if you start activating multiple environments at once.

其他捆绑Python的软件包(例如Anaconda)在激活它们时也倾向于操纵路径。 请注意这一点,以防您遇到其他环境的问题。 如果您立即开始激活多个环境,则可能会成为问题。

This begs the questions:

这引出了以下问题:

  • What’s the difference between these two executables anyway?
  • How is the virtual environment’s Python executable able to use something other than the system’s site-packages?
  • 无论如何,这两个可执行文件有什么区别?
  • 虚拟环境的Python可执行文件如何能够使用系统站点包以外的其他内容?

This can be explained by how Python starts up and where it is located on the system. There actually isn’t any difference between these two Python executables. It’s their directory locations that matter.

这可以通过Python如何启动以及它在系统上的位置来解释。 实际上,这两个Python可执行文件之间没有任何区别。 重要的是他们的目录位置。

When Python is starting up, it looks at the path of its binary (which, in a virtual environment, is actually just a copy of, or symlink to, your system’s Python binary). It then sets the location of sys.prefix and sys.exec_prefix based on this location, omitting the “bin” portion of the path.

当Python启动时,它会查看其二进制文件的路径(在虚拟环境中,它实际上只是系统Python二进制文件的副本或符号链接)。 然后, sys.exec_prefix基于该位置设置sys.prefixsys.exec_prefix的位置,并忽略路径的“ bin”部分。

The path located in sys.prefix is then used for locating the “site-packages” directory by searching the relative path lib/pythonX.X/site-packages/, where X.X is the version of Python you’re using.

然后,通过搜索相对路径lib/pythonX.X/site-packages/ ,将sys.prefix的路径用于查找“ site-packages”目录,其中XX是您使用的Python版本。

In our example, the binary is located at /Users/michaelherman/python-virtual-environments/env/bin, which means sys.prefix would be /Users/michaelherman/python-virtual-environments/env, and therefore the “site-packages” directory used would be /Users/michaelherman/python-virtual-environments/env/bin/lib/pythonX.X/site-packages. Finally, this path is stored in the sys.path array, which contains all of the locations that a package can reside.

在我们的示例中,二进制文件位于/Users/michaelherman/python-virtual-environments/env/bin ,这意味着sys.prefix/Users/michaelherman/python-virtual-environments/env sys.prefix /Users/michaelherman/python-virtual-environments/env ,因此“ site-包”目录为/Users/michaelherman/python-virtual-environments/env/bin/lib/pythonX.X/site-packages 。 最后,此路径存储在sys.path数组中,该数组包含程序包可以驻留的所有位置。

使用virtualenvwrapper管理虚拟环境 (Managing virtual environments with virtualenvwrapper)

While virtual environments certainly solve some big problems with package management, they’re not perfect. After creating a few environments, you’ll start to see that they create some problems of their own, most of which revolve around managing the environments themselves. To help with this, the virtualenvwrapper tool was created, which is just some wrapper scripts around the main virtualenv tool.

尽管虚拟环境肯定可以解决程序包管理中的一些大问题,但它们并不是完美的。 在创建了几个环境之后,您将开始看到它们创建了自己的一些问题,其中大部分围绕着管理环境本身。 为了解决这个问题,创建了virtualenvwrapper工具,它只是围绕主要virtualenv工具的一些包装脚本。

A few of the more useful features of virtualenvwrapper are that it:

virtualenvwrapper的一些更有用的功能是:

  • Organizes all of your virtual environments in one location;
  • Provides methods to help you easily create, delete, and copy environments; and,
  • Provides a single command to switch between environments
  • 将所有虚拟环境组织在一个位置;
  • 提供的方法可帮助您轻松创建,删除和复制环境; 和,
  • 提供单个命令在环境之间切换

While some of these features may seem small or insignificant, you’ll soon learn that they’re important tools to add to your workflow.

尽管其中一些功能看起来很小或微不足道,但您很快就会发现它们是添加到工作流程中的重要工具。

To get started, you can download the wrapper with pip:

首先,您可以使用pip下载包装器:

1
1

For Windows, you should use virtualenvwrapper-win instead.

对于Windows,您应该改用virtualenvwrapper-win 。

Once installed, we’ll need to activate its shell functions, which can be done by running source on the installed virtualenvwrapper.sh script. When you first install it with pip, the output of the installation will tell you the exact location of virtualenvwrapper.sh. Or you can simply run:

安装完成后,我们需要激活其外壳功能,这可以通过在已安装的virtualenvwrapper.sh脚本上运行source来完成。 首次使用pip安装时,安装的输出将告诉您virtualenvwrapper.sh的确切位置。 或者您可以简单地运行:

1
1
2
2

Using that path, add the following three lines to your shell’s startup file. If you’re using the Bash shell, you would place these lines in either the ~/.bashrc file or ~/.profile file. For other shells, like zsh, csh, or fish, you would need to use the startup files specific to that shell. All that matters is these commands are executed when you log in or open a new shell.

使用该路径,将以下三行添加到Shell的启动文件中。 如果您使用的是Bash shell,则可以将这些行放在~/.bashrc文件或~/.profile文件中。 对于其他shell,例如zsh,csh或fish,则需要使用特定于该shell的启动文件。 重要的是,这些命令在您登录或打开新的外壳程序时执行。

1
1
2
2
3
3

It’s not required to define the WORKON_HOME and PROJECT_HOME environment variables. virtualenvwrapper has default values for those, but you can override them by defining values.

不需要定义WORKON_HOMEPROJECT_HOME环境变量。 virtualenvwrapper具有这些默认值,但是您可以通过定义值来覆盖它们。

Finally, reload the startup file:

最后,重新加载启动文件:

1
1

There should now be a directory located at $WORKON_HOME that contains all of the virtualenvwrapper data/files:

现在应该在$WORKON_HOME上有一个目录,其中包含所有virtualenvwrapper数据/文件:

1
1
2
2

You’ll also now have the shell commands available to you to help you manage the environments. Here are just a few available:

现在,您还将可以使用Shell命令来帮助您管理环境。 这里只有一些可用的:

  • workon
  • deactivate
  • mkvirtualenv
  • cdvirtualenv
  • rmvirtualenv
  • 从事于
  • 停用
  • 虚拟环境
  • 光盘虚拟环境
  • 虚拟环境

For more info on commands, installation, and configuring virtualenvwrapper, check out their documentation.

有关命令,安装和配置virtualenvwrapper的更多信息,请查看其文档 。

Now anytime you want to start a new project, all you have to do is:

现在,无论何时您要开始一个新项目,您所需要做的就是:

1
1
2
2

This will create and activate a new environment in the directory located at $WORKON_HOME, where all virtualenvwrapper environments are stored.

这将在$WORKON_HOME的目录中创建并激活一个新环境,该目录中存储了所有virtualenvwrapper环境。

To stop using that environment, you just need to deactivate it like before:

要停止使用该环境,您只需像以前一样停用它:

1
1
2
2

If you have many environments to choose from, you can list them all with the workon function:

如果您有很多环境可供选择,则可以使用workon函数将它们全部列出:

1
1
2
2
3
3
4
4

And finally, to activate:

最后,激活:

1
1
2
2

Now you don’t have to remember where you installed your environments, you can easily delete or copy them as you wish, and your project directory is less cluttered!

现在,您不必记住安装环境的位置,可以根据需要轻松地删除或复制它们,并且项目目录也不会太混乱!

使用不同版本的python (Using different versions of python)

Unlike the old virtualenv tool, pyvenv doesn’t support creating environments with arbitrary versions of Python, which means you’re stuck using the default Python 3 installation for all of the environments you create. While you can upgrade an environment to the latest system version of Python (via the --upgrade option) if it changes, you still can’t actually specify a particular version.

与旧的virtualenv工具不同, pyvenv不支持使用任意版本的Python创建环境,这意味着您在创建的所有环境中都无法使用默认的Python 3安装。 如果可以更改环境,可以将环境升级到最新的Python系统版本(通过--upgrade选项),但实际上仍然不能指定特定版本。

There are quite a few ways to install Python, but few of them are easy enough or flexible enough to frequently uninstall and re-install different versions of the binary.

有很多安装Python的方法 ,但是很少有足够容易或灵活的方法来频繁卸载和重新安装不同版本的二进制文件。

This is where pyenv comes in to play.

这就是pyenv发挥作用的地方。

Despite the similarity in names (pyvenv vs pyenv), pyenv is different in that its focus is to help you switch between Python versions on a system-level as well as a project-level. So, while pyvenv’s purpose is to separate out modules, pyenv’s purpose is to separate Python versions.

尽管名称( pyvenvpyenv )相似,但pyenv的不同之处在于其重点是帮助您在系统级和项目级的Python版本之间切换。 因此,尽管pyvenv的目的是分离模块,而pyenv的目的是分离Python版本。

You can start by installing pyenv with either Homebrew (on OS X), or with the pyenv-installer project:

您可以使用Homebrew (在OS X上)或pyenv-installer项目安装pyenv来开始:

Homebrew

家酿

1
1

pyenv-installer

pyenv安装程序

1
1

Unfortunately, pyenv does not support Windows. A few alternatives to try are pywin and anyenv.

不幸的是, pyenv不支持Windows。 pywin和anyenv是一些尝试的替代方法。

Once you have pyenv on your system, here are a few of the basic commands you’re probably interested in:

在系统上安装了pyenv ,您可能pyenv以下一些基本命令感兴趣:

1
1
2
2
3
3

In these few lines we install the 3.5.0 version of Python, ask pyenv to show us all of the versions available to us, and then execute the python -V command using the pyenv-specified version.

在这几行中,我们安装了Python的3.5.0版本,请pyenv向我们展示所有可用的版本,然后使用pyenv版本执行python -V命令。

To give you even more control, you can then use any of the available versions for either “global” use or “local” use. Using pyenv with the local command sets the Python version for a specific project or directory by storing the version number in a local .python-version file. We can set the “local” version like this:

为了提供更多控制权,您可以将任何可用版本用于“全局”使用或“本地”使用。 通过将pyenvlocal命令一起使用,可以通过将版本号存储在本地.python-version文件中来设置特定项目或目录的Python版本。 我们可以这样设置“本地”版本:

1
1

This creates the .python-version file in our current directory, as you can see here:

这将在当前目录中创建.python-version文件,如下所示:

1
1
2
2
3
3
4
4
5
5
6
6

This file only contains the contents “2.7.11”. Now when you execute a script using pyenv, it’ll load this file and use the specified version, assuming it’s valid and exists on your system.

该文件仅包含内容“ 2.7.11”。 现在,当您使用pyenv执行脚本时,它将加载该文件并使用指定的版本,并假定该版本有效并且已存在于系统中。

Moving on with our example, let’s say we have a simple script called main.py in our project directory that looks like this:

继续我们的示例,假设我们在项目目录中有一个名为main.py的简单脚本,如下所示:

1
1
2
2

All it does is print out the version number of the Python executable being used. Using pyenv and the exec command, we can run the script with any of the different versions of Python we have installed.

它所做的只是打印出正在使用的Python可执行文件的版本号。 使用pyenvexec命令,我们可以使用已安装的任何不同版本的Python运行脚本。

1
1
2
2
3
3
4
4
5
5
6
6
7
7
8
8

Notice how pyenv exec python main.py uses our “global” Python version by default, but then it uses the “local” version after one is set for the current directory.

请注意,默认情况下pyenv exec python main.py如何使用我们的“全局” Python版本,但是在为当前目录设置一个版本之后,然后使用“本地”版本。

This can be very powerful for developers who have lots of projects with varying version requirements. Not only can you easily change the default version for all projects (via global), but you can also override it to specify special cases.

对于拥有许多具有不同版本要求的项目的开发人员来说,这可能非常强大。 您不仅可以轻松地更改所有项目的默认版本(通过global ),还可以覆盖它以指定特殊情况。

结论 (Conclusion)

In this article you learned about how Python dependencies are stored and resolved, and how to use different community tools to help get around various packaging and versioning problems.

在本文中,您了解了如何存储和解决Python依赖项,以及如何使用不同的社区工具来解决各种打包和版本控制问题。

As you can see, thanks to the huge Python community there are quite a few tools at your disposal to help with these common problems. As you progress as a developer, be sure to take time to learn how to use these tools to your advantage. You may even find unintended uses for them, or learn to apply similar concepts to other languages you use.

如您所见,由于庞大的Python社区,您可以使用许多工具来解决这些常见问题。 随着您成为开发人员的进步,一定要花时间学习如何利用这些工具来发挥自己的优势。 您甚至可能会发现它们的意想不到的用途,或者学习将类似的概念应用于您使用的其他语言。

翻译自: https://www.pybloggers/2016/03/python-virtual-environments-a-primer/

更多推荐

Python虚拟环境–入门