diff options
Diffstat (limited to 'docs/cookbook.rst')
| -rw-r--r-- | docs/cookbook.rst | 442 |
1 files changed, 3 insertions, 439 deletions
diff --git a/docs/cookbook.rst b/docs/cookbook.rst index dabd7328f..3da030c50 100644 --- a/docs/cookbook.rst +++ b/docs/cookbook.rst @@ -1,444 +1,8 @@ +:orphan: + ============ Cookbook ============ -.. _`Requirements Files`: - -Requirements Files -****************** - -"Requirements files" are files containing a list of items to be -installed using :ref:`pip install -r <install_--requirement>` like so: - - :: - - pip install -r requirements.txt - - -Details on the format of the files are here: :ref:`Requirements File Format`. - - -There are 3 common use cases for requirements files: - -1. When installing many things, it's easier to use a requirements file, - than specifying them all on the command line. - -2. Requirements files are often used to hold the result from :ref:`pip freeze` - for the purpose of achieving :ref:`repeatable installations <Repeatability>`. - In this case, your requirement file contains a pinned version of everything - that was installed when `pip freeze` was run. - - :: - - pip freeze > requirements.txt - pip install -r requirements.txt - -3. Requirements files can be used to force pip to properly resolve dependencies. - As it is now, pip `doesn't have true dependency resolution - <https://github.com/pypa/pip/issues/988>`_, but instead simply uses the first - specification it finds for a project. E.g if `pkg1` requires `pkg3>=1.0` and - `pkg2` requires `pkg3>=1.0,<=2.0`, and if `pkg1` is resolved first, pip will - only use `pkg3>=1.0`, and could easily end up installing a version of `pkg3` - that conflicts with the needs of `pkg2`. To solve this problem, you can - place `pkg3>=1.0,<=2.0` (i.e. the correct specification) into your - requirements file directly along with the other top level requirements. Like - so: - - :: - - pkg1 - pkg2 - pkg3>=1.0,<=2.0 - - -It's important to be clear that pip determines package dependencies using `install_requires metadata -<http://pythonhosted.org/setuptools/setuptools.html#declaring-dependencies>`_, not by discovering `requirements.txt` -files embedded in projects. - -For a good discussion on the conceptual differences between setup.py and -requirements, see `"setup.py vs requirements.txt" (an article by Donald Stufft) -<https://caremad.io/blog/setup-vs-requirement/>`_ - -See also: - -* :ref:`Requirements File Format` -* :ref:`pip freeze` - - -.. _`Fast & Local Installs`: - -Fast & Local Installs -********************* - -Often, you will want a fast install from local archives, without probing PyPI. - -First, :ref:`download the archives <Downloading Archives>` that fulfill your requirements:: - -$ pip install --download <DIR> -r requirements.txt - -Then, install using :ref:`--find-links <--find-links>` and :ref:`--no-index <--no-index>`:: - -$ pip install --no-index --find-links=[file://]<DIR> -r requirements.txt - - -.. _`Building and Installing Wheels`: - -Building and Installing Wheels -****************************** - -"Wheel" is a built, archive format that can greatly speed installation compared -to building and installing from source archives. For more information, see the -`Wheel docs <http://wheel.readthedocs.org>`_ , -`PEP427 <http://www.python.org/dev/peps/pep-0427>`_, and -`PEP425 <http://www.python.org/dev/peps/pep-0425>`_ - -Pip prefers Wheels where they are available, to disable this use the -:ref:`--no-use-wheel <install_--no-use-wheel>` flag for :ref:`pip install`. - -If no satisfactory wheels are found, pip will default to finding source archives. - -.. note:: - - pip currently disallows platform-specific wheels (except for Windows and Mac) - from being downloaded from PyPI. See :ref:`Should you upload wheels to PyPI`. - - -To install directly from a wheel archive: - -:: - - pip install SomePackage-1.0-py2.py3-none-any.whl - - -pip additionally offers :ref:`pip wheel` as a convenience, to build wheels for -your requirements and dependencies. - -:ref:`pip wheel` requires the `wheel package <https://pypi.python.org/pypi/wheel>`_ to be installed, -which provides the "bdist_wheel" setuptools extension that it uses. - -To build wheels for your requirements and all their dependencies to a local directory: - -:: - - pip install wheel - pip wheel --wheel-dir=/local/wheels -r requirements.txt - - -And *then* to install those requirements just using your local directory of wheels (and not from PyPI): - -:: - - pip install --no-index --find-links=/local/wheels -r requirements.txt - - -.. _`Should you upload wheels to PyPI`: - -Should you upload wheels to PyPI? ---------------------------------- - -The wheel format can eliminate a lot of redundant compilation but, alas, -it's not generally advisable to upload your pre-compiled linux-x86-64 -library binding to pypi. Wheel's tags are only designed to express -the most important *Python*-specific compatibility concerns (Python -version, ABI, and architecture) but do not represent other important -binary compatibility factors such as the OS release, patch level, and -the versions of all the shared library dependencies of any extensions -inside the package. - -Rather than representing all possible compatibility information in the -wheel itself, the wheel design suggests distribution-specific build -services (e.g. a separate index for Fedora Linux binary wheels, compiled -by the index maintainer). This is the same solution taken by Linux -distributions which all re-compile their own packages instead of installing -each other's binary packages. - -Some kinds of precompiled C extension modules can make sense on PyPI, even -for Linux. Good examples include things that can be sensibly statically -linked (a cryptographic hash function; an accelerator module that is -not a binding for an external library); the best example of something -that shouldn't be statically linked is a library like openssl that needs -to be constantly kept up-to-date for security. Regardless of whether a -compatible pre-build package is available, many Linux users will prefer -to always compile their own anyway. - -On Windows and Mac, the case for binary wheels on pypi is stronger due to the -systems being much more uniform than Linux and because it's harder for the end -user to compile their own. Windows and Mac wheels uploaded to pypi should be -compatible with the Python distributions downloaded from http://python.org/. If -you already upload other binary formats to pypi, upload wheels as well. Unlike -the older formats, wheels are compatible with virtual environments. - - -.. _`Downloading Archives`: - -Downloading archives -******************** - -pip allows you to *just* download the source archives for your requirements, without installing anything and without regard to what's already installed. - -:: - -$ pip install --download <DIR> -r requirements.txt - -or, for a specific package:: - -$ pip install --download <DIR> SomePackage - - -Unpacking archives -****************** - -pip allows you to *just* unpack archives to a build directory without installing them to site-packages. This can be useful to troubleshoot install errors or to inspect what is being installed. - -:: - -$ pip install --no-install SomePackage - -If you're in a virtualenv, the build dir is ``<virtualenv path>/build``. Otherwise, it's ``<OS temp dir>/pip-build-<username>`` - -Afterwards, to finish the job of installing unpacked archives, run:: - -$ pip install --no-download SomePackage - - - -Non-recursive upgrades -************************ - -``pip install --upgrade`` is currently written to perform a recursive upgrade. - -E.g. supposing: - -* `SomePackage-1.0` requires `AnotherPackage>=1.0` -* `SomePackage-2.0` requires `AnotherPackage>=1.0` and `OneMorePoject==1.0` -* `SomePackage-1.0` and `AnotherPackage-1.0` are currently installed -* `SomePackage-2.0` and `AnotherPackage-2.0` are the latest versions available on PyPI. - -Running ``pip install --upgrade SomePackage`` would upgrade `SomePackage` *and* `AnotherPackage` -despite `AnotherPackage` already being satisifed. - -If you would like to perform a non-recursive upgrade perform these 2 steps:: - - pip install --upgrade --no-deps SomePackage - pip install SomePackage - -The first line will upgrade `SomePackage`, but not dependencies like `AnotherPackage`. The 2nd line will fill in new dependencies like `OneMorePackage`. - - -.. _`Repeatability`: - -Ensuring Repeatability -********************** - -Three things are required to fully guarantee a repeatable installation using requirements files. - -1. The requirements file was generated by ``pip freeze`` or you're sure it only - contains requirements that specify a specific version. -2. The installation is performed using :ref:`--no-deps <install_--no-deps>`. - This guarantees that only what is explicitly listed in the requirements file is - installed. -3. The installation is performed against an index or find-links location that is - guaranteed to *not* allow archives to be changed and updated without a - version increase. Unfortunately, this is *not* true on PyPI. It is possible - for the same pypi distribution to have a different hash over time. Project - authors are allowed to delete a distribution, and then upload a new one with - the same name and version, but a different hash. See `Issue #1175 - <https://github.com/pypa/pip/issues/1175>`_ for plans to add hash - confirmation to pip, or a new "lock file" notion, but for now, know that the `peep - project <https://pypi.python.org/pypi/peep>`_ offers this feature on top of pip - using requirements file comments. - -User Installs -************* - -With Python 2.6 came the `"user scheme" for installation -<http://docs.python.org/install/index.html#alternate-installation-the-user-scheme>`_, which means that all -Python distributions support an alternative install location that is specific to a user. -The default location for each OS is explained in the python documentation -for the `site.USER_BASE <http://docs.python.org/library/site.html#site.USER_BASE>`_ variable. -This mode of installation can be turned on by -specifying the :ref:`--user <install_--user>` option to ``pip install``. - -Moreover, the "user scheme" can be customized by setting the -``PYTHONUSERBASE`` environment variable, which updates the value of ``site.USER_BASE``. - -To install "SomePackage" into an environment with site.USER_BASE customized to '/myappenv', do the following:: - - export PYTHONUSERBASE=/myappenv - pip install --user SomePackage - - -``pip install --user`` follows four rules: - -#. When globally installed packages are on the python path, and they *conflict* - with the installation requirements, they are ignored, and *not* - uninstalled. -#. When globally installed packages are on the python path, and they *satisfy* - the installation requirements, pip does nothing, and reports that - requirement is satisfied (similar to how global packages can satisfy - requirements when installing packages in a ``--system-site-packages`` - virtualenv). -#. pip will not perform a ``--user`` install in a ``--no-site-packages`` - virtualenv (i.e. the default kind of virtualenv), due to the user site not - being on the python path. The installation would be pointless. -#. In a ``--system-site-packages`` virtualenv, pip will not install a package - that conflicts with a package in the virtualenv site-packages. The --user - installation would lack sys.path precedence and be pointless. - - -To make the rules clearer, here are some examples: - - -From within a ``--no-site-packages`` virtualenv (i.e. the default kind):: - - $ pip install --user SomePackage - Can not perform a '--user' install. User site-packages are not visible in this virtualenv. - - -From within a ``--system-site-packages`` virtualenv where ``SomePackage==0.3`` is already installed in the virtualenv:: - - $ pip install --user SomePackage==0.4 - Will not install to the user site because it will lack sys.path precedence - - -From within a real python, where ``SomePackage`` is *not* installed globally:: - - $ pip install --user SomePackage - [...] - Successfully installed SomePackage - - -From within a real python, where ``SomePackage`` *is* installed globally, but is *not* the latest version:: - - $ pip install --user SomePackage - [...] - Requirement already satisfied (use --upgrade to upgrade) - - $ pip install --user --upgrade SomePackage - [...] - Successfully installed SomePackage - - -From within a real python, where ``SomePackage`` *is* installed globally, and is the latest version:: - - $ pip install --user SomePackage - [...] - Requirement already satisfied (use --upgrade to upgrade) - - $ pip install --user --upgrade SomePackage - [...] - Requirement already up-to-date: SomePackage - - # force the install - $ pip install --user --ignore-installed SomePackage - [...] - Successfully installed SomePackage - - - -Controlling setup_requires -************************** - -Setuptools offers the ``setup_requires`` -`setup() keyword <http://pythonhosted.org/setuptools/setuptools.html#new-and-changed-setup-keywords>`_ -for specifying dependencies that need to be present in order for the `setup.py` script to run. -Internally, Setuptools uses ``easy_install`` to fulfill these dependencies. - -pip has no way to control how these dependencies are located. -None of the :ref:`Package Index Options <Package Index Options>` have an effect. - -The solution is to configure a "system" or "personal" -`Distutils configuration file <http://docs.python.org/2/install/index.html#distutils-configuration-files>`_ -to manage the fulfillment. - -For example, to have the dependency located at an alternate index, add this: - -:: - - [easy_install] - index_url = https://my.index-mirror.com - -To have the dependency located from a local directory and not crawl PyPI, add this: - -:: - - [easy_install] - allow_hosts = '' - find_links = file:///path/to/local/archives - - -Upgrading from distribute to setuptools -*************************************** - -`distribute`_ has now been merged into `setuptools`_, and it is recommended to upgrade to setuptools when possible. - -To upgrade from `distribute`_ to `setuptools`_ using pip, run:: - - pip install --upgrade setuptools - -"ImportError: No module named setuptools" ------------------------------------------ - -Although using the upgrade command above works in isolation, it's possible to get -"ImportError: No module named setuptools" when using pip<1.4 to upgrade a -package that depends on setuptools or distribute. - -e.g. when running a command like this: `pip install --upgrade pyramid` - -Solution -~~~~~~~~ - -To prevent the problem in *new* environments (that aren't broken yet): - -* Option 1: - - * *First* run `pip install -U setuptools`, - * *Then* run the command to upgrade your package (e.g. `pip install --upgrade pyramid`) - -* Option 2: - - * Upgrade pip using :ref:`get-pip <get-pip>` - * *Then* run the command to upgrade your package (e.g. `pip install --upgrade pyramid`) - -To fix the problem once it's occurred, you'll need to manually install the new -setuptools, then rerun the upgrade that failed. - -1. Download `ez_setup.py` (https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py) -2. Run `python ez_setup.py` -3. Then rerun your upgrade (e.g. `pip install --upgrade pyramid`) - - -Cause -~~~~~ - -distribute-0.7.3 is just an empty wrapper that only serves to require the new -setuptools (setuptools>=0.7) so that it will be installed. (If you don't know -yet, the "new setuptools" is a merge of distribute and setuptools back into one -project). - -distribute-0.7.3 does its job well, when the upgrade is done in isolation. -E.g. if you're currently on distribute-0.6.X, then running `pip install -U -setuptools` works fine to upgrade you to setuptools>=0.7. - -The problem occurs when: - -1. you are currently using an older distribute (i.e. 0.6.X) -2. and you try to use pip to upgrade a package that *depends* on setuptools or - distribute. - -As part of the upgrade process, pip builds an install list that ends up -including distribute-0.7.3 and setuptools>=0.7 , but they can end up being -separated by other dependencies in the list, so what can happen is this: - -1. pip uninstalls the existing distribute -2. pip installs distribute-0.7.3 (which has no importable setuptools, that pip - *needs* internally to function) -3. pip moves on to install another dependency (before setuptools>=0.7) and is - unable to proceed without the setuptools package - -Note that pip v1.4 has fixes to prevent this. distribute-0.7.3 (or -setuptools>=0.7) by themselves cannot prevent this kind of problem. +This content is now covered in the :doc:`User Guide <user_guide>` -.. _setuptools: https://pypi.python.org/pypi/setuptools -.. _distribute: https://pypi.python.org/pypi/distribute -.. _PyPI: https://pypi.python.org |