diff options
Diffstat (limited to 'eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO')
8 files changed, 0 insertions, 8326 deletions
diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/PKG-INFO b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/PKG-INFO deleted file mode 100644 index ba54ddc..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/PKG-INFO +++ /dev/null @@ -1,8235 +0,0 @@ -Metadata-Version: 1.0 -Name: zc.buildout -Version: 1.5.2 -Summary: System for managing development buildouts -Home-page: http://pypi.python.org/pypi/zc.buildout -Author: Jim Fulton -Author-email: jim@zope.com -License: ZPL 2.1 -Description: ******** - Buildout - ******** - - .. contents:: - - The Buildout project provides support for creating applications, - especially Python applications. It provides tools for assembling - applications from multiple parts, Python or otherwise. An application - may actually contain multiple programs, processes, and configuration - settings. - - The word "buildout" refers to a description of a set of parts and the - software to create and assemble them. It is often used informally to - refer to an installed system based on a buildout definition. For - example, if we are creating an application named "Foo", then "the Foo - buildout" is the collection of configuration and application-specific - software that allows an instance of the application to be created. We - may refer to such an instance of the application informally as "a Foo - buildout". - - To get a feel for some of the things you might use buildouts for, see - the `Buildout examples`_. - - To lean more about using buildouts, see `Detailed Documentation`_. - - To see screencasts, talks, useful links and more documentation, visit - the `Buildout website <http://www.buildout.org>`_. - - Recipes - ******* - - Existing recipes include: - - `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_ - The egg recipe installes one or more eggs, with their - dependencies. It installs their console-script entry points with - the needed eggs included in their paths. It is suitable for use with - a "clean" Python: one without packages installed in site-packages. - - `z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_ - Like zc.recipe.egg, this recipe builds interpreter scripts and entry - point scripts based on eggs. It can be used with a Python that has - packages installed in site-packages, such as a system Python. The - interpreter also has more features than the one offered by - zc.recipe.egg. - - `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_ - The testrunner egg creates a test runner script for one or - more eggs. - - `zc.recipe.zope3checkout <http://pypi.python.org/pypi/zc.recipe.zope3checkout>`_ - The zope3checkout recipe installs a Zope 3 checkout into a - buildout. - - `zc.recipe.zope3instance <http://pypi.python.org/pypi/zc.recipe.zope3instance>`_ - The zope3instance recipe sets up a Zope 3 instance. - - `zc.recipe.filestorage <http://pypi.python.org/pypi/zc.recipe.filestorage>`_ - The filestorage recipe sets up a ZODB file storage for use in a - Zope 3 instance created by the zope3instance recipe. - - Buildout examples - ***************** - - Here are a few examples of what you can do with buildouts. We'll - present these as a set of use cases. - - Try out an egg - ============== - - Sometimes you want to try an egg (or eggs) that someone has released. - You'd like to get a Python interpreter that lets you try things - interactively or run sample scripts without having to do path - manipulations. If you can and don't mind modifying your Python - installation, you could use easy_install, otherwise, you could create - a directory somewhere and create a buildout.cfg file in that directory - containing:: - - [buildout] - parts = mypython - - [mypython] - recipe = zc.recipe.egg - interpreter = mypython - eggs = theegg - - where theegg is the name of the egg you want to try out. - - Run buildout in this directory. It will create a bin subdirectory - that includes a mypython script. If you run mypython without any - arguments you'll get an interactive interpreter with the egg in the - path. If you run it with a script and script arguments, the script - will run with the egg in its path. Of course, you can specify as many - eggs as you want in the eggs option. - - If the egg provides any scripts (console_scripts entry points), those - will be installed in your bin directory too. - - Work on a package - ================= - - I often work on packages that are managed separately. They don't have - scripts to be installed, but I want to be able to run their tests - using the `zope.testing test runner - <http://www.python.org/pypi/zope.testing>`_. In this kind of - application, the program to be installed is the test runner. A good - example of this is `zc.ngi <http://svn.zope.org/zc.ngi/trunk/>`_. - - Here I have a subversion project for the zc.ngi package. The software - is in the src directory. The configuration file is very simple:: - - [buildout] - develop = . - parts = test - - [test] - recipe = zc.recipe.testrunner - eggs = zc.ngi - - I use the develop option to create a develop egg based on the current - directory. I request a test script named "test" using the - zc.recipe.testrunner recipe. In the section for the test script, I - specify that I want to run the tests in the zc.ngi package. - - When I check out this project into a new sandbox, I run bootstrap.py - to get setuptools and zc.buildout and to create bin/buildout. I run - bin/buildout, which installs the test script, bin/test, which I can - then use to run the tests. - - This is probably the most common type of buildout. - - If I need to run a previous version of zc.buildout, I use the - `--version` option of the bootstrap.py script:: - - $ python bootstrap.py --version 1.1.3 - - The `zc.buildout project <http://svn.zope.org/zc.buildout/trunk>`_ - is a slightly more complex example of this type of buildout. - - Install egg-based scripts - ========================= - - A variation of the `Try out an egg`_ use case is to install scripts - into your ~/bin directory (on Unix, of course). My ~/bin directory is - a buildout with a configuration file that looks like:: - - - [buildout] - parts = foo bar - bin-directory = . - - [foo] - ... - - where foo and bar are packages with scripts that I want available. As - I need new scripts, I can add additional sections. The bin-directory - option specified that scripts should be installed into the current - directory. - - Multi-program multi-machine systems - =================================== - - Using an older prototype version of the buildout, we've build a number - of systems involving multiple programs, databases, and machines. One - typical example consists of: - - - Multiple Zope instances - - - Multiple ZEO servers - - - An LDAP server - - - Cache-invalidation and Mail delivery servers - - - Dozens of add-on packages - - - Multiple test runners - - - Multiple deployment modes, including dev, stage, and prod, - with prod deployment over multiple servers - - Parts installed include: - - - Application software installs, including Zope, ZEO and LDAP - software - - - Add-on packages - - - Bundles of configuration that define Zope, ZEO and LDAP instances - - - Utility scripts such as test runners, server-control - scripts, cron jobs. - - Questions and Bug Reporting - *************************** - - Please send questions and comments to the - `distutils SIG mailing list <mailto://distutils-sig@python.org>`_. - - Report bugs using the `zc.buildout Launchpad Bug Tracker - <https://launchpad.net/zc.buildout/+bugs>`_. - - System Python and zc.buildout 1.5 - ********************************* - - The 1.5 line of zc.buildout introduced a number of changes. - - Problems - ======== - - As usual, please send questions and comments to the `distutils SIG - mailing list <mailto://distutils-sig@python.org>`_. Report bugs using - the `zc.buildout Launchpad Bug Tracker - <https://launchpad.net/zc.buildout/+bugs>`_. - - If problems are keeping you from your work, here's an easy way to - revert to the old code temporarily: switch to a custom "emergency" - bootstrap script, available from - http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py . - - This customized script will select zc.buildout 1.4.4 by default. - zc.buildout 1.4.4 will not upgrade itself unless you explicitly specify - a new version. It will also prefer older versions of zc.recipe.egg and - some other common recipes. If you have trouble with other recipes, - consider using a standard buildout "versions" section to specify older - versions of these, as described in the Buildout documentation - (http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used). - - Working with a System Python - ============================ - - While there are a number of new features available in zc.buildout 1.5, - the biggest is that Buildout itself supports usage with a system Python. - This can work if you follow a couple of simple rules. - - 1. Use the new bootstrap.py (available from - svn://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py). - - 2. Use buildout recipes that have been upgraded to work with zc.buildout 1.5 - and higher. Specifically, they should use - ``zc.buildout.easy_install.sitepackage_safe_scripts`` to generate - their scripts, if any, rather than ``zc.buildout.easy_install.scripts``. - See the `Recipes That Support a System Python`_ section below for more - details on recipes that are available as of this writing, and - `Updating Recipes to Support a System Python`_ for instructions on - how to update a recipe. Note that you should generally only need to - update recipes that generate scripts. - - You can then use ``include-site-packages = false`` and - ``exec-sitecustomize = false`` buildout options to eliminate access to - your Python's site packages and not execute its sitecustomize file, if - it exists, respectively. - - Alternately, you can use the ``allowed-eggs-from-site-packages`` buildout - option as a glob-aware whitelist of eggs that may come from site-packages. - This value defaults to "*", accepting all eggs. - - It's important to note that recipes not upgraded for zc.buildout 1.5.0 - should continue to work--just without internal support for a system Python. - - Using a system Python is inherently fragile. Using a clean, - freshly-installed Python without customization in site-packages is more - robust and repeatable. See some of the regression tests added to the - 1.5.0 line for the kinds of issues that you can encounter with a system - Python, and see - http://pypi.python.org/pypi/z3c.recipe.scripts#including-site-packages-and-sitecustomize - for more discussion. - - However, using a system Python can be very convenient, and the - zc.buildout code for this feature has been tested by many users already. - Moreover, it has automated tests to exercise the problems that have been - encountered and fixed. Many people rely on it. - - Recipes That Support a System Python - ==================================== - - zc.recipe.egg continues to generate old-style scripts that are not safe - for use with a system Python. This was done for backwards - compatibility, because it is integral to so many buildouts and used as a - dependency of so many other recipes. - - If you want to generate new-style scripts that do support system Python - usage, use z3c.recipe.scripts instead - (http://pypi.python.org/pypi/z3c.recipe.scripts). z3c.recipe.scripts has - the same script and interpreter generation options as zc.recipe.egg, - plus a few more for the new features mentioned above. In the simplest - case, you should be able to simply change ``recipe = zc.recipe.egg`` to - ``recipe = z3c.recipe.scripts`` in the pertinent sections of your - buildout configuration and your generated scripts will work with a system - Python. - - Other updated recipes include zc.recipe.testrunner 1.4.0 and - z3c.recipe.tag 0.4.0. Others should be updated soon: see their change - documents for details, or see `Updating Recipes to Support a System - Python`_ for instructions on how to update recipes yourself. - - Templates for creating Python scripts with the z3c.recipe.filetemplate - recipe can be easily changed to support a system Python. - - - If you don't care about supporting relative paths, simply using a - generated interpreter with the eggs you want should be sufficient, as - it was before. For instance, if the interpreter is named "py", use - ``#!${buildout:bin-directory/py}`` or ``#!/usr/bin/env - ${buildout:bin-directory/py}``). - - - If you do care about relative paths, (``relative-paths = true`` in - your buildout configuration), then z3c.recipe.scripts does require a - bit more changes, as is usual for the relative path support in that - package. First, use z3c.recipe.scripts to generate a script or - interpreter with the dependencies you want. This will create a - directory in ``parts`` that has a site.py and sitecustomize.py. Then, - begin your script as in the snippet below. The example assumes that - the z3c.recipe.scripts generated were from a Buildout configuration - section labeled "scripts": adjust accordingly. - - :: - - #!${buildout:executable} -S - ${python-relative-path-setup} - import sys - sys.path.insert(0, ${scripts:parts-directory|path-repr}) - import site - - Updating Recipes to Support a System Python - =========================================== - - You should generally only need to update recipes that generate scripts. - These recipes need to change from using ``zc.buildout.easy_install.scripts`` - to be using ``zc.buildout.easy_install.sitepackage_safe_scripts``. - The signatures of the two functions are different. Please compare:: - - def scripts( - reqs, working_set, executable, dest, - scripts=None, - extra_paths=(), - arguments='', - interpreter=None, - initialization='', - relative_paths=False, - ): - - def sitepackage_safe_scripts( - dest, working_set, executable, site_py_dest, - reqs=(), - scripts=None, - interpreter=None, - extra_paths=(), - initialization='', - include_site_packages=False, - exec_sitecustomize=False, - relative_paths=False, - script_arguments='', - script_initialization='', - ): - - In most cases, the arguments are merely reordered. The ``reqs`` - argument is no longer required in order to make it easier to generate an - interpreter alone. The ``arguments`` argument was renamed to - ``script_arguments`` to clarify that it did not affect interpreter - generation. - - The only new required argument is ``site_py_dest``. It must be the path - to a directory in which the customized site.py and sitecustomize.py - files will be written. A typical generation in a recipe will look like - this. - - (In the recipe's __init__ method...) - - :: - - self.options = options - b_options = buildout['buildout'] - options['parts-directory'] = os.path.join( - b_options['parts-directory'], self.name) - - (In the recipe's install method...) - - :: - - options = self.options - generated = [] - if not os.path.exists(options['parts-directory']): - os.mkdir(options['parts-directory']) - generated.append(options['parts-directory']) - - Then ``options['parts-directory']`` can be used for the ``site_py_dest`` - value. - - If you want to support the other arguments (``include_site_packages``, - ``exec_sitecustomize``, ``script_initialization``, as well as the - ``allowed-eggs-from-site-packages`` option), you might want to look at - some of the code in - svn://svn.zope.org/repos/main/zc.buildout/trunk/z3c.recipe.scripts\_/src/z3c/recipe/scripts/scripts.py . - You might even be able to adopt some of it by subclassing or delegating. - The Scripts class in that file is the closest to what you might be used - to from zc.recipe.egg. - - Important note for recipe authors: As of buildout 1.5.2, the code in - recipes is *always run with the access to the site-packages as - configured in the buildout section*. - - virtualenv - ========== - - Using virtualenv (http://pypi.python.org/pypi/virtualenv) with the - --no-site-packages option already provided a simple way of using a - system Python. This is intended to continue to work, and some automated - tests exist to demonstrate this. - - However, it is only supported to the degree that people have found it to - work in the past. The existing Buildout tests for virtualenv are only - for problems encountered previously. They are very far from - comprehensive. - - Using Buildout with a system python has at least three advantages over - using Buildout in conjunction with virtualenv. They may or may not be - pertinent to your desired usage. - - - Unlike ``virtualenv --no-site-packages``, Buildout's support allows you - to choose to let packages from your system Python be available to your - software (see ``include-site-packages`` in - http://pypi.python.org/pypi/z3c.recipe.scripts). - - You can even specify which eggs installed in your system Python can be - allowed to fulfill some of your packages' dependencies (see - ``allowed-eggs-from-site-packages`` in - http://pypi.python.org/pypi/z3c.recipe.scripts). - - At the expense of some repeatability and platform dependency, this - flexibility means that, for instance, you can rely on - difficult-to-build eggs like lxml coming from your system Python. - - - Buildout's implementation has a full set of automated tests. - - - An integral Buildout implementation means fewer steps and fewer dependencies - to work with a system Python. - - Detailed Documentation - ********************** - - Buildouts - ========= - - The word "buildout" refers to a description of a set of parts and the - software to create and assemble them. It is often used informally to - refer to an installed system based on a buildout definition. For - example, if we are creating an application named "Foo", then "the Foo - buildout" is the collection of configuration and application-specific - software that allows an instance of the application to be created. We - may refer to such an instance of the application informally as "a Foo - buildout". - - This document describes how to define buildouts using buildout - configuration files and recipes. There are three ways to set up the - buildout software and create a buildout instance: - - 1. Install the zc.buildout egg with easy_install and use the buildout - script installed in a Python scripts area. - - 2. Use the buildout bootstrap script to create a buildout that - includes both the setuptools and zc.buildout eggs. This allows you - to use the buildout software without modifying a Python install. - The buildout script is installed into your buildout local scripts - area. - - 3. Use a buildout command from an already installed buildout to - bootstrap a new buildout. (See the section on bootstraping later - in this document.) - - Often, a software project will be managed in a software repository, - such as a subversion repository, that includes some software source - directories, buildout configuration files, and a copy of the buildout - bootstrap script. To work on the project, one would check out the - project from the repository and run the bootstrap script which - installs setuptools and zc.buildout into the checkout as well as any - parts defined. - - We have a sample buildout that we created using the bootstrap command - of an existing buildout (method 3 above). It has the absolute minimum - information. We have bin, develop-eggs, eggs and parts directories, - and a configuration file: - - >>> ls(sample_buildout) - d bin - - buildout.cfg - d develop-eggs - d eggs - d parts - - The bin directory contains scripts. - - >>> ls(sample_buildout, 'bin') - - buildout - - >>> ls(sample_buildout, 'eggs') - - setuptools-0.6-py2.4.egg - - zc.buildout-1.0-py2.4.egg - - The develop-eggs directory is initially empty: - - >>> ls(sample_buildout, 'develop-eggs') - - The develop-eggs directory holds egg links for software being - developed in the buildout. We separate develop-eggs and other eggs to - allow eggs directories to be shared across multiple buildouts. For - example, a common developer technique is to define a common eggs - directory in their home that all non-develop eggs are stored in. This - allows larger buildouts to be set up much more quickly and saves disk - space. - - The parts directory just contains some helpers for the buildout script - itself. - - >>> ls(sample_buildout, 'parts') - d buildout - - The parts directory provides an area where recipes can install - part data. For example, if we built a custom Python, we would - install it in the part directory. Part data is stored in a - sub-directory of the parts directory with the same name as the part. - - Buildouts are defined using configuration files. These are in the - format defined by the Python ConfigParser module, with extensions - that we'll describe later. By default, when a buildout is run, it - looks for the file buildout.cfg in the directory where the buildout is - run. - - The minimal configuration file has a buildout section that defines no - parts: - - >>> cat(sample_buildout, 'buildout.cfg') - [buildout] - parts = - - A part is simply something to be created by a buildout. It can be - almost anything, such as a Python package, a program, a directory, or - even a configuration file. - - Recipes - ------- - - A part is created by a recipe. Recipes are always installed as Python - eggs. They can be downloaded from a package server, such as the - Python Package Index, or they can be developed as part of a project - using a "develop" egg. - - A develop egg is a special kind of egg that gets installed as an "egg - link" that contains the name of a source directory. Develop eggs - don't have to be packaged for distribution to be used and can be - modified in place, which is especially useful while they are being - developed. - - Let's create a recipe as part of the sample project. We'll create a - recipe for creating directories. First, we'll create a recipes source - directory for our local recipes: - - >>> mkdir(sample_buildout, 'recipes') - - and then we'll create a source file for our mkdir recipe: - - >>> write(sample_buildout, 'recipes', 'mkdir.py', - ... """ - ... import logging, os, zc.buildout - ... - ... class Mkdir: - ... - ... def __init__(self, buildout, name, options): - ... self.name, self.options = name, options - ... options['path'] = os.path.join( - ... buildout['buildout']['directory'], - ... options['path'], - ... ) - ... if not os.path.isdir(os.path.dirname(options['path'])): - ... logging.getLogger(self.name).error( - ... 'Cannot create %s. %s is not a directory.', - ... options['path'], os.path.dirname(options['path'])) - ... raise zc.buildout.UserError('Invalid Path') - ... - ... - ... def install(self): - ... path = self.options['path'] - ... logging.getLogger(self.name).info( - ... 'Creating directory %s', os.path.basename(path)) - ... os.mkdir(path) - ... return path - ... - ... def update(self): - ... pass - ... """) - - Currently, recipes must define 3 methods [#future_recipe_methods]_: - - - a constructor, - - - an install method, and - - - an update method. - - The constructor is responsible for updating a parts options to reflect - data read from other sections. The buildout system keeps track of - whether a part specification has changed. A part specification has - changed if it's options, after adjusting for data read from other - sections, has changed, or if the recipe has changed. Only the options - for the part are considered. If data are read from other sections, - then that information has to be reflected in the parts options. In - the Mkdir example, the given path is interpreted relative to the - buildout directory, and data from the buildout directory is read. The - path option is updated to reflect this. If the directory option was - changed in the buildout sections, we would know to update parts - created using the mkdir recipe using relative path names. - - When buildout is run, it saves configuration data for installed parts - in a file named ".installed.cfg". In subsequent runs, it compares - part-configuration data stored in the .installed.cfg file and the - part-configuration data loaded from the configuration files as - modified by recipe constructors to decide if the configuration of a - part has changed. If the configuration has changed, or if the recipe - has changed, then the part is uninstalled and reinstalled. The - buildout only looks at the part's options, so any data used to - configure the part needs to be reflected in the part's options. It is - the job of a recipe constructor to make sure that the options include - all relevant data. - - Of course, parts are also uninstalled if they are no-longer used. - - The recipe defines a constructor that takes a buildout object, a part - name, and an options dictionary. It saves them in instance attributes. - If the path is relative, we'll interpret it as relative to the - buildout directory. The buildout object passed in is a mapping from - section name to a mapping of options for that section. The buildout - directory is available as the directory option of the buildout - section. We normalize the path and save it back into the options - directory. - - The install method is responsible for creating the part. In this - case, we need the path of the directory to create. We'll use a path - option from our options dictionary. The install method logs what it's - doing using the Python logging call. We return the path that we - installed. If the part is uninstalled or reinstalled, then the path - returned will be removed by the buildout machinery. A recipe install - method is expected to return a string, or an iterable of strings - containing paths to be removed if a part is uninstalled. For most - recipes, this is all of the uninstall support needed. For more complex - uninstallation scenarios use `Uninstall recipes`_. - - The update method is responsible for updating an already installed - part. An empty method is often provided, as in this example, if parts - can't be updated. An update method can return None, a string, or an - iterable of strings. If a string or iterable of strings is returned, - then the saved list of paths to be uninstalled is updated with the new - information by adding any new files returned by the update method. - - We need to provide packaging information so that our recipe can be - installed as a develop egg. The minimum information we need to specify - [#packaging_info]_ is a name. For recipes, we also need to define the - names of the recipe classes as entry points. Packaging information is - provided via a setup.py script: - - >>> write(sample_buildout, 'recipes', 'setup.py', - ... """ - ... from setuptools import setup - ... - ... setup( - ... name = "recipes", - ... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']}, - ... ) - ... """) - - Our setup script defines an entry point. Entry points provide - a way for an egg to define the services it provides. Here we've said - that we define a zc.buildout entry point named mkdir. Recipe - classes must be exposed as entry points in the zc.buildout group. we - give entry points names within the group. - - We also need a README.txt for our recipes to avoid an annoying warning - from distutils, on which setuptools and zc.buildout are based: - - >>> write(sample_buildout, 'recipes', 'README.txt', " ") - - Now let's update our buildout.cfg: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = mystuff - ... """) - - Let's go through the changes one by one:: - - develop = recipes - - This tells the buildout to install a development egg for our recipes. - Any number of paths can be listed. The paths can be relative or - absolute. If relative, they are treated as relative to the buildout - directory. They can be directory or file paths. If a file path is - given, it should point to a Python setup script. If a directory path - is given, it should point to a directory containing a setup.py file. - Development eggs are installed before building any parts, as they may - provide locally-defined recipes needed by the parts. - - :: - - parts = data-dir - - Here we've named a part to be "built". We can use any name we want - except that different part names must be unique and recipes will often - use the part name to decide what to do. - - :: - - [data-dir] - recipe = recipes:mkdir - path = mystuff - - - When we name a part, we also create a section of the same - name that contains part data. In this section, we'll define - the recipe to be used to install the part. In this case, we also - specify the path to be created. - - Let's run the buildout. We do so by running the build script in the - buildout: - - >>> import os - >>> os.chdir(sample_buildout) - >>> buildout = os.path.join(sample_buildout, 'bin', 'buildout') - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Installing data-dir. - data-dir: Creating directory mystuff - - We see that the recipe created the directory, as expected: - - >>> ls(sample_buildout) - - .installed.cfg - d bin - - buildout.cfg - d develop-eggs - d eggs - d mystuff - d parts - d recipes - - In addition, .installed.cfg has been created containing information - about the part we installed: - - >>> cat(sample_buildout, '.installed.cfg') - [buildout] - installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link - parts = data-dir - <BLANKLINE> - [data-dir] - __buildout_installed__ = /sample-buildout/mystuff - __buildout_signature__ = recipes-c7vHV6ekIDUPy/7fjAaYjg== - path = /sample-buildout/mystuff - recipe = recipes:mkdir - - Note that the directory we installed is included in .installed.cfg. - In addition, the path option includes the actual destination - directory. - - If we change the name of the directory in the configuration file, - we'll see that the directory gets removed and recreated: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = mydata - ... """) - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling data-dir. - Installing data-dir. - data-dir: Creating directory mydata - - >>> ls(sample_buildout) - - .installed.cfg - d bin - - buildout.cfg - d develop-eggs - d eggs - d mydata - d parts - d recipes - - If any of the files or directories created by a recipe are removed, - the part will be reinstalled: - - >>> rmdir(sample_buildout, 'mydata') - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling data-dir. - Installing data-dir. - data-dir: Creating directory mydata - - Error reporting - --------------- - - If a user makes an error, an error needs to be printed and work needs - to stop. This is accomplished by logging a detailed error message and - then raising a (or an instance of a subclass of a) - zc.buildout.UserError exception. Raising an error other than a - UserError still displays the error, but labels it as a bug in the - buildout software or recipe. In the sample above, of someone gives a - non-existent directory to create the directory in: - - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = /xxx/mydata - ... """) - - We'll get a user error, not a traceback. - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - data-dir: Cannot create /xxx/mydata. /xxx is not a directory. - While: - Installing. - Getting section data-dir. - Initializing part data-dir. - Error: Invalid Path - - - Recipe Error Handling - --------------------- - - If an error occurs during installation, it is up to the recipe to - clean up any system side effects, such as files created. Let's update - the mkdir recipe to support multiple paths: - - >>> write(sample_buildout, 'recipes', 'mkdir.py', - ... """ - ... import logging, os, zc.buildout - ... - ... class Mkdir: - ... - ... def __init__(self, buildout, name, options): - ... self.name, self.options = name, options - ... - ... # Normalize paths and check that their parent - ... # directories exist: - ... paths = [] - ... for path in options['path'].split(): - ... path = os.path.join(buildout['buildout']['directory'], path) - ... if not os.path.isdir(os.path.dirname(path)): - ... logging.getLogger(self.name).error( - ... 'Cannot create %s. %s is not a directory.', - ... options['path'], os.path.dirname(options['path'])) - ... raise zc.buildout.UserError('Invalid Path') - ... paths.append(path) - ... options['path'] = ' '.join(paths) - ... - ... def install(self): - ... paths = self.options['path'].split() - ... for path in paths: - ... logging.getLogger(self.name).info( - ... 'Creating directory %s', os.path.basename(path)) - ... os.mkdir(path) - ... return paths - ... - ... def update(self): - ... pass - ... """) - - If there is an error creating a path, the install method will exit and - leave previously created paths in place: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = foo bin - ... """) - - >>> print system(buildout), # doctest: +ELLIPSIS - Develop: '/sample-buildout/recipes' - Uninstalling data-dir. - Installing data-dir. - data-dir: Creating directory foo - data-dir: Creating directory bin - While: - Installing data-dir. - <BLANKLINE> - An internal error occurred due to a bug in either zc.buildout or in a - recipe being used: - Traceback (most recent call last): - ... - OSError: [Errno 17] File exists: '/sample-buildout/bin' - - We meant to create a directory bins, but typed bin. Now foo was - left behind. - - >>> os.path.exists('foo') - True - - If we fix the typo: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = foo bins - ... """) - - >>> print system(buildout), # doctest: +ELLIPSIS - Develop: '/sample-buildout/recipes' - Installing data-dir. - data-dir: Creating directory foo - While: - Installing data-dir. - <BLANKLINE> - An internal error occurred due to a bug in either zc.buildout or in a - recipe being used: - Traceback (most recent call last): - ... - OSError: [Errno 17] File exists: '/sample-buildout/foo' - - Now they fail because foo exists, because it was left behind. - - >>> remove('foo') - - Let's fix the recipe: - - >>> write(sample_buildout, 'recipes', 'mkdir.py', - ... """ - ... import logging, os, zc.buildout - ... - ... class Mkdir: - ... - ... def __init__(self, buildout, name, options): - ... self.name, self.options = name, options - ... - ... # Normalize paths and check that their parent - ... # directories exist: - ... paths = [] - ... for path in options['path'].split(): - ... path = os.path.join(buildout['buildout']['directory'], path) - ... if not os.path.isdir(os.path.dirname(path)): - ... logging.getLogger(self.name).error( - ... 'Cannot create %s. %s is not a directory.', - ... options['path'], os.path.dirname(options['path'])) - ... raise zc.buildout.UserError('Invalid Path') - ... paths.append(path) - ... options['path'] = ' '.join(paths) - ... - ... def install(self): - ... paths = self.options['path'].split() - ... created = [] - ... try: - ... for path in paths: - ... logging.getLogger(self.name).info( - ... 'Creating directory %s', os.path.basename(path)) - ... os.mkdir(path) - ... created.append(path) - ... except: - ... for d in created: - ... os.rmdir(d) - ... raise - ... - ... return paths - ... - ... def update(self): - ... pass - ... """) - - And put back the typo: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = foo bin - ... """) - - When we rerun the buildout: - - >>> print system(buildout), # doctest: +ELLIPSIS - Develop: '/sample-buildout/recipes' - Installing data-dir. - data-dir: Creating directory foo - data-dir: Creating directory bin - While: - Installing data-dir. - <BLANKLINE> - An internal error occurred due to a bug in either zc.buildout or in a - recipe being used: - Traceback (most recent call last): - ... - OSError: [Errno 17] File exists: '/sample-buildout/bin' - - .. Wait for the file to really disappear. My linux is weird. - - >>> wait_until("foo goes away", lambda : not os.path.exists('foo'), - ... timeout=200) - - we get the same error, but we don't get the directory left behind: - - >>> os.path.exists('foo') - False - - It's critical that recipes clean up partial effects when errors - occur. Because recipes most commonly create files and directories, - buildout provides a helper API for removing created files when an - error occurs. Option objects have a created method that can be called - to record files as they are created. If the install or update method - returns with an error, then any registered paths are removed - automatically. The method returns the files registered and can be - used to return the files created. Let's use this API to simplify the - recipe: - - >>> write(sample_buildout, 'recipes', 'mkdir.py', - ... """ - ... import logging, os, zc.buildout - ... - ... class Mkdir: - ... - ... def __init__(self, buildout, name, options): - ... self.name, self.options = name, options - ... - ... # Normalize paths and check that their parent - ... # directories exist: - ... paths = [] - ... for path in options['path'].split(): - ... path = os.path.join(buildout['buildout']['directory'], path) - ... if not os.path.isdir(os.path.dirname(path)): - ... logging.getLogger(self.name).error( - ... 'Cannot create %s. %s is not a directory.', - ... options['path'], os.path.dirname(options['path'])) - ... raise zc.buildout.UserError('Invalid Path') - ... paths.append(path) - ... options['path'] = ' '.join(paths) - ... - ... def install(self): - ... paths = self.options['path'].split() - ... for path in paths: - ... logging.getLogger(self.name).info( - ... 'Creating directory %s', os.path.basename(path)) - ... os.mkdir(path) - ... self.options.created(path) - ... - ... return self.options.created() - ... - ... def update(self): - ... pass - ... """) - - .. - - >>> remove(sample_buildout, 'recipes', 'mkdir.pyc') - - We returned by calling created, taking advantage of the fact that it - returns the registered paths. We did this for illustrative purposes. - It would be simpler just to return the paths as before. - - If we rerun the buildout, again, we'll get the error and no - directories will be created: - - >>> print system(buildout), # doctest: +ELLIPSIS - Develop: '/sample-buildout/recipes' - Installing data-dir. - data-dir: Creating directory foo - data-dir: Creating directory bin - While: - Installing data-dir. - <BLANKLINE> - An internal error occurred due to a bug in either zc.buildout or in a - recipe being used: - Traceback (most recent call last): - ... - OSError: [Errno 17] File exists: '/sample-buildout/bin' - - >>> os.path.exists('foo') - False - - Now, we'll fix the typo again and we'll get the directories we expect: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = foo bins - ... """) - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Installing data-dir. - data-dir: Creating directory foo - data-dir: Creating directory bins - - >>> os.path.exists('foo') - True - >>> os.path.exists('bins') - True - - Configuration file syntax - ------------------------- - - As mentioned earlier, buildout configuration files use the format - defined by the Python ConfigParser module with extensions. The - extensions are: - - - option names are case sensitive - - - option values can use a substitution syntax, described below, to - refer to option values in specific sections. - - - option values can be appended or removed using the - and + - operators. - - The ConfigParser syntax is very flexible. Section names can contain - any characters other than newlines and right square braces ("]"). - Option names can contain any characters other than newlines, colons, - and equal signs, can not start with a space, and don't include - trailing spaces. - - It is likely that, in the future, some characters will be given - special buildout-defined meanings. This is already true of the - characters ":", "$", "%", "(", and ")". For now, it is a good idea to - keep section and option names simple, sticking to alphanumeric - characters, hyphens, and periods. - - Annotated sections - ------------------ - - When used with the `annotate` command, buildout displays annotated sections. - All sections are displayed, sorted alphabetically. For each section, - all key-value pairs are displayed, sorted alphabetically, along with - the origin of the value (file name or COMPUTED_VALUE, DEFAULT_VALUE, - COMMAND_LINE_VALUE). - - >>> print system(buildout+ ' annotate'), - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - <BLANKLINE> - Annotated sections - ================== - <BLANKLINE> - [buildout] - accept-buildout-test-releases= false - DEFAULT_VALUE - allow-hosts= * - DEFAULT_VALUE - allow-picked-versions= true - DEFAULT_VALUE - allowed-eggs-from-site-packages= * - DEFAULT_VALUE - bin-directory= bin - DEFAULT_VALUE - develop= recipes - /sample-buildout/buildout.cfg - develop-eggs-directory= develop-eggs - DEFAULT_VALUE - directory= /sample-buildout - COMPUTED_VALUE - eggs-directory= eggs - DEFAULT_VALUE - exec-sitecustomize= true - DEFAULT_VALUE - executable= ... - DEFAULT_VALUE - find-links= - DEFAULT_VALUE - include-site-packages= true - DEFAULT_VALUE - install-from-cache= false - DEFAULT_VALUE - installed= .installed.cfg - DEFAULT_VALUE - log-format= - DEFAULT_VALUE - log-level= INFO - DEFAULT_VALUE - newest= true - DEFAULT_VALUE - offline= false - DEFAULT_VALUE - parts= data-dir - /sample-buildout/buildout.cfg - parts-directory= parts - DEFAULT_VALUE - prefer-final= false - DEFAULT_VALUE - python= buildout - DEFAULT_VALUE - relative-paths= false - DEFAULT_VALUE - socket-timeout= - DEFAULT_VALUE - unzip= false - DEFAULT_VALUE - use-dependency-links= true - DEFAULT_VALUE - <BLANKLINE> - [data-dir] - path= foo bins - /sample-buildout/buildout.cfg - recipe= recipes:mkdir - /sample-buildout/buildout.cfg - <BLANKLINE> - - Variable substitutions - ---------------------- - - Buildout configuration files support variable substitution. - To illustrate this, we'll create an debug recipe to - allow us to see interactions with the buildout: - - >>> write(sample_buildout, 'recipes', 'debug.py', - ... """ - ... class Debug: - ... - ... def __init__(self, buildout, name, options): - ... self.buildout = buildout - ... self.name = name - ... self.options = options - ... - ... def install(self): - ... items = self.options.items() - ... items.sort() - ... for option, value in items: - ... print option, value - ... return () - ... - ... update = install - ... """) - - This recipe doesn't actually create anything. The install method - doesn't return anything, because it didn't create any files or - directories. - - We also have to update our setup script: - - >>> write(sample_buildout, 'recipes', 'setup.py', - ... """ - ... from setuptools import setup - ... entry_points = ( - ... ''' - ... [zc.buildout] - ... mkdir = mkdir:Mkdir - ... debug = debug:Debug - ... ''') - ... setup(name="recipes", entry_points=entry_points) - ... """) - - We've rearranged the script a bit to make the entry points easier to - edit. In particular, entry points are now defined as a configuration - string, rather than a dictionary. - - Let's update our configuration to provide variable substitution - examples: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir debug - ... log-level = INFO - ... - ... [debug] - ... recipe = recipes:debug - ... File 1 = ${data-dir:path}/file - ... File 2 = ${debug:File 1}/log - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = mydata - ... """) - - We used a string-template substitution for File 1 and File 2. This - type of substitution uses the string.Template syntax. Names - substituted are qualified option names, consisting of a section name - and option name joined by a colon. - - Now, if we run the buildout, we'll see the options with the values - substituted. - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling data-dir. - Installing data-dir. - data-dir: Creating directory mydata - Installing debug. - File 1 /sample-buildout/mydata/file - File 2 /sample-buildout/mydata/file/log - recipe recipes:debug - - Note that the substitution of the data-dir path option reflects the - update to the option performed by the mkdir recipe. - - It might seem surprising that mydata was created again. This is - because we changed our recipes package by adding the debug module. - The buildout system didn't know if this module could effect the mkdir - recipe, so it assumed it could and reinstalled mydata. If we rerun - the buildout: - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Updating data-dir. - Updating debug. - File 1 /sample-buildout/mydata/file - File 2 /sample-buildout/mydata/file/log - recipe recipes:debug - - We can see that mydata was not recreated. - - Note that, in this case, we didn't specify a log level, so - we didn't get output about what the buildout was doing. - - Section and option names in variable substitutions are only allowed to - contain alphanumeric characters, hyphens, periods and spaces. This - restriction might be relaxed in future releases. - - We can ommit the section name in a variable substitution to refer to - the current section. We can also use the special option, - _buildout_section_name_ to get the current section name. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir debug - ... log-level = INFO - ... - ... [debug] - ... recipe = recipes:debug - ... File 1 = ${data-dir:path}/file - ... File 2 = ${:File 1}/log - ... my_name = ${:_buildout_section_name_} - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = mydata - ... """) - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Updating data-dir. - Installing debug. - File 1 /sample-buildout/mydata/file - File 2 /sample-buildout/mydata/file/log - my_name debug - recipe recipes:debug - - Automatic part selection and ordering - ------------------------------------- - - When a section with a recipe is referred to, either through variable - substitution or by an initializing recipe, the section is treated as a - part and added to the part list before the referencing part. For - example, we can leave data-dir out of the parts list: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... log-level = INFO - ... - ... [debug] - ... recipe = recipes:debug - ... File 1 = ${data-dir:path}/file - ... File 2 = ${debug:File 1}/log - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = mydata - ... """) - - - It will still be treated as a part: - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Updating data-dir. - Installing debug. - File 1 /sample-buildout/mydata/file - File 2 /sample-buildout/mydata/file/log - recipe recipes:debug - - >>> cat('.installed.cfg') # doctest: +ELLIPSIS - [buildout] - installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link - parts = data-dir debug - ... - - Note that the data-dir part is included *before* the debug part, - because the debug part refers to the data-dir part. Even if we list - the data-dir part after the debug part, it will be included before: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug data-dir - ... log-level = INFO - ... - ... [debug] - ... recipe = recipes:debug - ... File 1 = ${data-dir:path}/file - ... File 2 = ${debug:File 1}/log - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = mydata - ... """) - - - It will still be treated as a part: - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Updating data-dir. - Updating debug. - File 1 /sample-buildout/mydata/file - File 2 /sample-buildout/mydata/file/log - recipe recipes:debug - - >>> cat('.installed.cfg') # doctest: +ELLIPSIS - [buildout] - installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link - parts = data-dir debug - ... - - Extending sections (macros) - --------------------------- - - A section (other than the buildout section) can extend one or more - other sections using the ``<=`` option. Options from the referenced - sections are copied to the refering section *before* variable - substitution. This, together with the ability to refer to variables - of the current section allows sections to be used as macros. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = myfiles - ... log-level = INFO - ... - ... [debug] - ... recipe = recipes:debug - ... - ... [with_file1] - ... <= debug - ... file1 = ${:path}/file1 - ... color = red - ... - ... [with_file2] - ... <= debug - ... file2 = ${:path}/file2 - ... color = blue - ... - ... [myfiles] - ... <= with_file1 - ... with_file2 - ... path = mydata - ... """) - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Uninstalling data-dir. - Installing myfiles. - color blue - file1 mydata/file1 - file2 mydata/file2 - path mydata - recipe recipes:debug - - In this example, the debug, with_file1 and with_file2 sections act as - macros. In particular, the variable substitutions are performed - relative to the myfiles section. - - Adding and removing options - --------------------------- - - We can append and remove values to an option by using the + and - - operators. - - This is illustrated below; first we define a base configuration. - - >>> write(sample_buildout, 'base.cfg', - ... """ - ... [buildout] - ... parts = part1 part2 part3 - ... - ... [part1] - ... recipe = - ... option = a1 a2 - ... - ... [part2] - ... recipe = - ... option = b1 b2 b3 b4 - ... - ... [part3] - ... recipe = - ... option = c1 c2 - ... - ... """) - - Extending this configuration, we can "adjust" the values set in the - base configuration file. - - >>> write(sample_buildout, 'extension1.cfg', - ... """ - ... [buildout] - ... extends = base.cfg - ... - ... # appending values - ... [part1] - ... option += a3 a4 - ... - ... # removing values - ... [part2] - ... option -= b1 b2 - ... - ... # alt. spelling - ... [part3] - ... option+=c3 c4 c5 - ... - ... # normal assignment - ... [part4] - ... option = h1 h2 - ... - ... """) - - An additional extension. - - >>> write(sample_buildout, 'extension2.cfg', - ... """ - ... [buildout] - ... extends = extension1.cfg - ... - ... # appending values - ... [part1] - ... option += a5 - ... - ... # removing values - ... [part2] - ... option -= b1 b2 b3 - ... - ... """) - - To verify that the options are adjusted correctly, we'll set up an - extension that prints out the options. - - >>> mkdir(sample_buildout, 'demo') - >>> write(sample_buildout, 'demo', 'demo.py', - ... """ - ... def ext(buildout): - ... print [part['option'] for name, part in buildout.items() \ - ... if name.startswith('part')] - ... """) - - >>> write(sample_buildout, 'demo', 'setup.py', - ... """ - ... from setuptools import setup - ... - ... setup( - ... name="demo", - ... entry_points={'zc.buildout.extension': ['ext = demo:ext']}, - ... ) - ... """) - - Set up a buildout configuration for this extension. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = demo - ... parts = - ... """) - - >>> os.chdir(sample_buildout) - >>> print system(os.path.join(sample_buildout, 'bin', 'buildout')), - Develop: '/sample-buildout/demo' - Uninstalling myfiles. - Getting distribution for 'recipes'. - zip_safe flag not set; analyzing archive contents... - Got recipes 0.0.0. - warning: install_lib: 'build/lib' does not exist -- no Python modules to install - - Verify option values. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = demo - ... extensions = demo - ... extends = extension2.cfg - ... """) - - >>> print system(os.path.join('bin', 'buildout')), - ['a1 a2/na3 a4/na5', 'b1 b2 b3 b4', 'c1 c2/nc3 c4 c5', 'h1 h2'] - Develop: '/sample-buildout/demo' - - Annotated sections output shows which files are responsible for which - operations. - - >>> print system(os.path.join('bin', 'buildout') + ' annotate'), - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - <BLANKLINE> - Annotated sections - ================== - ... - <BLANKLINE> - [part1] - option= a1 a2 - a3 a4 - a5 - /sample-buildout/base.cfg - += /sample-buildout/extension1.cfg - += /sample-buildout/extension2.cfg - recipe= - /sample-buildout/base.cfg - <BLANKLINE> - [part2] - option= b1 b2 b3 b4 - /sample-buildout/base.cfg - -= /sample-buildout/extension1.cfg - -= /sample-buildout/extension2.cfg - recipe= - /sample-buildout/base.cfg - <BLANKLINE> - [part3] - option= c1 c2 - c3 c4 c5 - /sample-buildout/base.cfg - += /sample-buildout/extension1.cfg - recipe= - /sample-buildout/base.cfg - <BLANKLINE> - [part4] - option= h1 h2 - /sample-buildout/extension1.cfg - - Cleanup. - - >>> os.remove(os.path.join(sample_buildout, 'base.cfg')) - >>> os.remove(os.path.join(sample_buildout, 'extension1.cfg')) - >>> os.remove(os.path.join(sample_buildout, 'extension2.cfg')) - - Multiple configuration files - ---------------------------- - - A configuration file can "extend" another configuration file. - Options are read from the other configuration file if they aren't - already defined by your configuration file. - - The configuration files your file extends can extend - other configuration files. The same file may be - used more than once although, of course, cycles aren't allowed. - - To see how this works, we use an example: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... extends = base.cfg - ... - ... [debug] - ... op = buildout - ... """) - - >>> write(sample_buildout, 'base.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... - ... [debug] - ... recipe = recipes:debug - ... op = base - ... """) - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Installing debug. - op buildout - recipe recipes:debug - - The example is pretty trivial, but the pattern it illustrates is - pretty common. In a more practical example, the base buildout might - represent a product and the extending buildout might be a - customization. - - Here is a more elaborate example. - - >>> other = tmpdir('other') - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... extends = b1.cfg b2.cfg %(b3)s - ... - ... [debug] - ... op = buildout - ... """ % dict(b3=os.path.join(other, 'b3.cfg'))) - - >>> write(sample_buildout, 'b1.cfg', - ... """ - ... [buildout] - ... extends = base.cfg - ... - ... [debug] - ... op1 = b1 1 - ... op2 = b1 2 - ... """) - - >>> write(sample_buildout, 'b2.cfg', - ... """ - ... [buildout] - ... extends = base.cfg - ... - ... [debug] - ... op2 = b2 2 - ... op3 = b2 3 - ... """) - - >>> write(other, 'b3.cfg', - ... """ - ... [buildout] - ... extends = b3base.cfg - ... - ... [debug] - ... op4 = b3 4 - ... """) - - >>> write(other, 'b3base.cfg', - ... """ - ... [debug] - ... op5 = b3base 5 - ... """) - - >>> write(sample_buildout, 'base.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... - ... [debug] - ... recipe = recipes:debug - ... name = base - ... """) - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing debug. - name base - op buildout - op1 b1 1 - op2 b2 2 - op3 b2 3 - op4 b3 4 - op5 b3base 5 - recipe recipes:debug - - There are several things to note about this example: - - - We can name multiple files in an extends option. - - - We can reference files recursively. - - - Relative file names in extended options are interpreted relative to - the directory containing the referencing configuration file. - - Loading Configuration from URLs - ------------------------------- - - Configuration files can be loaded from URLs. To see how this works, - we'll set up a web server with some configuration files. - - >>> server_data = tmpdir('server_data') - - >>> write(server_data, "r1.cfg", - ... """ - ... [debug] - ... op1 = r1 1 - ... op2 = r1 2 - ... """) - - >>> write(server_data, "r2.cfg", - ... """ - ... [buildout] - ... extends = r1.cfg - ... - ... [debug] - ... op2 = r2 2 - ... op3 = r2 3 - ... """) - - >>> server_url = start_server(server_data) - - >>> write('client.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... extends = %(url)s/r2.cfg - ... - ... [debug] - ... recipe = recipes:debug - ... name = base - ... """ % dict(url=server_url)) - - - >>> print system(buildout+ ' -c client.cfg'), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing debug. - name base - op1 r1 1 - op2 r2 2 - op3 r2 3 - recipe recipes:debug - - Here we specified a URL for the file we extended. The file we - downloaded, itself referred to a file on the server using a relative - URL reference. Relative references are interpreted relative to the - base URL when they appear in configuration files loaded via URL. - - We can also specify a URL as the configuration file to be used by a - buildout. - - >>> os.remove('client.cfg') - >>> write(server_data, 'remote.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... extends = r2.cfg - ... - ... [debug] - ... recipe = recipes:debug - ... name = remote - ... """) - - >>> print system(buildout + ' -c ' + server_url + '/remote.cfg'), - While: - Initializing. - Error: Missing option: buildout:directory - - Normally, the buildout directory defaults to directory - containing a configuration file. This won't work for configuration - files loaded from URLs. In this case, the buildout directory would - normally be defined on the command line: - - >>> print system(buildout - ... + ' -c ' + server_url + '/remote.cfg' - ... + ' buildout:directory=' + sample_buildout - ... ), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing debug. - name remote - op1 r1 1 - op2 r2 2 - op3 r2 3 - recipe recipes:debug - - User defaults - ------------- - - If the file $HOME/.buildout/default.cfg, exists, it is read before - reading the configuration file. ($HOME is the value of the HOME - environment variable. The '/' is replaced by the operating system file - delimiter.) - - >>> old_home = os.environ['HOME'] - >>> home = tmpdir('home') - >>> mkdir(home, '.buildout') - >>> write(home, '.buildout', 'default.cfg', - ... """ - ... [debug] - ... op1 = 1 - ... op7 = 7 - ... """) - - >>> os.environ['HOME'] = home - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing debug. - name base - op buildout - op1 b1 1 - op2 b2 2 - op3 b2 3 - op4 b3 4 - op5 b3base 5 - op7 7 - recipe recipes:debug - - A buildout command-line argument, -U, can be used to suppress reading - user defaults: - - >>> print system(buildout + ' -U'), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing debug. - name base - op buildout - op1 b1 1 - op2 b2 2 - op3 b2 3 - op4 b3 4 - op5 b3base 5 - recipe recipes:debug - - >>> os.environ['HOME'] = old_home - - Log level - --------- - - We can control the level of logging by specifying a log level in out - configuration file. For example, so suppress info messages, we can - set the logging level to WARNING - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... log-level = WARNING - ... extends = b1.cfg b2.cfg - ... """) - - >>> print system(buildout), - name base - op1 b1 1 - op2 b2 2 - op3 b2 3 - recipe recipes:debug - - Uninstall recipes - ----------------- - - As we've seen, when parts are installed, buildout keeps track of files - and directories that they create. When the parts are uninstalled these - files and directories are deleted. - - Sometimes more clean up is needed. For example, a recipe might add a - system service by calling chkconfig --add during installation. Later - during uninstallation, chkconfig --del will need to be called to - remove the system service. - - In order to deal with these uninstallation issues, you can register - uninstall recipes. Uninstall recipes are registered using the - 'zc.buildout.uninstall' entry point. Parts specify uninstall recipes - using the 'uninstall' option. - - In comparison to regular recipes, uninstall recipes are much - simpler. They are simply callable objects that accept the name of the - part to be uninstalled and the part's options dictionary. Uninstall - recipes don't have access to the part itself since it maybe not be - able to be instantiated at uninstallation time. - - Here's a recipe that simulates installation of a system service, along - with an uninstall recipe that simulates removing the service. - - >>> write(sample_buildout, 'recipes', 'service.py', - ... """ - ... class Service: - ... - ... def __init__(self, buildout, name, options): - ... self.buildout = buildout - ... self.name = name - ... self.options = options - ... - ... def install(self): - ... print "chkconfig --add %s" % self.options['script'] - ... return () - ... - ... def update(self): - ... pass - ... - ... - ... def uninstall_service(name, options): - ... print "chkconfig --del %s" % options['script'] - ... """) - - To use these recipes we must register them using entry points. Make - sure to use the same name for the recipe and uninstall recipe. This is - required to let buildout know which uninstall recipe goes with which - recipe. - - >>> write(sample_buildout, 'recipes', 'setup.py', - ... """ - ... from setuptools import setup - ... entry_points = ( - ... ''' - ... [zc.buildout] - ... mkdir = mkdir:Mkdir - ... debug = debug:Debug - ... service = service:Service - ... - ... [zc.buildout.uninstall] - ... service = service:uninstall_service - ... ''') - ... setup(name="recipes", entry_points=entry_points) - ... """) - - Here's how these recipes could be used in a buildout: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = service - ... - ... [service] - ... recipe = recipes:service - ... script = /path/to/script - ... """) - - When the buildout is run the service will be installed - - >>> print system(buildout) - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing service. - chkconfig --add /path/to/script - <BLANKLINE> - - The service has been installed. If the buildout is run again with no - changes, the service shouldn't be changed. - - >>> print system(buildout) - Develop: '/sample-buildout/recipes' - Updating service. - <BLANKLINE> - - Now we change the service part to trigger uninstallation and - re-installation. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = service - ... - ... [service] - ... recipe = recipes:service - ... script = /path/to/a/different/script - ... """) - - >>> print system(buildout) - Develop: '/sample-buildout/recipes' - Uninstalling service. - Running uninstall recipe. - chkconfig --del /path/to/script - Installing service. - chkconfig --add /path/to/a/different/script - <BLANKLINE> - - Now we remove the service part, and add another part. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... - ... [debug] - ... recipe = recipes:debug - ... """) - - >>> print system(buildout) - Develop: '/sample-buildout/recipes' - Uninstalling service. - Running uninstall recipe. - chkconfig --del /path/to/a/different/script - Installing debug. - recipe recipes:debug - <BLANKLINE> - - Uninstall recipes don't have to take care of removing all the files - and directories created by the part. This is still done automatically, - following the execution of the uninstall recipe. An upshot is that an - uninstallation recipe can access files and directories created by a - recipe before they are deleted. - - For example, here's an uninstallation recipe that simulates backing up - a directory before it is deleted. It is designed to work with the - mkdir recipe introduced earlier. - - >>> write(sample_buildout, 'recipes', 'backup.py', - ... """ - ... import os - ... def backup_directory(name, options): - ... path = options['path'] - ... size = len(os.listdir(path)) - ... print "backing up directory %s of size %s" % (path, size) - ... """) - - It must be registered with the zc.buildout.uninstall entry - point. Notice how it is given the name 'mkdir' to associate it with - the mkdir recipe. - - >>> write(sample_buildout, 'recipes', 'setup.py', - ... """ - ... from setuptools import setup - ... entry_points = ( - ... ''' - ... [zc.buildout] - ... mkdir = mkdir:Mkdir - ... debug = debug:Debug - ... service = service:Service - ... - ... [zc.buildout.uninstall] - ... uninstall_service = service:uninstall_service - ... mkdir = backup:backup_directory - ... ''') - ... setup(name="recipes", entry_points=entry_points) - ... """) - - Now we can use it with a mkdir part. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = dir debug - ... - ... [dir] - ... recipe = recipes:mkdir - ... path = my_directory - ... - ... [debug] - ... recipe = recipes:debug - ... """) - - Run the buildout to install the part. - - >>> print system(buildout) - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing dir. - dir: Creating directory my_directory - Installing debug. - recipe recipes:debug - <BLANKLINE> - - Now we remove the part from the configuration file. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... - ... [debug] - ... recipe = recipes:debug - ... """) - - When the buildout is run the part is removed, and the uninstall recipe - is run before the directory is deleted. - - >>> print system(buildout) - Develop: '/sample-buildout/recipes' - Uninstalling dir. - Running uninstall recipe. - backing up directory /sample-buildout/my_directory of size 0 - Updating debug. - recipe recipes:debug - <BLANKLINE> - - Now we will return the registration to normal for the benefit of the - rest of the examples. - - >>> write(sample_buildout, 'recipes', 'setup.py', - ... """ - ... from setuptools import setup - ... entry_points = ( - ... ''' - ... [zc.buildout] - ... mkdir = mkdir:Mkdir - ... debug = debug:Debug - ... ''') - ... setup(name="recipes", entry_points=entry_points) - ... """) - - - Command-line usage - ------------------ - - A number of arguments can be given on the buildout command line. The - command usage is:: - - buildout [options and assignments] [command [command arguments]] - - The following options are supported: - - -h (or --help) - Print basic usage information. If this option is used, then all - other options are ignored. - - -c filename - The -c option can be used to specify a configuration file, rather than - buildout.cfg in the current directory. - - - -t socket_timeout - - Specify the socket timeout in seconds. - - -v - Increment the verbosity by 10. The verbosity is used to adjust - the logging level. The verbosity is subtracted from the numeric - value of the log-level option specified in the configuration file. - - -q - Decrement the verbosity by 10. - - -U - Don't read user-default configuration. - - -o - Run in off-line mode. This is equivalent to the assignment - buildout:offline=true. - - -O - Run in non-off-line mode. This is equivalent to the assignment - buildout:offline=false. This is the default buildout mode. The - -O option would normally be used to override a true offline - setting in a configuration file. - - -n - Run in newest mode. This is equivalent to the assignment - buildout:newest=true. With this setting, which is the default, - buildout will try to find the newest versions of distributions - available that satisfy its requirements. - - -N - Run in non-newest mode. This is equivalent to the assignment - buildout:newest=false. With this setting, buildout will not seek - new distributions if installed distributions satisfy it's - requirements. - - Assignments are of the form:: - - section_name:option_name=value - - Options and assignments can be given in any order. - - Here's an example: - - >>> write(sample_buildout, 'other.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... installed = .other.cfg - ... log-level = WARNING - ... - ... [debug] - ... name = other - ... recipe = recipes:debug - ... """) - - Note that we used the installed buildout option to specify an - alternate file to store information about installed parts. - - >>> print system(buildout+' -c other.cfg debug:op1=foo -v'), - Develop: '/sample-buildout/recipes' - Installing debug. - name other - op1 foo - recipe recipes:debug - - Here we used the -c option to specify an alternate configuration file, - and the -v option to increase the level of logging from the default, - WARNING. - - Options can also be combined in the usual Unix way, as in: - - >>> print system(buildout+' -vcother.cfg debug:op1=foo'), - Develop: '/sample-buildout/recipes' - Updating debug. - name other - op1 foo - recipe recipes:debug - - Here we combined the -v and -c options with the configuration file - name. Note that the -c option has to be last, because it takes an - argument. - - >>> os.remove(os.path.join(sample_buildout, 'other.cfg')) - >>> os.remove(os.path.join(sample_buildout, '.other.cfg')) - - The most commonly used command is 'install' and it takes a list of - parts to install. if any parts are specified, only those parts are - installed. To illustrate this, we'll update our configuration and run - the buildout in the usual way: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug d1 d2 d3 - ... - ... [d1] - ... recipe = recipes:mkdir - ... path = d1 - ... - ... [d2] - ... recipe = recipes:mkdir - ... path = d2 - ... - ... [d3] - ... recipe = recipes:mkdir - ... path = d3 - ... - ... [debug] - ... recipe = recipes:debug - ... """) - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling debug. - Installing debug. - recipe recipes:debug - Installing d1. - d1: Creating directory d1 - Installing d2. - d2: Creating directory d2 - Installing d3. - d3: Creating directory d3 - - >>> ls(sample_buildout) - - .installed.cfg - - b1.cfg - - b2.cfg - - base.cfg - d bin - - buildout.cfg - d d1 - d d2 - d d3 - d demo - d develop-eggs - d eggs - d parts - d recipes - - >>> cat(sample_buildout, '.installed.cfg') - ... # doctest: +NORMALIZE_WHITESPACE - [buildout] - installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link - parts = debug d1 d2 d3 - <BLANKLINE> - [debug] - __buildout_installed__ = - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - recipe = recipes:debug - <BLANKLINE> - [d1] - __buildout_installed__ = /sample-buildout/d1 - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - path = /sample-buildout/d1 - recipe = recipes:mkdir - <BLANKLINE> - [d2] - __buildout_installed__ = /sample-buildout/d2 - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - path = /sample-buildout/d2 - recipe = recipes:mkdir - <BLANKLINE> - [d3] - __buildout_installed__ = /sample-buildout/d3 - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - path = /sample-buildout/d3 - recipe = recipes:mkdir - - Now we'll update our configuration file: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug d2 d3 d4 - ... - ... [d2] - ... recipe = recipes:mkdir - ... path = data2 - ... - ... [d3] - ... recipe = recipes:mkdir - ... path = data3 - ... - ... [d4] - ... recipe = recipes:mkdir - ... path = ${d2:path}-extra - ... - ... [debug] - ... recipe = recipes:debug - ... x = 1 - ... """) - - and run the buildout specifying just d3 and d4: - - >>> print system(buildout+' install d3 d4'), - Develop: '/sample-buildout/recipes' - Uninstalling d3. - Installing d3. - d3: Creating directory data3 - Installing d4. - d4: Creating directory data2-extra - - >>> ls(sample_buildout) - - .installed.cfg - - b1.cfg - - b2.cfg - - base.cfg - d bin - - buildout.cfg - d d1 - d d2 - d data2-extra - d data3 - d demo - d develop-eggs - d eggs - d parts - d recipes - - Only the d3 and d4 recipes ran. d3 was removed and data3 and data2-extra - were created. - - The .installed.cfg is only updated for the recipes that ran: - - >>> cat(sample_buildout, '.installed.cfg') - ... # doctest: +NORMALIZE_WHITESPACE - [buildout] - installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link - parts = debug d1 d2 d3 d4 - <BLANKLINE> - [debug] - __buildout_installed__ = - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - recipe = recipes:debug - <BLANKLINE> - [d1] - __buildout_installed__ = /sample-buildout/d1 - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - path = /sample-buildout/d1 - recipe = recipes:mkdir - <BLANKLINE> - [d2] - __buildout_installed__ = /sample-buildout/d2 - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - path = /sample-buildout/d2 - recipe = recipes:mkdir - <BLANKLINE> - [d3] - __buildout_installed__ = /sample-buildout/data3 - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - path = /sample-buildout/data3 - recipe = recipes:mkdir - <BLANKLINE> - [d4] - __buildout_installed__ = /sample-buildout/data2-extra - __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== - path = /sample-buildout/data2-extra - recipe = recipes:mkdir - - Note that the installed data for debug, d1, and d2 haven't changed, - because we didn't install those parts and that the d1 and d2 - directories are still there. - - Now, if we run the buildout without the install command: - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Uninstalling d2. - Uninstalling d1. - Uninstalling debug. - Installing debug. - recipe recipes:debug - x 1 - Installing d2. - d2: Creating directory data2 - Updating d3. - Updating d4. - - We see the output of the debug recipe and that data2 was created. We - also see that d1 and d2 have gone away: - - >>> ls(sample_buildout) - - .installed.cfg - - b1.cfg - - b2.cfg - - base.cfg - d bin - - buildout.cfg - d data2 - d data2-extra - d data3 - d demo - d develop-eggs - d eggs - d parts - d recipes - - Alternate directory and file locations - -------------------------------------- - - The buildout normally puts the bin, eggs, and parts directories in the - directory in the directory containing the configuration file. You can - provide alternate locations, and even names for these directories. - - >>> alt = tmpdir('sample-alt') - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = - ... develop-eggs-directory = %(developbasket)s - ... eggs-directory = %(basket)s - ... bin-directory = %(scripts)s - ... parts-directory = %(work)s - ... """ % dict( - ... developbasket = os.path.join(alt, 'developbasket'), - ... basket = os.path.join(alt, 'basket'), - ... scripts = os.path.join(alt, 'scripts'), - ... work = os.path.join(alt, 'work'), - ... )) - - >>> print system(buildout), - Creating directory '/sample-alt/scripts'. - Creating directory '/sample-alt/work'. - Creating directory '/sample-alt/basket'. - Creating directory '/sample-alt/developbasket'. - Develop: '/sample-buildout/recipes' - Uninstalling d4. - Uninstalling d3. - Uninstalling d2. - Uninstalling debug. - - >>> ls(alt) - d basket - d developbasket - d scripts - d work - - >>> ls(alt, 'developbasket') - - recipes.egg-link - - You can also specify an alternate buildout directory: - - >>> rmdir(alt) - >>> alt = tmpdir('sample-alt') - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... directory = %(alt)s - ... develop = %(recipes)s - ... parts = - ... """ % dict( - ... alt=alt, - ... recipes=os.path.join(sample_buildout, 'recipes'), - ... )) - - >>> print system(buildout), - Creating directory '/sample-alt/bin'. - Creating directory '/sample-alt/parts'. - Creating directory '/sample-alt/eggs'. - Creating directory '/sample-alt/develop-eggs'. - Develop: '/sample-buildout/recipes' - - >>> ls(alt) - - .installed.cfg - d bin - d develop-eggs - d eggs - d parts - - >>> ls(alt, 'develop-eggs') - - recipes.egg-link - - Logging control - --------------- - - Three buildout options are used to control logging: - - log-level - specifies the log level - - verbosity - adjusts the log level - - log-format - allows an alternate logging for mat to be specified - - We've already seen the log level and verbosity. Let's look at an example - of changing the format: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = - ... log-level = 25 - ... verbosity = 5 - ... log-format = %(levelname)s %(message)s - ... """) - - Here, we've changed the format to include the log-level name, rather - than the logger name. - - We've also illustrated, with a contrived example, that the log level - can be a numeric value and that the verbosity can be specified in the - configuration file. Because the verbosity is subtracted from the log - level, we get a final log level of 20, which is the INFO level. - - >>> print system(buildout), - INFO Develop: '/sample-buildout/recipes' - - Predefined buildout options - --------------------------- - - Buildouts have a number of predefined options that recipes can use - and that users can override in their configuration files. To see - these, we'll run a minimal buildout configuration with a debug logging - level. One of the features of debug logging is that the configuration - database is shown. - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... parts = - ... """) - - >>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE - Installing 'zc.buildout', 'setuptools'. - We have a develop egg: zc.buildout X.X. - We have the best distribution that satisfies 'setuptools'. - Picked: setuptools = V.V - <BLANKLINE> - Configuration data: - [buildout] - accept-buildout-test-releases = false - allow-hosts = * - allow-picked-versions = true - allowed-eggs-from-site-packages = * - bin-directory = /sample-buildout/bin - develop-eggs-directory = /sample-buildout/develop-eggs - directory = /sample-buildout - eggs-directory = /sample-buildout/eggs - exec-sitecustomize = true - executable = python - find-links = - include-site-packages = true - install-from-cache = false - installed = /sample-buildout/.installed.cfg - log-format = - log-level = INFO - newest = true - offline = false - parts = - parts-directory = /sample-buildout/parts - prefer-final = false - python = buildout - relative-paths = false - socket-timeout = - unzip = false - use-dependency-links = true - verbosity = 20 - <BLANKLINE> - - All of these options can be overridden by configuration files or by - command-line assignments. We've discussed most of these options - already, but let's review them and touch on some we haven't discussed: - - allowed-eggs-from-site-packages - Sometimes you need or want to control what eggs from site-packages are - used. The allowed-eggs-from-site-packages option allows you to specify a - whitelist of project names that may be included from site-packages. You - can use globs to specify the value. It defaults to a single value of '*', - indicating that any package may come from site-packages. - - Here's a usage example:: - - [buildout] - ... - - allowed-eggs-from-site-packages = - demo - bigdemo - zope.* - - This option interacts with the ``include-site-packages`` option in the - following ways. - - If ``include-site-packages`` is true, then - ``allowed-eggs-from-site-packages`` filters what eggs from site-packages - may be chosen. Therefore, if ``allowed-eggs-from-site-packages`` is an - empty list, then no eggs from site-packages are chosen, but site-packages - will still be included at the end of path lists. - - If ``include-site-packages`` is false, the value of - ``allowed-eggs-from-site-packages`` is irrelevant. - - See the ``include-site-packages`` description for more information. - - bin-directory - The directory path where scripts are written. This can be a - relative path, which is interpreted relative to the directory - option. - - develop-eggs-directory - The directory path where development egg links are created for software - being created in the local project. This can be a relative path, - which is interpreted relative to the directory option. - - directory - The buildout directory. This is the base for other buildout file - and directory locations, when relative locations are used. - - eggs-directory - The directory path where downloaded eggs are put. It is common to share - this directory across buildouts. Eggs in this directory should - *never* be modified. This can be a relative path, which is - interpreted relative to the directory option. - - exec-sitecustomize - Normally the Python's real sitecustomize module is processed. - If you do not want it to be processed in order to increase the - repeatability of your buildout, set this value to 'false'. This will - be honored irrespective of the setting for include-site-packages. - This option will be honored by some recipes and not others. - z3c.recipe.scripts honors this and zc.recipe.egg does not, for - instance. - - executable - The Python executable used to run the buildout. See the python - option below. - - include-site-packages - You can choose not to have the site-packages of the underlying Python - available to your script or interpreter, in addition to the packages - from your eggs. This can increase repeatability for your buildout. - This option will be better used by some recipes than others. - z3c.recipe.scripts honors this fully and zc.recipe.egg only - partially, for instance. - - installed - The file path where information about the results of the previous - buildout run is written. This can be a relative path, which is - interpreted relative to the directory option. This file provides - an inventory of installed parts with information needed to decide - which if any parts need to be uninstalled. - - log-format - The format used for logging messages. - - log-level - The log level before verbosity adjustment - - parts - A white space separated list of parts to be installed. - - parts-directory - A working directory that parts can used to store data. - - python - The name of a section containing information about the default - Python interpreter. Recipes that need a installation - typically have options to tell them which Python installation to - use. By convention, if a section-specific option isn't used, the - option is looked for in the buildout section. The option must - point to a section with an executable option giving the path to a - Python executable. By default, the buildout section defines the - default Python as the Python used to run the buildout. - - relative-paths - The paths generated by zc.buildout are absolute by default, and this - option is ``false``. However, if you set this value to be ``true``, - bin/buildout will be generated with code that makes the paths relative. - Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this - value as well. - - unzip - By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false"). - This follows the policy followed by setuptools itself. Experience shows - this policy to to be inconvenient. Zipped eggs make debugging more - difficult and often import more slowly. You can include an unzip option in - the buildout section to change the default unzipping policy ("unzip = - true"). - - use-dependency-links - By default buildout will obey the setuptools dependency_links metadata - when it looks for dependencies. This behavior can be controlled with - the use-dependency-links buildout option:: - - [buildout] - ... - use-dependency-links = false - - The option defaults to true. If you set it to false, then dependency - links are only looked for in the locations specified by find-links. - - verbosity - A log-level adjustment. Typically, this is set via the -q and -v - command-line options. - - - Creating new buildouts and bootstrapping - ---------------------------------------- - - If zc.buildout is installed, you can use it to create a new buildout - with it's own local copies of zc.buildout and setuptools and with - local buildout scripts. - - >>> sample_bootstrapped = tmpdir('sample-bootstrapped') - - >>> print system(buildout - ... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg') - ... +' init'), - Creating '/sample-bootstrapped/setup.cfg'. - Creating directory '/sample-bootstrapped/bin'. - Creating directory '/sample-bootstrapped/parts'. - Creating directory '/sample-bootstrapped/eggs'. - Creating directory '/sample-bootstrapped/develop-eggs'. - Generated script '/sample-bootstrapped/bin/buildout'. - - Note that a basic setup.cfg was created for us. - - >>> ls(sample_bootstrapped) - d bin - d develop-eggs - d eggs - d parts - - setup.cfg - - >>> ls(sample_bootstrapped, 'bin') - - buildout - - >>> _ = (ls(sample_bootstrapped, 'eggs'), - ... ls(sample_bootstrapped, 'develop-eggs')) - - setuptools-0.6-py2.3.egg - - zc.buildout-1.0-py2.3.egg - - (We list both the eggs and develop-eggs directories because the - buildout or setuptools egg could be installed in the develop-eggs - directory if the original buildout had develop eggs for either - buildout or setuptools.) - - If relative-paths is ``true``, the buildout script uses relative paths. - - >>> write(sample_bootstrapped, 'setup.cfg', - ... ''' - ... [buildout] - ... relative-paths = true - ... parts = - ... ''') - - >>> print system(buildout - ... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg') - ... +' bootstrap'), - Generated script '/sample-bootstrapped/bin/buildout'. - - >>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout') - >>> import sys - >>> if sys.platform.startswith('win'): - ... buildout_script += '-script.py' - >>> print open(buildout_script).read() # doctest: +ELLIPSIS - #!... -S - <BLANKLINE> - import os - <BLANKLINE> - join = os.path.join - base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) - base = os.path.dirname(base) - <BLANKLINE> - import sys - sys.path[0:0] = [ - join(base, 'parts/buildout'), - ] - <BLANKLINE> - <BLANKLINE> - import os - path = sys.path[0] - if os.environ.get('PYTHONPATH'): - path = os.pathsep.join([path, os.environ['PYTHONPATH']]) - os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') - os.environ['PYTHONPATH'] = path - import site # imports custom buildout-generated site.py - <BLANKLINE> - import zc.buildout.buildout - <BLANKLINE> - if __name__ == '__main__': - zc.buildout.buildout.main() - <BLANKLINE> - - - Note that, in the above two examples, the buildout script was installed - but not run. To run the buildout, we'd have to run the installed - buildout script. - - If we have an existing buildout that already has a buildout.cfg, we'll - normally use the bootstrap command instead of init. It will complain - if there isn't a configuration file: - - >>> sample_bootstrapped2 = tmpdir('sample-bootstrapped2') - - >>> print system(buildout - ... +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg') - ... +' bootstrap'), - While: - Initializing. - Error: Couldn't open /sample-bootstrapped2/setup.cfg - - >>> write(sample_bootstrapped2, 'setup.cfg', - ... """ - ... [buildout] - ... parts = - ... """) - - >>> print system(buildout - ... +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg') - ... +' bootstrap'), - Creating directory '/sample-bootstrapped2/bin'. - Creating directory '/sample-bootstrapped2/parts'. - Creating directory '/sample-bootstrapped2/eggs'. - Creating directory '/sample-bootstrapped2/develop-eggs'. - Generated script '/sample-bootstrapped2/bin/buildout'. - - - Newest and Offline Modes - ------------------------ - - By default buildout and recipes will try to find the newest versions - of distributions needed to satisfy requirements. This can be very - time consuming, especially when incrementally working on setting up a - buildout or working on a recipe. The buildout newest option can be - used to to suppress this. If the newest option is set to false, then - new distributions won't be sought if an installed distribution meets - requirements. The newest option can be set to false using the -N - command-line option. - - The offline option goes a bit further. If the buildout offline option - is given a value of "true", the buildout and recipes that are aware of - the option will avoid doing network access. This is handy when - running the buildout when not connected to the internet. It also - makes buildouts run much faster. This option is typically set using - the buildout -o option. - - Preferring Final Releases - ------------------------- - - Currently, when searching for new releases of your project's - dependencies, the newest available release is used. This isn't usually - ideal, as you may get a development release or alpha releases not ready - to be widely used. You can request that final releases be preferred - using the ``prefer-final`` option in the buildout section:: - - [buildout] - ... - prefer-final = true - - When the ``prefer-final`` option is set to true, then when searching for - new releases, final releases are preferred. If there are final - releases that satisfy distribution requirements, then those releases - are used even if newer non-final releases are available. - - In buildout version 2, all final releases will be preferred by - default--that is ``prefer-final`` will also default to 'true'. You will - then need to use a 'false' value for ``prefer-final`` to get the newest - releases. - - A separate option controls the behavior of the build system itself. - When buildout looks for recipes, extensions, and for updates to itself, - it does prefer final releases by default, as of the 1.5.0 release. The - ``accept-buildout-test-releases`` option will let you override this behavior. - However, it is typically changed by the --accept-buildout-test-releases - option to the bootstrap script, since bootstrapping is the first step to - selecting a buildout. - - Finding distributions - --------------------- - - By default, buildout searches the Python Package Index when looking - for distributions. You can, instead, specify your own index to search - using the `index` option:: - - [buildout] - ... - index = http://index.example.com/ - - This index, or the default of http://pypi.python.org/simple/ if no - index is specified, will always be searched for distributions unless - running buildout with options that prevent searching for - distributions. The latest version of the distribution that meets the - requirements of the buildout will always be used. - - You can also specify more locations to search for distributions using - the `find-links` option. All locations specified will be searched for - distributions along with the package index as described before. - - Locations can be urls:: - - [buildout] - ... - find-links = http://download.zope.org/distribution/ - - They can also be directories on disk:: - - [buildout] - ... - find-links = /some/path - - Finally, they can also be direct paths to distributions:: - - [buildout] - ... - find-links = /some/path/someegg-1.0.0-py2.3.egg - - Any number of locations can be specified in the `find-links` option:: - - [buildout] - ... - find-links = - http://download.zope.org/distribution/ - /some/otherpath - /some/path/someegg-1.0.0-py2.3.egg - - Dependency links - ---------------- - - By default buildout will obey the setuptools dependency_links metadata - when it looks for dependencies. This behavior can be controlled with - the use-dependency-links buildout option:: - - [buildout] - ... - use-dependency-links = false - - The option defaults to true. If you set it to false, then dependency - links are only looked for in the locations specified by find-links. - - Controlling the installation database - ------------------------------------- - - The buildout installed option is used to specify the file used to save - information on installed parts. This option is initialized to - ".installed.cfg", but it can be overridden in the configuration file - or on the command line: - - >>> write('buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = debug - ... - ... [debug] - ... recipe = recipes:debug - ... """) - - >>> print system(buildout+' buildout:installed=inst.cfg'), - Develop: '/sample-buildout/recipes' - Installing debug. - recipe recipes:debug - - >>> ls(sample_buildout) - - b1.cfg - - b2.cfg - - base.cfg - d bin - - buildout.cfg - d demo - d develop-eggs - d eggs - - inst.cfg - d parts - d recipes - - The installation database can be disabled by supplying an empty - buildout installed option: - - >>> os.remove('inst.cfg') - >>> print system(buildout+' buildout:installed='), - Develop: '/sample-buildout/recipes' - Installing debug. - recipe recipes:debug - - >>> ls(sample_buildout) - - b1.cfg - - b2.cfg - - base.cfg - d bin - - buildout.cfg - d demo - d develop-eggs - d eggs - d parts - d recipes - - - Note that there will be no installation database if there are no parts: - - >>> write('buildout.cfg', - ... """ - ... [buildout] - ... parts = - ... """) - - >>> print system(buildout+' buildout:installed=inst.cfg'), - - >>> ls(sample_buildout) - - b1.cfg - - b2.cfg - - base.cfg - d bin - - buildout.cfg - d demo - d develop-eggs - d eggs - d parts - d recipes - - Extensions - ---------- - - A feature allows code to be loaded and run after - configuration files have been read but before the buildout has begun - any processing. The intent is to allow special plugins such as - urllib2 request handlers to be loaded. - - To load an extension, we use the extensions option and list one or - more distribution requirements, on separate lines. The distributions - named will be loaded and any ``zc.buildout.extension`` entry points found - will be called with the buildout as an argument. When buildout - finishes processing, any ``zc.buildout.unloadextension`` entry points - found will be called with the buildout as an argument. - - Let's create a sample extension in our sample buildout created in the - previous section: - - >>> mkdir(sample_bootstrapped, 'demo') - - >>> write(sample_bootstrapped, 'demo', 'demo.py', - ... """ - ... def ext(buildout): - ... print 'ext', list(buildout) - ... def unload(buildout): - ... print 'unload', list(buildout) - ... """) - - >>> write(sample_bootstrapped, 'demo', 'setup.py', - ... """ - ... from setuptools import setup - ... - ... setup( - ... name = "demo", - ... entry_points = { - ... 'zc.buildout.extension': ['ext = demo:ext'], - ... 'zc.buildout.unloadextension': ['ext = demo:unload'], - ... }, - ... ) - ... """) - - Our extension just prints out the word 'demo', and lists the sections - found in the buildout passed to it. - - We'll update our buildout.cfg to list the demo directory as a develop - egg to be built: - - >>> write(sample_bootstrapped, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = demo - ... parts = - ... """) - - >>> os.chdir(sample_bootstrapped) - >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')), - Develop: '/sample-bootstrapped/demo' - - Now we can add the extensions option. We were a bit tricky and ran - the buildout once with the demo develop egg defined but without the - extension option. This is because extensions are loaded before the - buildout creates develop eggs. We needed to use a separate buildout - run to create the develop egg. Normally, when eggs are loaded from - the network, we wouldn't need to do anything special. - - >>> write(sample_bootstrapped, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = demo - ... extensions = demo - ... parts = - ... """) - - We see that our extension is loaded and executed: - - >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')), - ext ['buildout'] - Develop: '/sample-bootstrapped/demo' - unload ['buildout'] - - Allow hosts - ----------- - - On some environments the links visited by `zc.buildout` can be forbidden - by paranoiac firewalls. These URL might be on the chain of links - visited by `zc.buildout` wheter they are defined in the `find-links` option, - wheter they are defined by various eggs in their `url`, `download_url`, - `dependency_links` metadata. - - It is even harder to track that package_index works like a spider and - might visit links and go to other location. - - The `allow-hosts` option provides a way to prevent this, and - works exactly like the one provided in `easy_install`. - - You can provide a list of allowed host, together with wildcards:: - - [buildout] - ... - - allow-hosts = - *.python.org - example.com - - All urls that does not match these hosts will not be visited. - - .. [#future_recipe_methods] In the future, additional methods may be - added. Older recipes with fewer methods will still be - supported. - - .. [#packaging_info] If we wanted to create a distribution from this - package, we would need specify much more information. See the - `setuptools documentation - <http://peak.telecommunity.com/DevCenter/setuptools>`_. - - Always unzipping eggs - ===================== - - By default, zc.buildout doesn't unzip zip-safe eggs. - - >>> write('buildout.cfg', - ... ''' - ... [buildout] - ... parts = eggs - ... find-links = %(link_server)s - ... - ... [eggs] - ... recipe = zc.recipe.egg - ... eggs = demo - ... ''' % globals()) - - >>> _ = system(buildout) - >>> ls('eggs') - - demo-0.4c1-py2.4.egg - - demoneeded-1.2c1-py2.4.egg - d setuptools-0.6c8-py2.4.egg - - zc.buildout.egg-link - - This follows the policy followed by setuptools itself. Experience shows - this policy to to be inconvenient. Zipped eggs make debugging more - difficult and often import more slowly. - - You can include an unzip option in the buildout section to change the - default unzipping policy. - - >>> write('buildout.cfg', - ... ''' - ... [buildout] - ... parts = eggs - ... find-links = %(link_server)s - ... unzip = true - ... - ... [eggs] - ... recipe = zc.recipe.egg - ... eggs = demo - ... ''' % globals()) - - - >>> import os - >>> for name in os.listdir('eggs'): - ... if name.startswith('demo'): - ... remove('eggs', name) - - >>> _ = system(buildout) - >>> ls('eggs') - d demo-0.4c1-py2.4.egg - d demoneeded-1.2c1-py2.4.egg - d setuptools-0.6c8-py2.4.egg - - zc.buildout.egg-link - - Repeatable buildouts: controlling eggs used - =========================================== - - One of the goals of zc.buildout is to provide enough control to make - buildouts repeatable. It should be possible to check the buildout - configuration files for a project into a version control system and - later use the checked in files to get the same buildout, subject to - changes in the environment outside the buildout. - - An advantage of using Python eggs is that depenencies of eggs used are - automatically determined and used. The automatic inclusion of - depenent distributions is at odds with the goal of repeatable - buildouts. - - To support repeatable buildouts, a versions section can be created - with options for each distribution name whos version is to be fixed. - The section can then be specified via the buildout versions option. - - To see how this works, we'll create two versions of a recipe egg: - - >>> mkdir('recipe') - >>> write('recipe', 'recipe.py', - ... ''' - ... class Recipe: - ... def __init__(*a): pass - ... def install(self): - ... print 'recipe v1' - ... return () - ... update = install - ... ''') - - >>> write('recipe', 'setup.py', - ... ''' - ... from setuptools import setup - ... setup(name='spam', version='1', py_modules=['recipe'], - ... entry_points={'zc.buildout': ['default = recipe:Recipe']}, - ... ) - ... ''') - - >>> write('recipe', 'README', '') - - >>> print system(buildout+' setup recipe bdist_egg'), # doctest: +ELLIPSIS - Running setup script 'recipe/setup.py'. - ... - - >>> rmdir('recipe', 'build') - - >>> write('recipe', 'recipe.py', - ... ''' - ... class Recipe: - ... def __init__(*a): pass - ... def install(self): - ... print 'recipe v2' - ... return () - ... update = install - ... ''') - - >>> write('recipe', 'setup.py', - ... ''' - ... from setuptools import setup - ... setup(name='spam', version='2', py_modules=['recipe'], - ... entry_points={'zc.buildout': ['default = recipe:Recipe']}, - ... ) - ... ''') - - - >>> print system(buildout+' setup recipe bdist_egg'), # doctest: +ELLIPSIS - Running setup script 'recipe/setup.py'. - ... - - and we'll configure a buildout to use it: - - >>> write('buildout.cfg', - ... ''' - ... [buildout] - ... parts = foo - ... find-links = %s - ... - ... [foo] - ... recipe = spam - ... ''' % join('recipe', 'dist')) - - If we run the buildout, it will use version 2: - - >>> print system(buildout), - Getting distribution for 'spam'. - Got spam 2. - Installing foo. - recipe v2 - - We can specify a versions section that lists our recipe and name it in - the buildout section: - - >>> write('buildout.cfg', - ... ''' - ... [buildout] - ... parts = foo - ... find-links = %s - ... versions = release-1 - ... - ... [release-1] - ... spam = 1 - ... eggs = 2.2 - ... - ... [foo] - ... recipe = spam - ... ''' % join('recipe', 'dist')) - - Here we created a release-1 section listing the version 1 for the spam - distribution. We told the buildout to use it by specifying release-1 - as in the versions option. - - Now, if we run the buildout, we'll use version 1 of the spam recipe: - - >>> print system(buildout), - Getting distribution for 'spam==1'. - Got spam 1. - Uninstalling foo. - Installing foo. - recipe v1 - - Running the buildout in verbose mode will help us get information - about versions used. If we run the buildout in verbose mode without - specifying a versions section: - - >>> print system(buildout+' buildout:versions= -v'), # doctest: +ELLIPSIS - Installing 'zc.buildout', 'setuptools'. - We have a develop egg: zc.buildout 1.0.0. - We have the best distribution that satisfies 'setuptools'. - Picked: setuptools = 0.6 - Installing 'spam'. - We have the best distribution that satisfies 'spam'. - Picked: spam = 2. - Uninstalling foo. - Installing foo. - recipe v2 - - We'll get output that includes lines that tell us what versions - buildout chose a for us, like:: - - zc.buildout.easy_install.picked: spam = 2 - - This allows us to discover versions that are picked dynamically, so - that we can fix them in a versions section. - - If we run the buildout with the versions section: - - >>> print system(buildout+' -v'), # doctest: +ELLIPSIS - Installing 'zc.buildout', 'setuptools'. - We have a develop egg: zc.buildout 1.0.0. - We have the best distribution that satisfies 'setuptools'. - Picked: setuptools = 0.6 - Installing 'spam'. - We have the distribution that satisfies 'spam==1'. - Uninstalling foo. - Installing foo. - recipe v1 - - We won't get output for the spam distribution, which we didn't pick, - but we will get output for setuptools, which we didn't specify - versions for. - - You can request buildout to generate an error if it picks any - versions: - - >>> write('buildout.cfg', - ... ''' - ... [buildout] - ... parts = foo - ... find-links = %s - ... versions = release-1 - ... allow-picked-versions = false - ... - ... [release-1] - ... spam = 1 - ... eggs = 2.2 - ... - ... [foo] - ... recipe = spam - ... ''' % join('recipe', 'dist')) - - Using the download utility - ========================== - - The ``zc.buildout.download`` module provides a download utility that handles - the details of downloading files needed for a buildout run from the internet. - It downloads files to the local file system, using the download cache if - desired and optionally checking the downloaded files' MD5 checksum. - - We setup an HTTP server that provides a file we want to download: - - >>> server_data = tmpdir('sample_files') - >>> write(server_data, 'foo.txt', 'This is a foo text.') - >>> server_url = start_server(server_data) - - We also use a fresh directory for temporary files in order to make sure that - all temporary files have been cleaned up in the end: - - >>> import tempfile - >>> old_tempdir = tempfile.tempdir - >>> tempfile.tempdir = tmpdir('tmp') - - - Downloading without using the cache - ----------------------------------- - - If no download cache should be used, the download utility is instantiated - without any arguments: - - >>> from zc.buildout.download import Download - >>> download = Download() - >>> print download.cache_dir - None - - Downloading a file is achieved by calling the utility with the URL as an - argument. A tuple is returned that consists of the path to the downloaded copy - of the file and a boolean value indicating whether this is a temporary file - meant to be cleaned up during the same buildout run: - - >>> path, is_temp = download(server_url+'foo.txt') - >>> print path - /.../buildout-... - >>> cat(path) - This is a foo text. - - As we aren't using the download cache and haven't specified a target path - either, the download has ended up in a temporary file: - - >>> is_temp - True - - >>> import tempfile - >>> path.startswith(tempfile.gettempdir()) - True - - We are responsible for cleaning up temporary files behind us: - - >>> remove(path) - - When trying to access a file that doesn't exist, we'll get an exception: - - >>> try: download(server_url+'not-there') # doctest: +ELLIPSIS - ... except: print 'download error' - ... else: print 'woops' - download error - - Downloading a local file doesn't produce a temporary file but simply returns - the local file itself: - - >>> download(join(server_data, 'foo.txt')) - ('/sample_files/foo.txt', False) - - We can also have the downloaded file's MD5 sum checked: - - >>> try: from hashlib import md5 - ... except ImportError: from md5 import new as md5 - - >>> path, is_temp = download(server_url+'foo.txt', - ... md5('This is a foo text.').hexdigest()) - >>> is_temp - True - >>> remove(path) - - >>> download(server_url+'foo.txt', - ... md5('The wrong text.').hexdigest()) - Traceback (most recent call last): - ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt' - - The error message in the event of an MD5 checksum mismatch for a local file - reads somewhat differently: - - >>> download(join(server_data, 'foo.txt'), - ... md5('This is a foo text.').hexdigest()) - ('/sample_files/foo.txt', False) - - >>> download(join(server_data, 'foo.txt'), - ... md5('The wrong text.').hexdigest()) - Traceback (most recent call last): - ChecksumError: MD5 checksum mismatch for local resource at '/sample_files/foo.txt'. - - Finally, we can download the file to a specified place in the file system: - - >>> target_dir = tmpdir('download-target') - >>> path, is_temp = download(server_url+'foo.txt', - ... path=join(target_dir, 'downloaded.txt')) - >>> print path - /download-target/downloaded.txt - >>> cat(path) - This is a foo text. - >>> is_temp - False - - Trying to download a file in offline mode will result in an error: - - >>> download = Download(cache=None, offline=True) - >>> download(server_url+'foo.txt') - Traceback (most recent call last): - UserError: Couldn't download 'http://localhost/foo.txt' in offline mode. - - As an exception to this rule, file system paths and URLs in the ``file`` - scheme will still work: - - >>> cat(download(join(server_data, 'foo.txt'))[0]) - This is a foo text. - >>> cat(download('file:' + join(server_data, 'foo.txt'))[0]) - This is a foo text. - - >>> remove(path) - - - Downloading using the download cache - ------------------------------------ - - In order to make use of the download cache, we need to configure the download - utility differently. To do this, we pass a directory path as the ``cache`` - attribute upon instantiation: - - >>> cache = tmpdir('download-cache') - >>> download = Download(cache=cache) - >>> print download.cache_dir - /download-cache/ - - Simple usage - ~~~~~~~~~~~~ - - When using the cache, a file will be stored in the cache directory when it is - first downloaded. The file system path returned by the download utility points - to the cached copy: - - >>> ls(cache) - >>> path, is_temp = download(server_url+'foo.txt') - >>> print path - /download-cache/foo.txt - >>> cat(path) - This is a foo text. - >>> is_temp - False - - Whenever the file is downloaded again, the cached copy is used. Let's change - the file on the server to see this: - - >>> write(server_data, 'foo.txt', 'The wrong text.') - >>> path, is_temp = download(server_url+'foo.txt') - >>> print path - /download-cache/foo.txt - >>> cat(path) - This is a foo text. - - If we specify an MD5 checksum for a file that is already in the cache, the - cached copy's checksum will be verified: - - >>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest()) - Traceback (most recent call last): - ChecksumError: MD5 checksum mismatch for cached download - from 'http://localhost/foo.txt' at '/download-cache/foo.txt' - - Trying to access another file at a different URL which has the same base name - will result in the cached copy being used: - - >>> mkdir(server_data, 'other') - >>> write(server_data, 'other', 'foo.txt', 'The wrong text.') - >>> path, is_temp = download(server_url+'other/foo.txt') - >>> print path - /download-cache/foo.txt - >>> cat(path) - This is a foo text. - - Given a target path for the download, the utility will provide a copy of the - file at that location both when first downloading the file and when using a - cached copy: - - >>> remove(cache, 'foo.txt') - >>> ls(cache) - >>> write(server_data, 'foo.txt', 'This is a foo text.') - - >>> path, is_temp = download(server_url+'foo.txt', - ... path=join(target_dir, 'downloaded.txt')) - >>> print path - /download-target/downloaded.txt - >>> cat(path) - This is a foo text. - >>> is_temp - False - >>> ls(cache) - - foo.txt - - >>> remove(path) - >>> write(server_data, 'foo.txt', 'The wrong text.') - - >>> path, is_temp = download(server_url+'foo.txt', - ... path=join(target_dir, 'downloaded.txt')) - >>> print path - /download-target/downloaded.txt - >>> cat(path) - This is a foo text. - >>> is_temp - False - - In offline mode, downloads from any URL will be successful if the file is - found in the cache: - - >>> download = Download(cache=cache, offline=True) - >>> cat(download(server_url+'foo.txt')[0]) - This is a foo text. - - Local resources will be cached just like any others since download caches are - sometimes used to create source distributions: - - >>> remove(cache, 'foo.txt') - >>> ls(cache) - - >>> write(server_data, 'foo.txt', 'This is a foo text.') - >>> download = Download(cache=cache) - - >>> cat(download('file:' + join(server_data, 'foo.txt'), path=path)[0]) - This is a foo text. - >>> ls(cache) - - foo.txt - - >>> remove(cache, 'foo.txt') - - >>> cat(download(join(server_data, 'foo.txt'), path=path)[0]) - This is a foo text. - >>> ls(cache) - - foo.txt - - >>> remove(cache, 'foo.txt') - - However, resources with checksum mismatches will not be copied to the cache: - - >>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest()) - Traceback (most recent call last): - ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt' - >>> ls(cache) - - >>> remove(path) - - Finally, let's see what happens if the download cache to be used doesn't exist - as a directory in the file system yet: - - >>> Download(cache=join(cache, 'non-existent'))(server_url+'foo.txt') - Traceback (most recent call last): - UserError: The directory: - '/download-cache/non-existent' - to be used as a download cache doesn't exist. - - Using namespace sub-directories of the download cache - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - It is common to store cached copies of downloaded files within sub-directories - of the download cache to keep some degree of order. For example, zc.buildout - stores downloaded distributions in a sub-directory named "dist". Those - sub-directories are also known as namespaces. So far, we haven't specified any - namespaces to use, so the download utility stored files directly inside the - download cache. Let's use a namespace "test" instead: - - >>> download = Download(cache=cache, namespace='test') - >>> print download.cache_dir - /download-cache/test - - The namespace sub-directory hasn't been created yet: - - >>> ls(cache) - - Downloading a file now creates the namespace sub-directory and places a copy - of the file inside it: - - >>> path, is_temp = download(server_url+'foo.txt') - >>> print path - /download-cache/test/foo.txt - >>> ls(cache) - d test - >>> ls(cache, 'test') - - foo.txt - >>> cat(path) - This is a foo text. - >>> is_temp - False - - The next time we want to download that file, the copy from inside the cache - namespace is used. To see this clearly, we put a file with the same name but - different content both on the server and in the cache's root directory: - - >>> write(server_data, 'foo.txt', 'The wrong text.') - >>> write(cache, 'foo.txt', 'The wrong text.') - - >>> path, is_temp = download(server_url+'foo.txt') - >>> print path - /download-cache/test/foo.txt - >>> cat(path) - This is a foo text. - - >>> rmdir(cache, 'test') - >>> remove(cache, 'foo.txt') - >>> write(server_data, 'foo.txt', 'This is a foo text.') - - Using a hash of the URL as the filename in the cache - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - So far, the base name of the downloaded file read from the URL has been used - for the name of the cached copy of the file. This may not be desirable in some - cases, for example when downloading files from different locations that have - the same base name due to some naming convention, or if the file content - depends on URL parameters. In such cases, an MD5 hash of the complete URL may - be used as the filename in the cache: - - >>> download = Download(cache=cache, hash_name=True) - >>> path, is_temp = download(server_url+'foo.txt') - >>> print path - /download-cache/09f5793fcdc1716727f72d49519c688d - >>> cat(path) - This is a foo text. - >>> ls(cache) - - 09f5793fcdc1716727f72d49519c688d - - The path was printed just to illustrate matters; we cannot know the real - checksum since we don't know which port the server happens to listen at when - the test is run, so we don't actually know the full URL of the file. Let's - check that the checksum actually belongs to the particular URL used: - - >>> path.lower() == join(cache, md5(server_url+'foo.txt').hexdigest()).lower() - True - - The cached copy is used when downloading the file again: - - >>> write(server_data, 'foo.txt', 'The wrong text.') - >>> (path, is_temp) == download(server_url+'foo.txt') - True - >>> cat(path) - This is a foo text. - >>> ls(cache) - - 09f5793fcdc1716727f72d49519c688d - - If we change the URL, even in such a way that it keeps the base name of the - file the same, the file will be downloaded again this time and put in the - cache under a different name: - - >>> path2, is_temp = download(server_url+'other/foo.txt') - >>> print path2 - /download-cache/537b6d73267f8f4447586989af8c470e - >>> path == path2 - False - >>> path2.lower() == join(cache, md5(server_url+'other/foo.txt').hexdigest()).lower() - True - >>> cat(path) - This is a foo text. - >>> cat(path2) - The wrong text. - >>> ls(cache) - - 09f5793fcdc1716727f72d49519c688d - - 537b6d73267f8f4447586989af8c470e - - >>> remove(path) - >>> remove(path2) - >>> write(server_data, 'foo.txt', 'This is a foo text.') - - - Using the cache purely as a fall-back - ------------------------------------- - - Sometimes it is desirable to try downloading a file from the net if at all - possible, and use the cache purely as a fall-back option when a server is - down or if we are in offline mode. This mode is only in effect if a download - cache is configured in the first place: - - >>> download = Download(cache=cache, fallback=True) - >>> print download.cache_dir - /download-cache/ - - A downloaded file will be cached: - - >>> ls(cache) - >>> path, is_temp = download(server_url+'foo.txt') - >>> ls(cache) - - foo.txt - >>> cat(cache, 'foo.txt') - This is a foo text. - >>> is_temp - False - - If the file cannot be served, the cached copy will be used: - - >>> remove(server_data, 'foo.txt') - >>> try: Download()(server_url+'foo.txt') # doctest: +ELLIPSIS - ... except: print 'download error' - ... else: print 'woops' - download error - >>> path, is_temp = download(server_url+'foo.txt') - >>> cat(path) - This is a foo text. - >>> is_temp - False - - Similarly, if the file is served but we're in offline mode, we'll fall back to - using the cache: - - >>> write(server_data, 'foo.txt', 'The wrong text.') - >>> get(server_url+'foo.txt') - 'The wrong text.' - - >>> offline_download = Download(cache=cache, offline=True, fallback=True) - >>> path, is_temp = offline_download(server_url+'foo.txt') - >>> print path - /download-cache/foo.txt - >>> cat(path) - This is a foo text. - >>> is_temp - False - - However, when downloading the file normally with the cache being used in - fall-back mode, the file will be downloaded from the net and the cached copy - will be replaced with the new content: - - >>> cat(download(server_url+'foo.txt')[0]) - The wrong text. - >>> cat(cache, 'foo.txt') - The wrong text. - - When trying to download a resource whose checksum does not match, the cached - copy will neither be used nor overwritten: - - >>> write(server_data, 'foo.txt', 'This is a foo text.') - >>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest()) - Traceback (most recent call last): - ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt' - >>> cat(cache, 'foo.txt') - The wrong text. - - - Configuring the download utility from buildout options - ------------------------------------------------------ - - The configuration options explained so far derive from the build logic - implemented by the calling code. Other options configure the download utility - for use in a particular project or buildout run; they are read from the - ``buildout`` configuration section. The latter can be passed directly as the - first argument to the download utility's constructor. - - The location of the download cache is specified by the ``download-cache`` - option: - - >>> download = Download({'download-cache': cache}, namespace='cmmi') - >>> print download.cache_dir - /download-cache/cmmi - - If the ``download-cache`` option specifies a relative path, it is understood - relative to the current working directory, or to the buildout directory if - that is given: - - >>> download = Download({'download-cache': 'relative-cache'}) - >>> print download.cache_dir - /sample-buildout/relative-cache/ - - >>> download = Download({'directory': join(sample_buildout, 'root'), - ... 'download-cache': 'relative-cache'}) - >>> print download.cache_dir - /sample-buildout/root/relative-cache/ - - Keyword parameters take precedence over the corresponding options: - - >>> download = Download({'download-cache': cache}, cache=None) - >>> print download.cache_dir - None - - Whether to assume offline mode can be inferred from either the ``offline`` or - the ``install-from-cache`` option. As usual with zc.buildout, these options - must assume one of the values 'true' and 'false': - - >>> download = Download({'offline': 'true'}) - >>> download.offline - True - - >>> download = Download({'offline': 'false'}) - >>> download.offline - False - - >>> download = Download({'install-from-cache': 'true'}) - >>> download.offline - True - - >>> download = Download({'install-from-cache': 'false'}) - >>> download.offline - False - - These two options are combined using logical 'or': - - >>> download = Download({'offline': 'true', 'install-from-cache': 'false'}) - >>> download.offline - True - - >>> download = Download({'offline': 'false', 'install-from-cache': 'true'}) - >>> download.offline - True - - The ``offline`` keyword parameter takes precedence over both the ``offline`` - and ``install-from-cache`` options: - - >>> download = Download({'offline': 'true'}, offline=False) - >>> download.offline - False - - >>> download = Download({'install-from-cache': 'false'}, offline=True) - >>> download.offline - True - - - Regressions - ----------- - - MD5 checksum calculation needs to be reliable on all supported systems, which - requires text files to be treated as binary to avoid implicit line-ending - conversions: - - >>> text = 'First line of text.\r\nSecond line.\r\n' - >>> f = open(join(server_data, 'foo.txt'), 'wb') - >>> f.write(text) - >>> f.close() - >>> path, is_temp = Download()(server_url+'foo.txt', md5(text).hexdigest()) - >>> remove(path) - - - Clean up - -------- - - We should have cleaned up all temporary files created by downloading things: - - >>> ls(tempfile.tempdir) - - Reset the global temporary directory: - - >>> tempfile.tempdir = old_tempdir - - Using a download cache - ====================== - - Normally, when distributions are installed, if any processing is - needed, they are downloaded from the internet to a temporary directory - and then installed from there. A download cache can be used to avoid - the download step. This can be useful to reduce network access and to - create source distributions of an entire buildout. - - The buildout download-cache option can be used to specify a directory - to be used as a download cache. - - In this example, we'll create a directory to hold the cache: - - >>> cache = tmpdir('cache') - - And set up a buildout that downloads some eggs: - - >>> write('buildout.cfg', - ... ''' - ... [buildout] - ... parts = eggs - ... download-cache = %(cache)s - ... find-links = %(link_server)s - ... - ... [eggs] - ... recipe = zc.recipe.egg - ... eggs = demo ==0.2 - ... ''' % globals()) - - We specified a link server that has some distributions available for - download: - - >>> print get(link_server), - <html><body> - <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> - <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> - <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> - <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> - <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> - <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> - <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> - <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> - <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> - <a href="index/">index/</a><br> - <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> - </body></html> - - - We'll enable logging on the link server so we can see what's going on: - - >>> get(link_server+'enable_server_logging') - GET 200 /enable_server_logging - '' - - We also specified a download cache. - - If we run the buildout, we'll see the eggs installed from the link - server as usual: - - >>> print system(buildout), - GET 200 / - GET 200 /demo-0.2-py2.4.egg - GET 200 /demoneeded-1.2c1.zip - Installing eggs. - Getting distribution for 'demo==0.2'. - Got demo 0.2. - Getting distribution for 'demoneeded'. - Got demoneeded 1.2c1. - Generated script '/sample-buildout/bin/demo'. - - We'll also get the download cache populated. The buildout doesn't put - files in the cache directly. It creates an intermediate directory, - dist: - - - >>> ls(cache) - d dist - - >>> ls(cache, 'dist') - - demo-0.2-py2.4.egg - - demoneeded-1.2c1.zip - - If we remove the installed eggs from eggs directory and re-run the buildout: - - >>> import os - >>> for f in os.listdir('eggs'): - ... if f.startswith('demo'): - ... remove('eggs', f) - - >>> print system(buildout), - GET 200 / - Updating eggs. - Getting distribution for 'demo==0.2'. - Got demo 0.2. - Getting distribution for 'demoneeded'. - Got demoneeded 1.2c1. - - We see that the distributions aren't downloaded, because they're - downloaded from the cache. - - Installing solely from a download cache - --------------------------------------- - - A download cache can be used as the basis of application source - releases. In an application source release, we want to distribute an - application that can be built without making any network accesses. In - this case, we distribute a buildout with download cache and tell the - buildout to install from the download cache only, without making - network accesses. The buildout install-from-cache option can be used - to signal that packages should be installed only from the download - cache. - - Let's remove our installed eggs and run the buildout with the - install-from-cache option set to true: - - >>> for f in os.listdir('eggs'): - ... if f.startswith('demo'): - ... remove('eggs', f) - - >>> write('buildout.cfg', - ... ''' - ... [buildout] - ... parts = eggs - ... download-cache = %(cache)s - ... install-from-cache = true - ... find-links = %(link_server)s - ... - ... [eggs] - ... recipe = zc.recipe.egg - ... eggs = demo - ... ''' % globals()) - - >>> print system(buildout), - Uninstalling eggs. - Installing eggs. - Getting distribution for 'demo'. - Got demo 0.2. - Getting distribution for 'demoneeded'. - Got demoneeded 1.2c1. - Generated script '/sample-buildout/bin/demo'. - - Caching extended configuration - ============================== - - As mentioned in the general buildout documentation, configuration files can - extend each other, including the ability to download configuration being - extended from a URL. If desired, zc.buildout caches downloaded configuration - in order to be able to use it when run offline. - - As we're going to talk about downloading things, let's start an HTTP server. - Also, all of the following will take place inside the sample buildout. - - >>> server_data = tmpdir('server_data') - >>> server_url = start_server(server_data) - >>> cd(sample_buildout) - - We also use a fresh directory for temporary files in order to make sure that - all temporary files have been cleaned up in the end: - - >>> import tempfile - >>> old_tempdir = tempfile.tempdir - >>> tempfile.tempdir = tmpdir('tmp') - - - Basic use of the extends cache - ------------------------------ - - We put some base configuration on a server and reference it from a sample - buildout: - - >>> write(server_data, 'base.cfg', """\ - ... [buildout] - ... parts = - ... foo = bar - ... """) - - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = %sbase.cfg - ... """ % server_url) - - When trying to run this buildout offline, we'll find that we cannot read all - of the required configuration: - - >>> print system(buildout + ' -o') - While: - Initializing. - Error: Couldn't download 'http://localhost/base.cfg' in offline mode. - - Trying the same online, we can: - - >>> print system(buildout) - Unused options for buildout: 'foo'. - - As long as we haven't said anything about caching downloaded configuration, - nothing gets cached. Offline mode will still cause the buildout to fail: - - >>> print system(buildout + ' -o') - While: - Initializing. - Error: Couldn't download 'http://localhost/base.cfg' in offline mode. - - Let's now specify a cache for base configuration files. This cache is - different from the download cache used by recipes for caching distributions - and other files; one might, however, use a namespace subdirectory of the - download cache for it. The configuration cache we specify will be created when - running buildout and the base.cfg file will be put in it (with the file name - being a hash of the complete URL): - - >>> mkdir('cache') - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = %sbase.cfg - ... extends-cache = cache - ... """ % server_url) - - >>> print system(buildout) - Unused options for buildout: 'foo'. - - >>> cache = join(sample_buildout, 'cache') - >>> ls(cache) - - 5aedc98d7e769290a29d654a591a3a45 - - >>> import os - >>> cat(cache, os.listdir(cache)[0]) - [buildout] - parts = - foo = bar - - We can now run buildout offline as it will read base.cfg from the cache: - - >>> print system(buildout + ' -o') - Unused options for buildout: 'foo'. - - The cache is being used purely as a fall-back in case we are offline or don't - have access to a configuration file to be downloaded. As long as we are - online, buildout attempts to download a fresh copy of each file even if a - cached copy of the file exists. To see this, we put different configuration in - the same place on the server and run buildout in offline mode so it takes - base.cfg from the cache: - - >>> write(server_data, 'base.cfg', """\ - ... [buildout] - ... parts = - ... bar = baz - ... """) - - >>> print system(buildout + ' -o') - Unused options for buildout: 'foo'. - - In online mode, buildout will download and use the modified version: - - >>> print system(buildout) - Unused options for buildout: 'bar'. - - Trying offline mode again, the new version will be used as it has been put in - the cache now: - - >>> print system(buildout + ' -o') - Unused options for buildout: 'bar'. - - Clean up: - - >>> rmdir(cache) - - - Specifying extends cache and offline mode - ----------------------------------------- - - Normally, the values of buildout options such as the location of a download - cache or whether to use offline mode are determined by first reading the - user's default configuration, updating it with the project's configuration and - finally applying command-line options. User and project configuration are - assembled by reading a file such as ``~/.buildout/default.cfg``, - ``buildout.cfg`` or a URL given on the command line, recursively (depth-first) - downloading any base configuration specified by the ``buildout:extends`` - option read from each of those config files, and finally evaluating each - config file to provide default values for options not yet read. - - This works fine for all options that do not influence how configuration is - downloaded in the first place. The ``extends-cache`` and ``offline`` options, - however, are treated differently from the procedure described in order to make - it simple and obvious to see where a particular configuration file came from - under any particular circumstances. - - - Offline and extends-cache settings are read from the two root config files - exclusively. Otherwise one could construct configuration files that, when - read, imply that they should have been read from a different source than - they have. Also, specifying the extends cache within a file that might have - to be taken from the cache before being read wouldn't make a lot of sense. - - - Offline and extends-cache settings given by the user's defaults apply to the - process of assembling the project's configuration. If no extends cache has - been specified by the user's default configuration, the project's root - config file must be available, be it from disk or from the net. - - - Offline mode turned on by the ``-o`` command line option is honoured from - the beginning even though command line options are applied to the - configuration last. If offline mode is not requested by the command line, it - may be switched on by either the user's or the project's config root. - - Extends cache - ~~~~~~~~~~~~~ - - Let's see the above rules in action. We create a new home directory for our - user and write user and project configuration that recursively extends online - bases, using different caches: - - >>> mkdir('home') - >>> mkdir('home', '.buildout') - >>> mkdir('cache') - >>> mkdir('user-cache') - >>> os.environ['HOME'] = join(sample_buildout, 'home') - >>> write('home', '.buildout', 'default.cfg', """\ - ... [buildout] - ... extends = fancy_default.cfg - ... extends-cache = user-cache - ... """) - >>> write('home', '.buildout', 'fancy_default.cfg', """\ - ... [buildout] - ... extends = %sbase_default.cfg - ... """ % server_url) - >>> write(server_data, 'base_default.cfg', """\ - ... [buildout] - ... foo = bar - ... offline = false - ... """) - - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = fancy.cfg - ... extends-cache = cache - ... """) - >>> write('fancy.cfg', """\ - ... [buildout] - ... extends = %sbase.cfg - ... """ % server_url) - >>> write(server_data, 'base.cfg', """\ - ... [buildout] - ... parts = - ... offline = false - ... """) - - Buildout will now assemble its configuration from all of these 6 files, - defaults first. The online resources end up in the respective extends caches: - - >>> print system(buildout) - Unused options for buildout: 'foo'. - - >>> ls('user-cache') - - 10e772cf422123ef6c64ae770f555740 - >>> cat('user-cache', os.listdir('user-cache')[0]) - [buildout] - foo = bar - offline = false - - >>> ls('cache') - - c72213127e6eb2208a3e1fc1dba771a7 - >>> cat('cache', os.listdir('cache')[0]) - [buildout] - parts = - offline = false - - If, on the other hand, the extends caches are specified in files that get - extended themselves, they won't be used for assembling the configuration they - belong to (user's or project's, resp.). The extends cache specified by the - user's defaults does, however, apply to downloading project configuration. - Let's rewrite the config files, clean out the caches and re-run buildout: - - >>> write('home', '.buildout', 'default.cfg', """\ - ... [buildout] - ... extends = fancy_default.cfg - ... """) - >>> write('home', '.buildout', 'fancy_default.cfg', """\ - ... [buildout] - ... extends = %sbase_default.cfg - ... extends-cache = user-cache - ... """ % server_url) - - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = fancy.cfg - ... """) - >>> write('fancy.cfg', """\ - ... [buildout] - ... extends = %sbase.cfg - ... extends-cache = cache - ... """ % server_url) - - >>> remove('user-cache', os.listdir('user-cache')[0]) - >>> remove('cache', os.listdir('cache')[0]) - - >>> print system(buildout) - Unused options for buildout: 'foo'. - - >>> ls('user-cache') - - 0548bad6002359532de37385bb532e26 - >>> cat('user-cache', os.listdir('user-cache')[0]) - [buildout] - parts = - offline = false - - >>> ls('cache') - - Clean up: - - >>> rmdir('user-cache') - >>> rmdir('cache') - - Offline mode and installation from cache - ----------------------------~~~~~~~~~~~~ - - If we run buildout in offline mode now, it will fail because it cannot get at - the remote configuration file needed by the user's defaults: - - >>> print system(buildout + ' -o') - While: - Initializing. - Error: Couldn't download 'http://localhost/base_default.cfg' in offline mode. - - Let's now successively turn on offline mode by different parts of the - configuration and see when buildout applies this setting in each case: - - >>> write('home', '.buildout', 'default.cfg', """\ - ... [buildout] - ... extends = fancy_default.cfg - ... offline = true - ... """) - >>> print system(buildout) - While: - Initializing. - Error: Couldn't download 'http://localhost/base_default.cfg' in offline mode. - - >>> write('home', '.buildout', 'default.cfg', """\ - ... [buildout] - ... extends = fancy_default.cfg - ... """) - >>> write('home', '.buildout', 'fancy_default.cfg', """\ - ... [buildout] - ... extends = %sbase_default.cfg - ... offline = true - ... """ % server_url) - >>> print system(buildout) - While: - Initializing. - Error: Couldn't download 'http://localhost/base.cfg' in offline mode. - - >>> write('home', '.buildout', 'fancy_default.cfg', """\ - ... [buildout] - ... extends = %sbase_default.cfg - ... """ % server_url) - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = fancy.cfg - ... offline = true - ... """) - >>> print system(buildout) - While: - Initializing. - Error: Couldn't download 'http://localhost/base.cfg' in offline mode. - - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = fancy.cfg - ... """) - >>> write('fancy.cfg', """\ - ... [buildout] - ... extends = %sbase.cfg - ... offline = true - ... """ % server_url) - >>> print system(buildout) - Unused options for buildout: 'foo'. - - The ``install-from-cache`` option is treated accordingly: - - >>> write('home', '.buildout', 'default.cfg', """\ - ... [buildout] - ... extends = fancy_default.cfg - ... install-from-cache = true - ... """) - >>> print system(buildout) - While: - Initializing. - Error: Couldn't download 'http://localhost/base_default.cfg' in offline mode. - - >>> write('home', '.buildout', 'default.cfg', """\ - ... [buildout] - ... extends = fancy_default.cfg - ... """) - >>> write('home', '.buildout', 'fancy_default.cfg', """\ - ... [buildout] - ... extends = %sbase_default.cfg - ... install-from-cache = true - ... """ % server_url) - >>> print system(buildout) - While: - Initializing. - Error: Couldn't download 'http://localhost/base.cfg' in offline mode. - - >>> write('home', '.buildout', 'fancy_default.cfg', """\ - ... [buildout] - ... extends = %sbase_default.cfg - ... """ % server_url) - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = fancy.cfg - ... install-from-cache = true - ... """) - >>> print system(buildout) - While: - Initializing. - Error: Couldn't download 'http://localhost/base.cfg' in offline mode. - - >>> write('buildout.cfg', """\ - ... [buildout] - ... extends = fancy.cfg - ... """) - >>> write('fancy.cfg', """\ - ... [buildout] - ... extends = %sbase.cfg - ... install-from-cache = true - ... """ % server_url) - >>> print system(buildout) - While: - Installing. - Checking for upgrades. - An internal error occurred ... - ValueError: install_from_cache set to true with no download cache - - - Clean up - -------- - - We should have cleaned up all temporary files created by downloading things: - - >>> ls(tempfile.tempdir) - - Reset the global temporary directory: - - >>> tempfile.tempdir = old_tempdir - - Using zc.buildout to run setup scripts - ====================================== - - zc buildout has a convenience command for running setup scripts. Why? - There are two reasons. If a setup script doesn't import setuptools, - you can't use any setuptools-provided commands, like bdist_egg. When - buildout runs a setup script, it arranges to import setuptools before - running the script so setuptools-provided commands are available. - - If you use a squeaky-clean Python to do your development, the setup - script that would import setuptools because setuptools isn't in the - path. Because buildout requires setuptools and knows where it has - installed a setuptools egg, it adds the setuptools egg to the Python - path before running the script. To run a setup script, use the - buildout setup command, passing the name of a script or a directory - containing a setup script and arguments to the script. Let's look at - an example: - - >>> mkdir('test') - >>> cd('test') - >>> write('setup.py', - ... ''' - ... from distutils.core import setup - ... setup(name='sample') - ... ''') - - We've created a super simple (stupid) setup script. Note that it - doesn't import setuptools. Let's try running it to create an egg. - We'll use the buildout script from our sample buildout: - - >>> print system(buildout+' setup'), - ... # doctest: +NORMALIZE_WHITESPACE - Error: The setup command requires the path to a setup script or - directory containing a setup script, and its arguments. - - Oops, we forgot to give the name of the setup script: - - >>> print system(buildout+' setup setup.py bdist_egg'), - ... # doctest: +ELLIPSIS - Running setup script 'setup.py'. - ... - - >>> ls('dist') - - sample-0.0.0-py2.5.egg - - Note that we can specify a directory name. This is often shorter and - preferred by the lazy :) - - >>> print system(buildout+' setup . bdist_egg'), # doctest: +ELLIPSIS - Running setup script './setup.py'. - ... - - Automatic Buildout Updates - ========================== - - When a buildout is run, one of the first steps performed is to check - for updates to either zc.buildout or setuptools. To demonstrate this, - we've created some "new releases" of buildout and setuptools in a - new_releases folder: - - >>> ls(new_releases) - d setuptools - - setuptools-99.99-py2.4.egg - d zc.buildout - - zc.buildout-100.0b1-pyN.N.egg - - zc.buildout-99.99-py2.4.egg - - Let's update the sample buildout.cfg to look in this area: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... find-links = %(new_releases)s - ... index = %(new_releases)s - ... parts = show-versions - ... develop = showversions - ... - ... [show-versions] - ... recipe = showversions - ... """ % dict(new_releases=new_releases)) - - We'll also include a recipe that echos the versions of setuptools and - zc.buildout used: - - >>> mkdir(sample_buildout, 'showversions') - - >>> write(sample_buildout, 'showversions', 'showversions.py', - ... """ - ... import pkg_resources - ... - ... class Recipe: - ... - ... def __init__(self, buildout, name, options): - ... pass - ... - ... def install(self): - ... for project in 'zc.buildout', 'setuptools': - ... req = pkg_resources.Requirement.parse(project) - ... print project, pkg_resources.working_set.find(req).version - ... return () - ... update = install - ... """) - - - >>> write(sample_buildout, 'showversions', 'setup.py', - ... """ - ... from setuptools import setup - ... - ... setup( - ... name = "showversions", - ... entry_points = {'zc.buildout': ['default = showversions:Recipe']}, - ... ) - ... """) - - - Now if we run the buildout, the buildout will upgrade itself to the - new versions found in new releases: - - >>> print system(buildout), - Getting distribution for 'zc.buildout'. - Got zc.buildout 99.99. - Getting distribution for 'setuptools'. - Got setuptools 99.99. - Upgraded: - zc.buildout version 99.99, - setuptools version 99.99; - restarting. - Generated script '/sample-buildout/bin/buildout'. - Develop: '/sample-buildout/showversions' - Installing show-versions. - zc.buildout 99.99 - setuptools 99.99 - - Notice that, even though we have a newer beta version of zc.buildout - available, the final "99.99" was selected. If you want to get non-final - versions, specify a specific version in your buildout's versions - section, you typically want to use the --accept-buildout-test-releases - option to the bootstrap script, which internally uses the - ``accept-buildout-test-releases = true`` discussed below. - - Our buildout script's site.py has been updated to use the new eggs: - - >>> cat(sample_buildout, 'parts', 'buildout', 'site.py') - ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS - "... - def addsitepackages(known_paths): - """Add site packages, as determined by zc.buildout. - <BLANKLINE> - See original_addsitepackages, below, for the original version.""" - setuptools_path = '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg' - sys.path.append(setuptools_path) - known_paths.add(os.path.normcase(setuptools_path)) - import pkg_resources - buildout_paths = [ - '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg', - '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg' - ] - for path in buildout_paths: - sitedir, sitedircase = makepath(path) - if not sitedircase in known_paths and os.path.exists(sitedir): - sys.path.append(sitedir) - known_paths.add(sitedircase) - pkg_resources.working_set.add_entry(sitedir) - sys.__egginsert = len(buildout_paths) # Support setuptools. - original_paths = [ - ... - ] - for path in original_paths: - if path == setuptools_path or path not in known_paths: - addsitedir(path, known_paths) - return known_paths - ... - - Now, let's recreate the sample buildout. If we specify constraints on - the versions of zc.buildout and setuptools (or distribute) to use, - running the buildout will install earlier versions of these packages: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... find-links = %(new_releases)s - ... index = %(new_releases)s - ... parts = show-versions - ... develop = showversions - ... zc.buildout-version = < 99 - ... setuptools-version = < 99 - ... distribute-version = < 99 - ... - ... [show-versions] - ... recipe = showversions - ... """ % dict(new_releases=new_releases)) - - Now we can see that we actually "upgrade" to an earlier version. - - >>> print system(buildout), - Upgraded: - zc.buildout version 1.0.0, - setuptools version 0.6; - restarting. - Develop: '/sample-buildout/showversions' - Updating show-versions. - zc.buildout 1.0.0 - setuptools 0.6 - - There are a number of cases, described below, in which the updates - don't happen. - - We won't upgrade in offline mode: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... find-links = %(new_releases)s - ... index = %(new_releases)s - ... parts = show-versions - ... develop = showversions - ... - ... [show-versions] - ... recipe = showversions - ... """ % dict(new_releases=new_releases)) - - >>> print system(buildout+' -o'), - Develop: '/sample-buildout/showversions' - Updating show-versions. - zc.buildout 1.0.0 - setuptools 0.6 - - Or in non-newest mode: - - >>> print system(buildout+' -N'), - Develop: '/sample-buildout/showversions' - Updating show-versions. - zc.buildout 1.0.0 - setuptools 0.6 - - We also won't upgrade if the buildout script being run isn't in the - buildout's bin directory. To see this we'll create a new buildout - directory: - - >>> sample_buildout2 = tmpdir('sample_buildout2') - >>> write(sample_buildout2, 'buildout.cfg', - ... """ - ... [buildout] - ... find-links = %(new_releases)s - ... index = %(new_releases)s - ... parts = - ... """ % dict(new_releases=new_releases)) - - >>> cd(sample_buildout2) - >>> print system(buildout), - Creating directory '/sample_buildout2/bin'. - Creating directory '/sample_buildout2/parts'. - Creating directory '/sample_buildout2/eggs'. - Creating directory '/sample_buildout2/develop-eggs'. - Getting distribution for 'zc.buildout'. - Got zc.buildout 99.99. - Getting distribution for 'setuptools'. - Got setuptools 99.99. - Not upgrading because not running a local buildout command. - - >>> ls('bin') - - As mentioned above, the ``accept-buildout-test-releases = true`` means that - newer non-final versions of these dependencies are preferred. Typically - users are not expected to actually manipulate this value. Instead, the - bootstrap script creates a buildout buildout script that passes in the - value as a command line override. This then results in the buildout - script being rewritten to remember the decision. - - We'll mimic this by passing the argument actually in the command line. - - >>> cd(sample_buildout) - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... find-links = %(new_releases)s - ... index = %(new_releases)s - ... parts = show-versions - ... develop = showversions - ... - ... [show-versions] - ... recipe = showversions - ... """ % dict(new_releases=new_releases)) - - >>> print system(buildout + - ... ' buildout:accept-buildout-test-releases=true'), - ... # doctest: +NORMALIZE_WHITESPACE - Getting distribution for 'zc.buildout'. - Got zc.buildout 100.0b1. - Upgraded: - zc.buildout version 100.0b1, - setuptools version 99.99; - restarting. - Generated script '/sample-buildout/bin/buildout'. - NOTE: Accepting early releases of build system packages. Rerun bootstrap - without --accept-buildout-test-releases (-t) to return to default - behavior. - Develop: '/sample-buildout/showversions' - Updating show-versions. - zc.buildout 100.0b1 - setuptools 99.99 - - The buildout script shows the change. - - >>> buildout_script = join(sample_buildout, 'bin', 'buildout') - >>> import sys - >>> if sys.platform.startswith('win'): - ... buildout_script += '-script.py' - >>> print open(buildout_script).read() # doctest: +ELLIPSIS - #... - sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true') - print ('NOTE: Accepting early releases of build system packages. Rerun ' - 'bootstrap without --accept-buildout-test-releases (-t) to return to ' - 'default behavior.') - ... - - Debugging buildouts - =================== - - Buildouts can be pretty complex. When things go wrong, it isn't - always obvious why. Errors can occur due to problems in user input or - due to bugs in zc.buildout or recipes. When an error occurs, Python's - post-mortem debugger can be used to inspect the state of the buildout - or recipe code where the error occurred. To enable this, use the -D - option to the buildout. Let's create a recipe that has a bug: - - >>> mkdir(sample_buildout, 'recipes') - - >>> write(sample_buildout, 'recipes', 'mkdir.py', - ... """ - ... import os, zc.buildout - ... - ... class Mkdir: - ... - ... def __init__(self, buildout, name, options): - ... self.name, self.options = name, options - ... options['path'] = os.path.join( - ... buildout['buildout']['directory'], - ... options['path'], - ... ) - ... - ... def install(self): - ... directory = self.options['directory'] - ... os.mkdir(directory) - ... return directory - ... - ... def update(self): - ... pass - ... """) - - >>> write(sample_buildout, 'recipes', 'setup.py', - ... """ - ... from setuptools import setup - ... - ... setup(name = "recipes", - ... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']}, - ... ) - ... """) - - And create a buildout that uses it: - - >>> write(sample_buildout, 'buildout.cfg', - ... """ - ... [buildout] - ... develop = recipes - ... parts = data-dir - ... - ... [data-dir] - ... recipe = recipes:mkdir - ... path = mystuff - ... """) - - If we run the buildout, we'll get an error: - - >>> print system(buildout), - Develop: '/sample-buildout/recipes' - Installing data-dir. - While: - Installing data-dir. - Error: Missing option: data-dir:directory - - - If we want to debug the error, we can add the -D option. Here's we'll - supply some input: - - >>> print system(buildout+" -D", """\ - ... up - ... p self.options.keys() - ... q - ... """), - Develop: '/sample-buildout/recipes' - Installing data-dir. - > /zc/buildout/buildout.py(925)__getitem__() - -> raise MissingOption("Missing option: %s:%s" % (self.name, key)) - (Pdb) > /sample-buildout/recipes/mkdir.py(14)install() - -> directory = self.options['directory'] - (Pdb) ['path', 'recipe'] - (Pdb) While: - Installing data-dir. - Traceback (most recent call last): - File "/zc/buildout/buildout.py", line 1352, in main - getattr(buildout, command)(args) - File "/zc/buildout/buildout.py", line 383, in install - installed_files = self[part]._call(recipe.install) - File "/zc/buildout/buildout.py", line 961, in _call - return f() - File "/sample-buildout/recipes/mkdir.py", line 14, in install - directory = self.options['directory'] - File "/zc/buildout/buildout.py", line 925, in __getitem__ - raise MissingOption("Missing option: %s:%s" % (self.name, key)) - MissingOption: Missing option: data-dir:directory - <BLANKLINE> - Starting pdb: - - Testing Support - =============== - - The zc.buildout.testing module provides an API that can be used when - writing recipe tests. This API is documented below. Many examples of - using this API can be found in the zc.buildout, zc.recipe.egg, and - zc.recipe.testrunner tests. - - zc.buildout.testing.buildoutSetUp(test) - --------------------------------------- - - The buildoutSetup function can be used as a doctest setup function. - It creates a sample buildout that can be used by tests, changing the - current working directory to the sample_buildout. It also adds a - number of names to the test namespace: - - ``sample_buildout`` - This is the name of a buildout with a basic configuration. - - ``buildout`` - This is the path of the buildout script in the sample buildout. - - ``ls(*path)`` - List the contents of a directory. The directory path is provided as one or - more strings, to be joined with os.path.join. - - ``cat(*path)`` - Display the contents of a file. The file path is provided as one or - more strings, to be joined with os.path.join. - - On Windows, if the file doesn't exist, the function will try - adding a '-script.py' suffix. This helps to work around a - difference in script generation on windows. - - ``mkdir(*path)`` - Create a directory. The directory path is provided as one or - more strings, to be joined with os.path.join. - - ``rmdir(*path)`` - Remove a directory. The directory path is provided as one or - more strings, to be joined with os.path.join. - - ``remove(*path)`` - Remove a directory or file. The path is provided as one or - more strings, to be joined with os.path.join. - - ``tmpdir(name)`` - Create a temporary directory with the given name. The directory - will be automatically removed at the end of the test. The path of - the created directory is returned. - - Further, if the the normalize_path normlaizing substitution (see - below) is used, then any paths starting with this path will be - normalized to:: - - /name/restofpath - - No two temporary directories can be created with the same name. A - directory created with tmpdir can be removed with rmdir and recreated. - - Note that the sample_buildout directory is created by calling this - function. - - ``write(*path_and_contents)`` - Create a file. The file path is provided as one or more strings, - to be joined with os.path.join. The last argument is the file contents. - - ``system(command, input='')`` - Execute a system command with the given input passed to the - command's standard input. The output (error and regular output) - from the command is returned. - - ``get(url)`` - Get a web page. - - ``cd(*path)`` - Change to the given directory. The directory path is provided as one or - more strings, to be joined with os.path.join. - - The directory will be reset at the end of the test. - - ``join(*path)`` - A convenient reference to os.path.join. - - ``register_teardown(func)`` - Register a tear-down function. The function will be called with - no arguments at the end of the test. - - ``start_server(path)`` - Start a web server on the given path. The server will be shut - down at the end of the test. The server URL is returned. - - You can cause the server to start and stop logging it's output - using: - - >>> get(server_url+'enable_server_logging') - - and: - - >>> get(server_url+'disable_server_logging') - - This can be useful to see how buildout is interacting with a - server. - - - ``sdist(setup, dest)`` - Create a source distribution by running the given setup file and - placing the result in the given destination directory. If the - setup argument is a directory, the thge setup.py file in that - directory is used. - - ``bdist_egg(setup, executable, dest)`` - Create an egg by running the given setup file with the given - Python executable and placing the result in the given destination - directory. If the setup argument is a directory, then the - setup.py file in that directory is used. - - ``find_python(version)`` - Find a Python executable for the given version, where version is a - string like "2.4". - - This function uses the following strategy to find a Python of the - given version: - - - Look for an environment variable of the form PYTHON%(version)s. - - - On windows, look for \Pythonm%(version)s\python - - - on Unix, try running python%(version)s or just python to get the - executable - - ``zc.buildout.testing.buildoutTearDown(test)`` - ---------------------------------------------- - - Tear down everything set up by zc.buildout.testing.buildoutSetUp. Any - functions passed to register_teardown are called as well. - - ``install(project, destination)`` - --------------------------------- - - Install eggs for a given project into a destination. If the - destination is a test object, then the eggs directory of the - sample buildout (sample_buildout) defined by the test will be used. - Tests will use this to install the distributions for the packages - being tested (and their dependencies) into a sample buildout. The egg - to be used should already be loaded, by importing one of the modules - provided, before calling this function. - - ``install_develop(project, destination)`` - ----------------------------------------- - - Like install, but a develop egg is installed even if the current egg - if not a develop egg. - - ``Output normalization`` - ------------------------ - - Recipe tests often generate output that is dependent on temporary file - locations, operating system conventions or Python versions. To deal - with these dependencies, we often use - zope.testing.renormalizing.RENormalizing to normalize test output. - zope.testing.renormalizing.RENormalizing takes pairs of regular - expressions and substitutions. The zc.buildout.testing module provides - a few helpful variables that define regular-expression/substitution - pairs that you can pass to zope.testing.renormalizing.RENormalizing. - - - ``normalize_path`` - Converts tests paths, based on directories created with tmpdir(), - to simple paths. - - ``normalize_script`` - On Unix-like systems, scripts are implemented in single files - without suffixes. On windows, scripts are implemented with 2 - files, a -script.py file and a .exe file. This normalization - converts directory listings of Windows scripts to the form - generated on UNix-like systems. - - ``normalize_egg_py`` - Normalize Python version and platform indicators, if specified, in - egg names. - - Python API for egg and script installation - ========================================== - - The easy_install module provides some functions to provide support for - egg and script installation. It provides functionality at the python - level that is similar to easy_install, with a few exceptions: - - - By default, we look for new packages *and* the packages that - they depend on. This is somewhat like (and uses) the --upgrade - option of easy_install, except that we also upgrade required - packages. - - - If the highest-revision package satisfying a specification is - already present, then we don't try to get another one. This saves a - lot of search time in the common case that packages are pegged to - specific versions. - - - If there is a develop egg that satisfies a requirement, we don't - look for additional distributions. We always give preference to - develop eggs. - - - Distutils options for building extensions can be passed. - - Distribution installation - ------------------------- - - The easy_install module provides a function, install, for installing one - or more packages and their dependencies. The install function takes 2 - positional arguments: - - - An iterable of setuptools requirement strings for the distributions - to be installed, and - - - A destination directory to install to and to satisfy requirements - from. The destination directory can be None, in which case, no new - distributions are downloaded and there will be an error if the - needed distributions can't be found among those already installed. - - It supports a number of optional keyword arguments: - - links - A sequence of URLs, file names, or directories to look for - links to distributions. - - index - The URL of an index server, or almost any other valid URL. :) - - If not specified, the Python Package Index, - http://pypi.python.org/simple/, is used. You can specify an - alternate index with this option. If you use the links option and - if the links point to the needed distributions, then the index can - be anything and will be largely ignored. In the examples, here, - we'll just point to an empty directory on our link server. This - will make our examples run a little bit faster. - - executable - A path to a Python executable. Distributions will be installed - using this executable and will be for the matching Python version. - - path - A list of additional directories to search for locally-installed - distributions. - - always_unzip - A flag indicating that newly-downloaded distributions should be - directories even if they could be installed as zip files. - - working_set - An existing working set to be augmented with additional - distributions, if necessary to satisfy requirements. This allows - you to call install multiple times, if necessary, to gather - multiple sets of requirements. - - newest - A boolean value indicating whether to search for new distributions - when already-installed distributions meet the requirement. When - this is true, the default, and when the destination directory is - not None, then the install function will search for the newest - distributions that satisfy the requirements. - - versions - A dictionary mapping project names to version numbers to be used - when selecting distributions. This can be used to specify a set of - distribution versions independent of other requirements. - - use_dependency_links - A flag indicating whether to search for dependencies using the - setup dependency_links metadata or not. If true, links are searched - for using dependency_links in preference to other - locations. Defaults to true. - - include_site_packages - A flag indicating whether Python's non-standard-library packages should - be available for finding dependencies. Defaults to true. - - Paths outside of Python's standard library--or more precisely, those that - are not included when Python is started with the -S argument--are loosely - referred to as "site-packages" here. - - relative_paths - Adjust egg paths so they are relative to the script path. This - allows scripts to work when scripts and eggs are moved, as long as - they are both moved in the same way. - - The install method returns a working set containing the distributions - needed to meet the given requirements. - - We have a link server that has a number of eggs: - - >>> print get(link_server), - <html><body> - <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> - <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> - <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> - <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> - <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> - <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> - <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> - <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> - <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> - <a href="index/">index/</a><br> - <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> - </body></html> - - Let's make a directory and install the demo egg to it, using the demo: - - >>> dest = tmpdir('sample-install') - >>> import zc.buildout.easy_install - >>> ws = zc.buildout.easy_install.install( - ... ['demo==0.2'], dest, - ... links=[link_server], index=link_server+'index/') - - We requested version 0.2 of the demo distribution to be installed into - the destination server. We specified that we should search for links - on the link server and that we should use the (empty) link server - index directory as a package index. - - The working set contains the distributions we retrieved. - - >>> for dist in ws: - ... print dist - demo 0.2 - demoneeded 1.1 - - We got demoneeded because it was a dependency of demo. - - And the actual eggs were added to the eggs directory. - - >>> ls(dest) - - demo-0.2-py2.4.egg - - demoneeded-1.1-py2.4.egg - - If we remove the version restriction on demo, but specify a false - value for newest, no new distributions will be installed: - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... newest=False) - >>> ls(dest) - - demo-0.2-py2.4.egg - - demoneeded-1.1-py2.4.egg - - If we leave off the newest option, we'll get an update for demo: - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/') - >>> ls(dest) - - demo-0.2-py2.4.egg - - demo-0.3-py2.4.egg - - demoneeded-1.1-py2.4.egg - - Note that we didn't get the newest versions available. There were - release candidates for newer versions of both packages. By default, - final releases are preferred. We can change this behavior using the - prefer_final function: - - >>> zc.buildout.easy_install.prefer_final(False) - True - - The old setting is returned. - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/') - >>> for dist in ws: - ... print dist - demo 0.4c1 - demoneeded 1.2c1 - - >>> ls(dest) - - demo-0.2-py2.4.egg - - demo-0.3-py2.4.egg - - demo-0.4c1-py2.4.egg - - demoneeded-1.1-py2.4.egg - - demoneeded-1.2c1-py2.4.egg - - Let's put the setting back to the default. - - >>> zc.buildout.easy_install.prefer_final(True) - False - - We can supply additional distributions. We can also supply - specifications for distributions that would normally be found via - dependencies. We might do this to specify a specific version. - - >>> ws = zc.buildout.easy_install.install( - ... ['demo', 'other', 'demoneeded==1.0'], dest, - ... links=[link_server], index=link_server+'index/') - - >>> for dist in ws: - ... print dist - demo 0.3 - other 1.0 - demoneeded 1.0 - - >>> ls(dest) - - demo-0.2-py2.4.egg - - demo-0.3-py2.4.egg - - demo-0.4c1-py2.4.egg - - demoneeded-1.0-py2.4.egg - - demoneeded-1.1-py2.4.egg - - demoneeded-1.2c1-py2.4.egg - d other-1.0-py2.4.egg - - We can request that eggs be unzipped even if they are zip safe. This - can be useful when debugging. (Note that Distribute will unzip eggs by - default, so if you are using Distribute, most or all eggs will already be - unzipped without this flag.) - - >>> rmdir(dest) - >>> dest = tmpdir('sample-install') - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... always_unzip=True) - - >>> ls(dest) - d demo-0.3-py2.4.egg - d demoneeded-1.1-py2.4.egg - - >>> rmdir(dest) - >>> dest = tmpdir('sample-install') - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... always_unzip=False) - - >>> ls(dest) - - demo-0.3-py2.4.egg - - demoneeded-1.1-py2.4.egg - - We can also set a default by calling the always_unzip function: - - >>> zc.buildout.easy_install.always_unzip(True) - False - - The old default is returned: - - >>> rmdir(dest) - >>> dest = tmpdir('sample-install') - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/') - - >>> ls(dest) - d demo-0.3-py2.4.egg - d demoneeded-1.1-py2.4.egg - - - >>> zc.buildout.easy_install.always_unzip(False) - True - - >>> rmdir(dest) - >>> dest = tmpdir('sample-install') - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/') - - >>> ls(dest) - - demo-0.3-py2.4.egg - - demoneeded-1.1-py2.4.egg - - >>> rmdir(dest) - >>> dest = tmpdir('sample-install') - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... always_unzip=True) - - >>> ls(dest) - d demo-0.3-py2.4.egg - d demoneeded-1.1-py2.4.egg - - Specifying version information independent of requirements - ---------------------------------------------------------- - - Sometimes it's useful to specify version information independent of - normal requirements specifications. For example, a buildout may need - to lock down a set of versions, without having to put put version - numbers in setup files or part definitions. If a dictionary is passed - to the install function, mapping project names to version numbers, - then the versions numbers will be used. - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... versions = dict(demo='0.2', demoneeded='1.0')) - >>> [d.version for d in ws] - ['0.2', '1.0'] - - In this example, we specified a version for demoneeded, even though we - didn't define a requirement for it. The versions specified apply to - dependencies as well as the specified requirements. - - If we specify a version that's incompatible with a requirement, then - we'll get an error: - - >>> from zope.testing.loggingsupport import InstalledHandler - >>> handler = InstalledHandler('zc.buildout.easy_install') - >>> import logging - >>> logging.getLogger('zc.buildout.easy_install').propagate = False - - >>> ws = zc.buildout.easy_install.install( - ... ['demo >0.2'], dest, links=[link_server], - ... index=link_server+'index/', - ... versions = dict(demo='0.2', demoneeded='1.0')) - Traceback (most recent call last): - ... - IncompatibleVersionError: Bad version 0.2 - - >>> print handler - zc.buildout.easy_install DEBUG - Installing 'demo >0.2'. - zc.buildout.easy_install ERROR - The version, 0.2, is not consistent with the requirement, 'demo>0.2'. - - >>> handler.clear() - - If no versions are specified, a debugging message will be output - reporting that a version was picked automatically: - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... ) - - >>> print handler - zc.buildout.easy_install DEBUG - Installing 'demo'. - zc.buildout.easy_install DEBUG - We have the best distribution that satisfies 'demo'. - zc.buildout.easy_install DEBUG - Picked: demo = 0.3 - zc.buildout.easy_install DEBUG - Getting required 'demoneeded' - zc.buildout.easy_install DEBUG - required by demo 0.3. - zc.buildout.easy_install DEBUG - We have the best distribution that satisfies 'demoneeded'. - zc.buildout.easy_install DEBUG - Picked: demoneeded = 1.1 - - >>> handler.uninstall() - >>> logging.getLogger('zc.buildout.easy_install').propagate = True - - We can request that we get an error if versions are picked: - - >>> zc.buildout.easy_install.allow_picked_versions(False) - True - - (The old setting is returned.) - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... ) - Traceback (most recent call last): - ... - UserError: Picked: demo = 0.3 - - >>> zc.buildout.easy_install.allow_picked_versions(True) - False - - The function default_versions can be used to get and set default - version information to be used when no version information is passes. - If called with an argument, it sets the default versions: - - >>> zc.buildout.easy_install.default_versions(dict(demoneeded='1')) - {} - - It always returns the previous default versions. If called without an - argument, it simply returns the default versions without changing - them: - - >>> zc.buildout.easy_install.default_versions() - {'demoneeded': '1'} - - So with the default versions set, we'll get the requested version even - if the versions option isn't used: - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... ) - - >>> [d.version for d in ws] - ['0.3', '1.0'] - - Of course, we can unset the default versions by passing an empty - dictionary: - - >>> zc.buildout.easy_install.default_versions({}) - {'demoneeded': '1'} - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, links=[link_server], index=link_server+'index/', - ... ) - - >>> [d.version for d in ws] - ['0.3', '1.1'] - - Dependencies in Site Packages - ----------------------------- - - Paths outside of Python's standard library--or more precisely, those that are - not included when Python is started with the -S argument--are loosely referred - to as "site-packages" here. These site-packages are searched by default for - distributions. This can be disabled, so that, for instance, a system Python - can be used with buildout, cleaned of any packages installed by a user or - system package manager. - - The default behavior can be controlled and introspected using - zc.buildout.easy_install.include_site_packages. - - >>> zc.buildout.easy_install.include_site_packages() - True - - Here's an example of using a Python executable that includes our dependencies. - - Our "py_path" will have the "demoneeded," and "demo" packages available. - We'll simply be asking for "demoneeded" here, but without any external - index or links. - - >>> from zc.buildout.tests import create_sample_sys_install - >>> py_path, site_packages_path = make_py() - >>> create_sample_sys_install(site_packages_path) - - >>> example_dest = tmpdir('site-packages-example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['demoneeded'], example_dest, links=[], executable=py_path, - ... index=None) - >>> [dist.project_name for dist in workingset] - ['demoneeded'] - - That worked fine. Let's try again with site packages not allowed. We'll - change the policy by changing the default. Notice that the function for - changing the default value returns the previous value. - - >>> zc.buildout.easy_install.include_site_packages(False) - True - - >>> zc.buildout.easy_install.include_site_packages() - False - - >>> zc.buildout.easy_install.clear_index_cache() - >>> rmdir(example_dest) - >>> example_dest = tmpdir('site-packages-example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['demoneeded'], example_dest, links=[], executable=py_path, - ... index=None) - Traceback (most recent call last): - ... - MissingDistribution: Couldn't find a distribution for 'demoneeded'. - >>> zc.buildout.easy_install.clear_index_cache() - - Now we'll reset the default. - - >>> zc.buildout.easy_install.include_site_packages(True) - False - - >>> zc.buildout.easy_install.include_site_packages() - True - - Dependency links - ---------------- - - Setuptools allows metadata that describes where to search for package - dependencies. This option is called dependency_links. Buildout has its - own notion of where to look for dependencies, but it also uses the - setup tools dependency_links information if it's available. - - Let's demo this by creating an egg that specifies dependency_links. - - To begin, let's create a new egg repository. This repository hold a - newer version of the 'demoneeded' egg than the sample repository does. - - >>> repoloc = tmpdir('repo') - >>> from zc.buildout.tests import create_egg - >>> create_egg('demoneeded', '1.2', repoloc) - >>> link_server2 = start_server(repoloc) - - Turn on logging on this server so that we can see when eggs are pulled - from it. - - >>> get(link_server2 + 'enable_server_logging') - GET 200 /enable_server_logging - '' - - Now we can create an egg that specifies that its dependencies are - found on this server. - - >>> repoloc = tmpdir('repo2') - >>> create_egg('hasdeps', '1.0', repoloc, - ... install_requires = "'demoneeded'", - ... dependency_links = [link_server2]) - - Let's add the egg to another repository. - - >>> link_server3 = start_server(repoloc) - - Now let's install the egg. - - >>> example_dest = tmpdir('example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['hasdeps'], example_dest, - ... links=[link_server3], index=link_server3+'index/') - GET 200 / - GET 200 /demoneeded-1.2-pyN.N.egg - - The server logs show that the dependency was retrieved from the server - specified in the dependency_links. - - Now let's see what happens if we provide two different ways to retrieve - the dependencies. - - >>> rmdir(example_dest) - >>> example_dest = tmpdir('example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['hasdeps'], example_dest, index=link_server+'index/', - ... links=[link_server, link_server3]) - GET 200 / - GET 200 /demoneeded-1.2-pyN.N.egg - - Once again the dependency is fetched from the logging server even - though it is also available from the non-logging server. This is - because the version on the logging server is newer and buildout - normally chooses the newest egg available. - - If you wish to control where dependencies come from regardless of - dependency_links setup metadata use the 'use_dependency_links' option - to zc.buildout.easy_install.install(). - - >>> rmdir(example_dest) - >>> example_dest = tmpdir('example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['hasdeps'], example_dest, index=link_server+'index/', - ... links=[link_server, link_server3], - ... use_dependency_links=False) - - Notice that this time the dependency egg is not fetched from the - logging server. When you specify not to use dependency_links, eggs - will only be searched for using the links you explicitly provide. - - Another way to control this option is with the - zc.buildout.easy_install.use_dependency_links() function. This - function sets the default behavior for the zc.buildout.easy_install() - function. - - >>> zc.buildout.easy_install.use_dependency_links(False) - True - - The function returns its previous setting. - - >>> rmdir(example_dest) - >>> example_dest = tmpdir('example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['hasdeps'], example_dest, index=link_server+'index/', - ... links=[link_server, link_server3]) - - It can be overridden by passing a keyword argument to the install - function. - - >>> rmdir(example_dest) - >>> example_dest = tmpdir('example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['hasdeps'], example_dest, index=link_server+'index/', - ... links=[link_server, link_server3], - ... use_dependency_links=True) - GET 200 /demoneeded-1.2-pyN.N.egg - - To return the dependency_links behavior to normal call the function again. - - >>> zc.buildout.easy_install.use_dependency_links(True) - False - >>> rmdir(example_dest) - >>> example_dest = tmpdir('example-install') - >>> workingset = zc.buildout.easy_install.install( - ... ['hasdeps'], example_dest, index=link_server+'index/', - ... links=[link_server, link_server3]) - GET 200 /demoneeded-1.2-pyN.N.egg - - - Script generation - ----------------- - - The easy_install module provides support for creating scripts from eggs. - It provides two competing functions. One, ``scripts``, is a - well-established approach to generating reliable scripts with a "clean" - Python--e.g., one that does not have any packages in its site-packages. - The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is - designed to work with a Python that has code in its site-packages, such - as a system Python. - - Both are similar to setuptools except that they provides facilities for - baking a script's path into the script. This has two advantages: - - - The eggs to be used by a script are not chosen at run time, making - startup faster and, more importantly, deterministic. - - - The script doesn't have to import pkg_resources because the logic that - pkg_resources would execute at run time is executed at script-creation - time. (There is an exception in ``sitepackage_safe_scripts`` if you - want to have your Python's site packages available, as discussed - below, but even in that case pkg_resources is only partially - activated, which can be a significant time savings.) - - - The ``scripts`` function - ~~~~~~~~~~~~~~~~~~~~~~~~ - - The ``scripts`` function is the first way to generate scripts that we'll - examine. It is the earlier approach that the package offered. Let's - create a destination directory for it to place them in: - - >>> bin = tmpdir('bin') - - Now, we'll use the scripts function to generate scripts in this directory - from the demo egg: - - >>> import sys - >>> scripts = zc.buildout.easy_install.scripts( - ... ['demo'], ws, sys.executable, bin) - - the four arguments we passed were: - - 1. A sequence of distribution requirements. These are of the same - form as setuptools requirements. Here we passed a single - requirement, for the version 0.1 demo distribution. - - 2. A working set, - - 3. The Python executable to use, and - - 3. The destination directory. - - The bin directory now contains a generated script: - - >>> ls(bin) - - demo - - The return value is a list of the scripts generated: - - >>> import os, sys - >>> if sys.platform == 'win32': - ... scripts == [os.path.join(bin, 'demo.exe'), - ... os.path.join(bin, 'demo-script.py')] - ... else: - ... scripts == [os.path.join(bin, 'demo')] - True - - Note that in Windows, 2 files are generated for each script. A script - file, ending in '-script.py', and an exe file that allows the script - to be invoked directly without having to specify the Python - interpreter and without having to provide a '.py' suffix. - - The demo script run the entry point defined in the demo egg: - - >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - <BLANKLINE> - import sys - sys.path[0:0] = [ - '/sample-install/demo-0.3-py2.4.egg', - '/sample-install/demoneeded-1.1-py2.4.egg', - ] - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main() - - Some things to note: - - - The demo and demoneeded eggs are added to the beginning of sys.path. - - - The module for the script entry point is imported and the entry - point, in this case, 'main', is run. - - Rather than requirement strings, you can pass tuples containing 3 - strings: - - - A script name, - - - A module, - - - An attribute expression for an entry point within the module. - - For example, we could have passed entry point information directly - rather than passing a requirement: - - >>> scripts = zc.buildout.easy_install.scripts( - ... [('demo', 'eggrecipedemo', 'main')], - ... ws, sys.executable, bin) - - >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - <BLANKLINE> - import sys - sys.path[0:0] = [ - '/sample-install/demo-0.3-py2.4.egg', - '/sample-install/demoneeded-1.1-py2.4.egg', - ] - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main() - - Passing entry-point information directly is handy when using eggs (or - distributions) that don't declare their entry points, such as - distributions that aren't based on setuptools. - - The interpreter keyword argument can be used to generate a script that can - be used to invoke the Python interactive interpreter with the path set - based on the working set. This generated script can also be used to - run other scripts with the path set on the working set: - - >>> scripts = zc.buildout.easy_install.scripts( - ... ['demo'], ws, sys.executable, bin, interpreter='py') - - - >>> ls(bin) - - demo - - py - - >>> if sys.platform == 'win32': - ... scripts == [os.path.join(bin, 'demo.exe'), - ... os.path.join(bin, 'demo-script.py'), - ... os.path.join(bin, 'py.exe'), - ... os.path.join(bin, 'py-script.py')] - ... else: - ... scripts == [os.path.join(bin, 'demo'), - ... os.path.join(bin, 'py')] - True - - The py script simply runs the Python interactive interpreter with - the path set: - - >>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - <BLANKLINE> - import sys - <BLANKLINE> - sys.path[0:0] = [ - '/sample-install/demo-0.3-pyN.N.egg', - '/sample-install/demoneeded-1.1-pyN.N.egg', - ] - <BLANKLINE> - _interactive = True - if len(sys.argv) > 1: - _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') - _interactive = False - for (_opt, _val) in _options: - if _opt == '-i': - _interactive = True - elif _opt == '-c': - exec _val - elif _opt == '-m': - sys.argv[1:] = _args - _args = [] - __import__("runpy").run_module( - _val, {}, "__main__", alter_sys=True) - <BLANKLINE> - if _args: - sys.argv[:] = _args - __file__ = _args[0] - del _options, _args - execfile(__file__) - <BLANKLINE> - if _interactive: - del _interactive - __import__("code").interact(banner="", local=globals()) - - If invoked with a script name and arguments, it will run that script, instead. - - >>> write('ascript', ''' - ... "demo doc" - ... print sys.argv - ... print (__name__, __file__, __doc__) - ... ''') - >>> print system(join(bin, 'py')+' ascript a b c'), - ['ascript', 'a', 'b', 'c'] - ('__main__', 'ascript', 'demo doc') - - For Python 2.5 and higher, you can also use the -m option to run a - module: - - >>> print system(join(bin, 'py')+' -m pdb'), - usage: pdb.py scriptfile [arg] ... - - >>> print system(join(bin, 'py')+' -m pdb what'), - Error: what does not exist - - An additional argument can be passed to define which scripts to install - and to provide script names. The argument is a dictionary mapping - original script names to new script names. - - >>> bin = tmpdir('bin2') - >>> scripts = zc.buildout.easy_install.scripts( - ... ['demo'], ws, sys.executable, bin, dict(demo='run')) - - >>> if sys.platform == 'win32': - ... scripts == [os.path.join(bin, 'run.exe'), - ... os.path.join(bin, 'run-script.py')] - ... else: - ... scripts == [os.path.join(bin, 'run')] - True - >>> ls(bin) - - run - - >>> print system(os.path.join(bin, 'run')), - 3 1 - - The ``scripts`` function: Including extra paths in scripts - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - We can pass a keyword argument, extra paths, to cause additional paths - to be included in the a generated script: - - >>> foo = tmpdir('foo') - >>> scripts = zc.buildout.easy_install.scripts( - ... ['demo'], ws, sys.executable, bin, dict(demo='run'), - ... extra_paths=[foo]) - - >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - <BLANKLINE> - import sys - sys.path[0:0] = [ - '/sample-install/demo-0.3-py2.4.egg', - '/sample-install/demoneeded-1.1-py2.4.egg', - '/foo', - ] - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main() - - The ``scripts`` function: Providing script arguments - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - An "argument" keyword argument can be used to pass arguments to an - entry point. The value passed is a source string to be placed between the - parentheses in the call: - - >>> scripts = zc.buildout.easy_install.scripts( - ... ['demo'], ws, sys.executable, bin, dict(demo='run'), - ... arguments='1, 2') - - >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - import sys - sys.path[0:0] = [ - '/sample-install/demo-0.3-py2.4.egg', - '/sample-install/demoneeded-1.1-py2.4.egg', - ] - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main(1, 2) - - The ``scripts`` function: Passing initialization code - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - You can also pass script initialization code: - - >>> scripts = zc.buildout.easy_install.scripts( - ... ['demo'], ws, sys.executable, bin, dict(demo='run'), - ... arguments='1, 2', - ... initialization='import os\nos.chdir("foo")') - - >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - import sys - sys.path[0:0] = [ - '/sample-install/demo-0.3-py2.4.egg', - '/sample-install/demoneeded-1.1-py2.4.egg', - ] - <BLANKLINE> - import os - os.chdir("foo") - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main(1, 2) - - The ``scripts`` function: Relative paths - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Sometimes, you want to be able to move a buildout directory around and - have scripts still work without having to rebuild them. We can - control this using the relative_paths option to install. You need - to pass a common base directory of the scripts and eggs: - - >>> bo = tmpdir('bo') - >>> ba = tmpdir('ba') - >>> mkdir(bo, 'eggs') - >>> mkdir(bo, 'bin') - >>> mkdir(bo, 'other') - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], join(bo, 'eggs'), links=[link_server], - ... index=link_server+'index/') - - >>> scripts = zc.buildout.easy_install.scripts( - ... ['demo'], ws, sys.executable, join(bo, 'bin'), dict(demo='run'), - ... extra_paths=[ba, join(bo, 'bar')], - ... interpreter='py', - ... relative_paths=bo) - - >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - <BLANKLINE> - import os - <BLANKLINE> - join = os.path.join - base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) - base = os.path.dirname(base) - <BLANKLINE> - import sys - sys.path[0:0] = [ - join(base, 'eggs/demo-0.3-pyN.N.egg'), - join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), - '/ba', - join(base, 'bar'), - ] - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main() - - Note that the extra path we specified that was outside the directory - passed as relative_paths wasn't converted to a relative path. - - Of course, running the script works: - - >>> print system(join(bo, 'bin', 'run')), - 3 1 - - We specified an interpreter and its paths are adjusted too: - - >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 - <BLANKLINE> - import os - <BLANKLINE> - join = os.path.join - base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) - base = os.path.dirname(base) - <BLANKLINE> - import sys - <BLANKLINE> - sys.path[0:0] = [ - join(base, 'eggs/demo-0.3-pyN.N.egg'), - join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), - '/ba', - join(base, 'bar'), - ] - <BLANKLINE> - _interactive = True - if len(sys.argv) > 1: - _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') - _interactive = False - for (_opt, _val) in _options: - if _opt == '-i': - _interactive = True - elif _opt == '-c': - exec _val - elif _opt == '-m': - sys.argv[1:] = _args - _args = [] - __import__("runpy").run_module( - _val, {}, "__main__", alter_sys=True) - <BLANKLINE> - if _args: - sys.argv[:] = _args - __file__ = _args[0] - del _options, _args - execfile(__file__) - <BLANKLINE> - if _interactive: - del _interactive - __import__("code").interact(banner="", local=globals()) - - The ``sitepackage_safe_scripts`` function - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - The newer function for creating scripts is ``sitepackage_safe_scripts``. - It has the same basic functionality as the ``scripts`` function: it can - create scripts to run arbitrary entry points, and to run a Python - interpreter. The following are the differences from a user's - perspective. - - - It can be used safely with a Python that has packages installed itself, - such as a system-installed Python. - - - In contrast to the interpreter generated by the ``scripts`` method, which - supports only a small subset of the usual Python executable's options, - the interpreter generated by ``sitepackage_safe_scripts`` supports all - of them. This makes it possible to use as full Python replacement for - scripts that need the distributions specified in your buildout. - - - Both the interpreter and the entry point scripts allow you to include the - site packages, and/or the sitecustomize, of the Python executable, if - desired. - - It works by creating site.py and sitecustomize.py files that set up the - desired paths and initialization. These must be placed within an otherwise - empty directory. Typically this is in a recipe's parts directory. - - Here's the simplest example, building an interpreter script. - - >>> interpreter_dir = tmpdir('interpreter') - >>> interpreter_parts_dir = os.path.join( - ... interpreter_dir, 'parts', 'interpreter') - >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin') - >>> mkdir(interpreter_bin_dir) - >>> mkdir(interpreter_dir, 'eggs') - >>> mkdir(interpreter_dir, 'parts') - >>> mkdir(interpreter_parts_dir) - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], - ... index=link_server+'index/') - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... interpreter='py') - - Depending on whether the machine being used is running Windows or not, this - produces either three or four files. In both cases, we have site.py and - sitecustomize.py generated in the parts/interpreter directory. For Windows, - we have py.exe and py-script.py; for other operating systems, we have py. - - >>> sitecustomize_path = os.path.join( - ... interpreter_parts_dir, 'sitecustomize.py') - >>> site_path = os.path.join(interpreter_parts_dir, 'site.py') - >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py') - >>> if sys.platform == 'win32': - ... py_path = os.path.join(interpreter_bin_dir, 'py-script.py') - ... expected = [sitecustomize_path, - ... site_path, - ... os.path.join(interpreter_bin_dir, 'py.exe'), - ... py_path] - ... else: - ... py_path = interpreter_path - ... expected = [sitecustomize_path, site_path, py_path] - ... - >>> assert generated == expected, repr((generated, expected)) - - We didn't ask for any initialization, and we didn't ask to use the underlying - sitecustomization, so sitecustomize.py is empty. - - >>> cat(sitecustomize_path) - - The interpreter script is simple. It puts the directory with the - site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python. - - >>> cat(py_path) - #!/usr/bin/python -S - import os - import sys - <BLANKLINE> - argv = [sys.executable] + sys.argv[1:] - environ = os.environ.copy() - path = '/interpreter/parts/interpreter' - if environ.get('PYTHONPATH'): - path = os.pathsep.join([path, environ['PYTHONPATH']]) - environ['PYTHONPATH'] = path - os.execve(sys.executable, argv, environ) - - The site.py file is a modified version of the underlying Python's site.py. - The most important modification is that it has a different version of the - addsitepackages function. It sets up the Python path, similarly to the - behavior of the function it replaces. The following shows the part that - buildout inserts, in the simplest case. - - >>> sys.stdout.write('#\n'); cat(site_path) - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - #... - def addsitepackages(known_paths): - """Add site packages, as determined by zc.buildout. - <BLANKLINE> - See original_addsitepackages, below, for the original version.""" - buildout_paths = [ - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' - ] - for path in buildout_paths: - sitedir, sitedircase = makepath(path) - if not sitedircase in known_paths and os.path.exists(sitedir): - sys.path.append(sitedir) - known_paths.add(sitedircase) - return known_paths - <BLANKLINE> - def original_addsitepackages(known_paths):... - - Here are some examples of the interpreter in use. - - >>> print call_py(interpreter_path, "print 16+26") - 42 - <BLANKLINE> - >>> res = call_py(interpreter_path, "import sys; print sys.path") - >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - ['', - '/interpreter/parts/interpreter', - ..., - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] - <BLANKLINE> - >>> clean_paths = eval(res.strip()) # This is used later for comparison. - - If you provide initialization, it goes in sitecustomize.py. - - >>> def reset_interpreter(): - ... # This is necessary because, in our tests, the timestamps of the - ... # .pyc files are not outdated when we want them to be. - ... rmdir(interpreter_bin_dir) - ... mkdir(interpreter_bin_dir) - ... rmdir(interpreter_parts_dir) - ... mkdir(interpreter_parts_dir) - ... - >>> reset_interpreter() - - >>> initialization_string = """\ - ... import os - ... os.environ['FOO'] = 'bar baz bing shazam'""" - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... interpreter='py', initialization=initialization_string) - >>> cat(sitecustomize_path) - import os - os.environ['FOO'] = 'bar baz bing shazam' - >>> print call_py(interpreter_path, "import os; print os.environ['FOO']") - bar baz bing shazam - <BLANKLINE> - - If you use relative paths, this affects the interpreter and site.py. (This is - again the UNIX version; the Windows version uses subprocess instead of - os.execve.) - - >>> reset_interpreter() - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... interpreter='py', relative_paths=interpreter_dir) - >>> cat(py_path) - #!/usr/bin/python -S - import os - import sys - <BLANKLINE> - join = os.path.join - base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) - base = os.path.dirname(base) - <BLANKLINE> - argv = [sys.executable] + sys.argv[1:] - environ = os.environ.copy() - path = join(base, 'parts/interpreter') - if environ.get('PYTHONPATH'): - path = os.pathsep.join([path, environ['PYTHONPATH']]) - environ['PYTHONPATH'] = path - os.execve(sys.executable, argv, environ) - - For site.py, we again show only the pertinent parts. Notice that the egg - paths join a base to a path, as with the use of this argument in the - ``scripts`` function. - - >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS - #... - def addsitepackages(known_paths): - """Add site packages, as determined by zc.buildout. - <BLANKLINE> - See original_addsitepackages, below, for the original version.""" - join = os.path.join - base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) - base = os.path.dirname(base) - base = os.path.dirname(base) - buildout_paths = [ - join(base, 'eggs/demo-0.3-pyN.N.egg'), - join(base, 'eggs/demoneeded-1.1-pyN.N.egg') - ]... - - The paths resolve in practice as you would expect. - - >>> print call_py(interpreter_path, - ... "import sys, pprint; pprint.pprint(sys.path)") - ... # doctest: +ELLIPSIS - ['', - '/interpreter/parts/interpreter', - ..., - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] - <BLANKLINE> - - The ``extra_paths`` argument affects the path in site.py. Notice that - /interpreter/other is added after the eggs. - - >>> reset_interpreter() - >>> mkdir(interpreter_dir, 'other') - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... interpreter='py', extra_paths=[join(interpreter_dir, 'other')]) - >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS - #... - def addsitepackages(known_paths): - """Add site packages, as determined by zc.buildout. - <BLANKLINE> - See original_addsitepackages, below, for the original version.""" - buildout_paths = [ - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', - '/interpreter/other' - ]... - - >>> print call_py(interpreter_path, - ... "import sys, pprint; pprint.pprint(sys.path)") - ... # doctest: +ELLIPSIS - ['', - '/interpreter/parts/interpreter', - ..., - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', - '/interpreter/other'] - <BLANKLINE> - - The ``sitepackage_safe_scripts`` function: using site-packages - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - The ``sitepackage_safe_scripts`` function supports including site - packages. This has some advantages and some serious dangers. - - A typical reason to include site-packages is that it is easier to - install one or more dependencies in your Python than it is with - buildout. Some packages, such as lxml or Python PostgreSQL integration, - have dependencies that can be much easier to build and/or install using - other mechanisms, such as your operating system's package manager. By - installing some core packages into your Python's site-packages, this can - significantly simplify some application installations. - - However, doing this has a significant danger. One of the primary goals - of buildout is to provide repeatability. Some packages (one of the - better known Python openid packages, for instance) change their behavior - depending on what packages are available. If Python curl bindings are - available, these may be preferred by the library. If a certain XML - package is installed, it may be preferred by the library. These hidden - choices may cause small or large behavior differences. The fact that - they can be rarely encountered can actually make it worse: you forget - that this might be a problem, and debugging the differences can be - difficult. If you allow site-packages to be included in your buildout, - and the Python you use is not managed precisely by your application (for - instance, it is a system Python), you open yourself up to these - possibilities. Don't be unaware of the dangers. - - That explained, let's see how it works. If you don't use namespace packages, - this is very straightforward. - - >>> reset_interpreter() - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... interpreter='py', include_site_packages=True) - >>> sys.stdout.write('#\n'); cat(site_path) - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - #... - def addsitepackages(known_paths): - """Add site packages, as determined by zc.buildout. - <BLANKLINE> - See original_addsitepackages, below, for the original version.""" - setuptools_path = None - buildout_paths = [ - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' - ] - for path in buildout_paths: - sitedir, sitedircase = makepath(path) - if not sitedircase in known_paths and os.path.exists(sitedir): - sys.path.append(sitedir) - known_paths.add(sitedircase) - sys.__egginsert = len(buildout_paths) # Support distribute. - original_paths = [ - ... - ] - for path in original_paths: - if path == setuptools_path or path not in known_paths: - addsitedir(path, known_paths) - return known_paths - <BLANKLINE> - def original_addsitepackages(known_paths):... - - It simply adds the original paths using addsitedir after the code to add the - buildout paths. - - Here's an example of the new script in use. Other documents and tests in - this package give the feature a more thorough workout, but this should - give you an idea of the feature. - - >>> res = call_py(interpreter_path, "import sys; print sys.path") - >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - ['', - '/interpreter/parts/interpreter', - ..., - '/interpreter/eggs/demo-0.3-py2.4.egg', - '/interpreter/eggs/demoneeded-1.1-py2.4.egg', - ...] - <BLANKLINE> - - The clean_paths gathered earlier is a subset of this full list of paths. - - >>> full_paths = eval(res.strip()) - >>> len(clean_paths) < len(full_paths) - True - >>> set(os.path.normpath(p) for p in clean_paths).issubset( - ... os.path.normpath(p) for p in full_paths) - True - - Unfortunately, because of how setuptools namespace packages are implemented - differently for operating system packages (debs or rpms) as opposed to - standard setuptools installation, there's a slightly trickier dance if you - use them. To show this we'll needs some extra eggs that use namespaces. - We'll use the ``tellmy.fortune`` package, which we'll need to make an initial - call to another text fixture to create. - - >>> from zc.buildout.tests import create_sample_namespace_eggs - >>> namespace_eggs = tmpdir('namespace_eggs') - >>> create_sample_namespace_eggs(namespace_eggs) - - >>> reset_interpreter() - >>> ws = zc.buildout.easy_install.install( - ... ['demo', 'tellmy.fortune'], join(interpreter_dir, 'eggs'), - ... links=[link_server, namespace_eggs], index=link_server+'index/') - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... interpreter='py', include_site_packages=True) - >>> sys.stdout.write('#\n'); cat(site_path) - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - #... - def addsitepackages(known_paths): - """Add site packages, as determined by zc.buildout. - <BLANKLINE> - See original_addsitepackages, below, for the original version.""" - setuptools_path = '...setuptools...' - sys.path.append(setuptools_path) - known_paths.add(os.path.normcase(setuptools_path)) - import pkg_resources - buildout_paths = [ - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', - '...setuptools...', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' - ] - for path in buildout_paths: - sitedir, sitedircase = makepath(path) - if not sitedircase in known_paths and os.path.exists(sitedir): - sys.path.append(sitedir) - known_paths.add(sitedircase) - pkg_resources.working_set.add_entry(sitedir) - sys.__egginsert = len(buildout_paths) # Support distribute. - original_paths = [ - ... - ] - for path in original_paths: - if path == setuptools_path or path not in known_paths: - addsitedir(path, known_paths) - return known_paths - <BLANKLINE> - def original_addsitepackages(known_paths):... - - >>> print call_py(interpreter_path, "import sys; print sys.path") - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - ['', - '/interpreter/parts/interpreter', - ..., - '...setuptools...', - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', - ...] - - As you can see, the script now first imports pkg_resources. Then we - need to process egg files specially to look for namespace packages there - *before* we process process lines in .pth files that use the "import" - feature--lines that might be part of the setuptools namespace package - implementation for system packages, as mentioned above, and that must - come after processing egg namespaces. - - The most complex that this function gets is if you use namespace packages, - include site-packages, and use relative paths. For completeness, we'll look - at that result. - - >>> reset_interpreter() - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... interpreter='py', include_site_packages=True, - ... relative_paths=interpreter_dir) - >>> sys.stdout.write('#\n'); cat(site_path) - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - #... - def addsitepackages(known_paths): - """Add site packages, as determined by zc.buildout. - <BLANKLINE> - See original_addsitepackages, below, for the original version.""" - join = os.path.join - base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) - base = os.path.dirname(base) - base = os.path.dirname(base) - setuptools_path = '...setuptools...' - sys.path.append(setuptools_path) - known_paths.add(os.path.normcase(setuptools_path)) - import pkg_resources - buildout_paths = [ - join(base, 'eggs/demo-0.3-pyN.N.egg'), - join(base, 'eggs/tellmy.fortune-1.0-pyN.N.egg'), - '...setuptools...', - join(base, 'eggs/demoneeded-1.1-pyN.N.egg') - ] - for path in buildout_paths: - sitedir, sitedircase = makepath(path) - if not sitedircase in known_paths and os.path.exists(sitedir): - sys.path.append(sitedir) - known_paths.add(sitedircase) - pkg_resources.working_set.add_entry(sitedir) - sys.__egginsert = len(buildout_paths) # Support distribute. - original_paths = [ - ... - ] - for path in original_paths: - if path == setuptools_path or path not in known_paths: - addsitedir(path, known_paths) - return known_paths - <BLANKLINE> - def original_addsitepackages(known_paths):... - - >>> print call_py(interpreter_path, "import sys; print sys.path") - ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE - ['', - '/interpreter/parts/interpreter', - ..., - '...setuptools...', - '/interpreter/eggs/demo-0.3-pyN.N.egg', - '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', - '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', - ...] - - The ``exec_sitecustomize`` argument does the same thing for the - sitecustomize module--it allows you to include the code from the - sitecustomize module in the underlying Python if you set the argument to - True. The z3c.recipe.scripts package sets up the full environment necessary - to demonstrate this piece. - - The ``sitepackage_safe_scripts`` function: writing scripts for entry points - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - All of the examples so far for this function have been creating - interpreters. The function can also write scripts for entry - points. They are almost identical to the scripts that we saw for the - ``scripts`` function except that they ``import site`` after setting the - sys.path to include our custom site.py and sitecustomize.py files. These - files then initialize the Python environment as we have already seen. Let's - see a simple example. - - >>> reset_interpreter() - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], - ... index=link_server+'index/') - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... reqs=['demo']) - - As before, in Windows, 2 files are generated for each script. A script - file, ending in '-script.py', and an exe file that allows the script - to be invoked directly without having to specify the Python - interpreter and without having to provide a '.py' suffix. This is in addition - to the site.py and sitecustomize.py files that are generated as with our - interpreter examples above. - - >>> if sys.platform == 'win32': - ... demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py') - ... expected = [sitecustomize_path, - ... site_path, - ... os.path.join(interpreter_bin_dir, 'demo.exe'), - ... demo_path] - ... else: - ... demo_path = os.path.join(interpreter_bin_dir, 'demo') - ... expected = [sitecustomize_path, site_path, demo_path] - ... - >>> assert generated == expected, repr((generated, expected)) - - The demo script runs the entry point defined in the demo egg: - - >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 -S - <BLANKLINE> - import sys - sys.path[0:0] = [ - '/interpreter/parts/interpreter', - ] - <BLANKLINE> - <BLANKLINE> - import os - path = sys.path[0] - if os.environ.get('PYTHONPATH'): - path = os.pathsep.join([path, os.environ['PYTHONPATH']]) - os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') - os.environ['PYTHONPATH'] = path - import site # imports custom buildout-generated site.py - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main() - - >>> demo_call = join(interpreter_bin_dir, 'demo') - >>> if sys.platform == 'win32': - ... demo_call = '"%s"' % demo_call - >>> print system(demo_call) - 3 1 - <BLANKLINE> - - There are a few differences from the ``scripts`` function. First, the - ``reqs`` argument (an iterable of string requirements or entry point - tuples) is a keyword argument here. We see that in the example above. - Second, the ``arguments`` argument is now named ``script_arguments`` to - try and clarify that it does not affect interpreters. While the - ``initialization`` argument continues to affect both the interpreters - and the entry point scripts, if you have initialization that is only - pertinent to the entry point scripts, you can use the - ``script_initialization`` argument. - - Let's see ``script_arguments`` and ``script_initialization`` in action. - - >>> reset_interpreter() - >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( - ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, - ... reqs=['demo'], script_arguments='1, 2', - ... script_initialization='import os\nos.chdir("foo")') - - >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE - #!/usr/local/bin/python2.4 -S - import sys - sys.path[0:0] = [ - '/interpreter/parts/interpreter', - ] - <BLANKLINE> - import os - path = sys.path[0] - if os.environ.get('PYTHONPATH'): - path = os.pathsep.join([path, os.environ['PYTHONPATH']]) - os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') - os.environ['PYTHONPATH'] = path - import site # imports custom buildout-generated site.py - import os - os.chdir("foo") - <BLANKLINE> - import eggrecipedemo - <BLANKLINE> - if __name__ == '__main__': - eggrecipedemo.main(1, 2) - - Handling custom build options for extensions provided in source distributions - ----------------------------------------------------------------------------- - - Sometimes, we need to control how extension modules are built. The - build function provides this level of control. It takes a single - package specification, downloads a source distribution, and builds it - with specified custom build options. - - The build function takes 3 positional arguments: - - spec - A package specification for a source distribution - - dest - A destination directory - - build_ext - A dictionary of options to be passed to the distutils build_ext - command when building extensions. - - It supports a number of optional keyword arguments: - - links - a sequence of URLs, file names, or directories to look for - links to distributions, - - index - The URL of an index server, or almost any other valid URL. :) - - If not specified, the Python Package Index, - http://pypi.python.org/simple/, is used. You can specify an - alternate index with this option. If you use the links option and - if the links point to the needed distributions, then the index can - be anything and will be largely ignored. In the examples, here, - we'll just point to an empty directory on our link server. This - will make our examples run a little bit faster. - - executable - A path to a Python executable. Distributions will be installed - using this executable and will be for the matching Python version. - - path - A list of additional directories to search for locally-installed - distributions. - - newest - A boolean value indicating whether to search for new distributions - when already-installed distributions meet the requirement. When - this is true, the default, and when the destination directory is - not None, then the install function will search for the newest - distributions that satisfy the requirements. - - versions - A dictionary mapping project names to version numbers to be used - when selecting distributions. This can be used to specify a set of - distribution versions independent of other requirements. - - - Our link server included a source distribution that includes a simple - extension, extdemo.c:: - - #include <Python.h> - #include <extdemo.h> - - static PyMethodDef methods[] = {}; - - PyMODINIT_FUNC - initextdemo(void) - { - PyObject *m; - m = Py_InitModule3("extdemo", methods, ""); - #ifdef TWO - PyModule_AddObject(m, "val", PyInt_FromLong(2)); - #else - PyModule_AddObject(m, "val", PyInt_FromLong(EXTDEMO)); - #endif - } - - The extension depends on a system-dependent include file, extdemo.h, - that defines a constant, EXTDEMO, that is exposed by the extension. - - We'll add an include directory to our sample buildout and add the - needed include file to it: - - >>> mkdir('include') - >>> write('include', 'extdemo.h', - ... """ - ... #define EXTDEMO 42 - ... """) - - Now, we can use the build function to create an egg from the source - distribution: - - >>> zc.buildout.easy_install.build( - ... 'extdemo', dest, - ... {'include-dirs': os.path.join(sample_buildout, 'include')}, - ... links=[link_server], index=link_server+'index/') - ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] - - The function returns the list of eggs - - Now if we look in our destination directory, we see we have an extdemo egg: - - >>> ls(dest) - - demo-0.2-py2.4.egg - d demo-0.3-py2.4.egg - - demoneeded-1.0-py2.4.egg - d demoneeded-1.1-py2.4.egg - d extdemo-1.4-py2.4-unix-i686.egg - - Let's update our link server with a new version of extdemo: - - >>> update_extdemo() - >>> print get(link_server), - <html><body> - <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> - <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> - <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> - <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> - <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> - <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> - <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> - <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> - <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> - <a href="extdemo-1.5.zip">extdemo-1.5.zip</a><br> - <a href="index/">index/</a><br> - <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> - </body></html> - - The easy_install caches information about servers to reduce network - access. To see the update, we have to call the clear_index_cache - function to clear the index cache: - - >>> zc.buildout.easy_install.clear_index_cache() - - If we run build with newest set to False, we won't get an update: - - >>> zc.buildout.easy_install.build( - ... 'extdemo', dest, - ... {'include-dirs': os.path.join(sample_buildout, 'include')}, - ... links=[link_server], index=link_server+'index/', - ... newest=False) - ['/sample-install/extdemo-1.4-py2.4-linux-i686.egg'] - - >>> ls(dest) - - demo-0.2-py2.4.egg - d demo-0.3-py2.4.egg - - demoneeded-1.0-py2.4.egg - d demoneeded-1.1-py2.4.egg - d extdemo-1.4-py2.4-unix-i686.egg - - But if we run it with the default True setting for newest, then we'll - get an updated egg: - - >>> zc.buildout.easy_install.build( - ... 'extdemo', dest, - ... {'include-dirs': os.path.join(sample_buildout, 'include')}, - ... links=[link_server], index=link_server+'index/') - ['/sample-install/extdemo-1.5-py2.4-unix-i686.egg'] - - >>> ls(dest) - - demo-0.2-py2.4.egg - d demo-0.3-py2.4.egg - - demoneeded-1.0-py2.4.egg - d demoneeded-1.1-py2.4.egg - d extdemo-1.4-py2.4-unix-i686.egg - d extdemo-1.5-py2.4-unix-i686.egg - - The versions option also influences the versions used. For example, - if we specify a version for extdemo, then that will be used, even - though it isn't the newest. Let's clean out the destination directory - first: - - >>> import os - >>> for name in os.listdir(dest): - ... remove(dest, name) - - >>> zc.buildout.easy_install.build( - ... 'extdemo', dest, - ... {'include-dirs': os.path.join(sample_buildout, 'include')}, - ... links=[link_server], index=link_server+'index/', - ... versions=dict(extdemo='1.4')) - ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] - - >>> ls(dest) - d extdemo-1.4-py2.4-unix-i686.egg - - Handling custom build options for extensions in develop eggs - ------------------------------------------------------------ - - The develop function is similar to the build function, except that, - rather than building an egg from a source directory containing a - setup.py script. - - The develop function takes 2 positional arguments: - - setup - The path to a setup script, typically named "setup.py", or a - directory containing a setup.py script. - - dest - The directory to install the egg link to - - It supports some optional keyword argument: - - build_ext - A dictionary of options to be passed to the distutils build_ext - command when building extensions. - - executable - A path to a Python executable. Distributions will be installed - using this executable and will be for the matching Python version. - - We have a local directory containing the extdemo source: - - >>> ls(extdemo) - - MANIFEST - - MANIFEST.in - - README - - extdemo.c - - setup.py - - Now, we can use the develop function to create a develop egg from the source - distribution: - - >>> zc.buildout.easy_install.develop( - ... extdemo, dest, - ... {'include-dirs': os.path.join(sample_buildout, 'include')}) - '/sample-install/extdemo.egg-link' - - The name of the egg link created is returned. - - Now if we look in our destination directory, we see we have an extdemo - egg link: - - >>> ls(dest) - d extdemo-1.4-py2.4-unix-i686.egg - - extdemo.egg-link - - And that the source directory contains the compiled extension: - - >>> ls(extdemo) - - MANIFEST - - MANIFEST.in - - README - d build - - extdemo.c - d extdemo.egg-info - - extdemo.so - - setup.py - - Download cache - -------------- - - Normally, when distributions are installed, if any processing is - needed, they are downloaded from the internet to a temporary directory - and then installed from there. A download cache can be used to avoid - the download step. This can be useful to reduce network access and to - create source distributions of an entire buildout. - - A download cache is specified by calling the download_cache - function. The function always returns the previous setting. If no - argument is passed, then the setting is unchanged. If an argument is - passed, the download cache is set to the given path, which must point - to an existing directory. Passing None clears the cache setting. - - To see this work, we'll create a directory and set it as the cache - directory: - - >>> cache = tmpdir('cache') - >>> zc.buildout.easy_install.download_cache(cache) - - We'll recreate our destination directory: - - >>> remove(dest) - >>> dest = tmpdir('sample-install') - - We'd like to see what is being fetched from the server, so we'll - enable server logging: - - >>> get(link_server+'enable_server_logging') - GET 200 /enable_server_logging - '' - - Now, if we install demo, and extdemo: - - >>> ws = zc.buildout.easy_install.install( - ... ['demo==0.2'], dest, - ... links=[link_server], index=link_server+'index/', - ... always_unzip=True) - GET 200 / - GET 404 /index/demo/ - GET 200 /index/ - GET 200 /demo-0.2-py2.4.egg - GET 404 /index/demoneeded/ - GET 200 /demoneeded-1.1.zip - - >>> zc.buildout.easy_install.build( - ... 'extdemo', dest, - ... {'include-dirs': os.path.join(sample_buildout, 'include')}, - ... links=[link_server], index=link_server+'index/') - GET 404 /index/extdemo/ - GET 200 /extdemo-1.5.zip - ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] - - Not only will we get eggs in our destination directory: - - >>> ls(dest) - d demo-0.2-py2.4.egg - d demoneeded-1.1-py2.4.egg - d extdemo-1.5-py2.4-linux-i686.egg - - But we'll get distributions in the cache directory: - - >>> ls(cache) - - demo-0.2-py2.4.egg - - demoneeded-1.1.zip - - extdemo-1.5.zip - - The cache directory contains uninstalled distributions, such as zipped - eggs or source distributions. - - Let's recreate our destination directory and clear the index cache: - - >>> remove(dest) - >>> dest = tmpdir('sample-install') - >>> zc.buildout.easy_install.clear_index_cache() - - Now when we install the distributions: - - >>> ws = zc.buildout.easy_install.install( - ... ['demo==0.2'], dest, - ... links=[link_server], index=link_server+'index/', - ... always_unzip=True) - GET 200 / - GET 404 /index/demo/ - GET 200 /index/ - GET 404 /index/demoneeded/ - - >>> zc.buildout.easy_install.build( - ... 'extdemo', dest, - ... {'include-dirs': os.path.join(sample_buildout, 'include')}, - ... links=[link_server], index=link_server+'index/') - GET 404 /index/extdemo/ - ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] - - >>> ls(dest) - d demo-0.2-py2.4.egg - d demoneeded-1.1-py2.4.egg - d extdemo-1.5-py2.4-linux-i686.egg - - Note that we didn't download the distributions from the link server. - - If we remove the restriction on demo, we'll download a newer version - from the link server: - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, - ... links=[link_server], index=link_server+'index/', - ... always_unzip=True) - GET 200 /demo-0.3-py2.4.egg - - Normally, the download cache is the preferred source of downloads, but - not the only one. - - Installing solely from a download cache - --------------------------------------- - - A download cache can be used as the basis of application source - releases. In an application source release, we want to distribute an - application that can be built without making any network accesses. In - this case, we distribute a download cache and tell the easy_install - module to install from the download cache only, without making network - accesses. The install_from_cache function can be used to signal that - packages should be installed only from the download cache. The - function always returns the previous setting. Calling it with no - arguments returns the current setting without changing it: - - >>> zc.buildout.easy_install.install_from_cache() - False - - Calling it with a boolean value changes the setting and returns the - previous setting: - - >>> zc.buildout.easy_install.install_from_cache(True) - False - - Let's remove demo-0.3-py2.4.egg from the cache, clear the index cache, - recreate the destination directory, and reinstall demo: - - >>> for f in os.listdir(cache): - ... if f.startswith('demo-0.3-'): - ... remove(cache, f) - - >>> zc.buildout.easy_install.clear_index_cache() - >>> remove(dest) - >>> dest = tmpdir('sample-install') - - >>> ws = zc.buildout.easy_install.install( - ... ['demo'], dest, - ... links=[link_server], index=link_server+'index/', - ... always_unzip=True) - - >>> ls(dest) - d demo-0.2-py2.4.egg - d demoneeded-1.1-py2.4.egg - - This time, we didn't download from or even query the link server. - - .. Disable the download cache: - - >>> zc.buildout.easy_install.download_cache(None) - '/cache' - - >>> zc.buildout.easy_install.install_from_cache(False) - True - - Distribute Support - ================== - - Distribute is a drop-in replacement for Setuptools. - - zc.buildout is now compatible with Distribute 0.6. To use Distribute in your - buildout, you need use the ``--distribute`` option of the ``bootstrap.py`` - script:: - - $ python bootstrap.py --distribute - - This will download and install the latest Distribute 0.6 release in the - ``eggs`` directory, and use this version for the scripts that are created - in ``bin``. - - Notice that if you have a shared eggs directory, a buildout that uses - Distribute will not interfer with other buildouts that are based on Setuptools - and that are sharing the same eggs directory. - - Form more information about the Distribute project, see: - http://python-distribute.org - - - - Change History - ************** - - 1.5.2 (2010-10-11) - ================== - - - changed metadata 'url' to pypi.python.org in order to solve - a temporary outage of buildout.org - - - IMPORTANT: For better backwards compatibility with the pre-1.5 line, - this release has two big changes from 1.5.0 and 1.5.1. - - - Buildout defaults to including site packages. - - - Buildout loads recipes and extensions with the same constraints to - site-packages that it builds eggs, instead of never allowing access - to site-packages. - - This means that the default configuration should better support - pre-existing use of system Python in recipes or builds. - - - To make it easier to detect the fact that buildout has set the PYTHONPATH, - BUILDOUT_ORIGINAL_PYTHONPATH is always set in the environment, even if - PYTHONPATH was not originally set. BUILDOUT_ORIGINAL_PYTHONPATH will - be an empty string if PYTHONPATH was not set. - - 1.5.1 (2010-08-29) - ================== - - New features: - - - Scripts store the old PYTHONPATH in BUILDOUT_ORIGINAL_PYTHONPATH if it - existed, and store nothing in the value if it did not exist. This allows - code that does not want subprocesses to have the system-Python-protected - site.py to set the environment of the subprocess as it was originally. - - Bugs fixed: - - - https://bugs.launchpad.net/bugs/623590 : If include-site-packages were - true and versions were not set explicitly, system eggs were preferred - over newer released eggs. Fixed. - - 1.5.0 (2010-08-23) - ================== - - New features: - - - zc.buildout supports Python 2.7. - - - By default, Buildout and the bootstrap script now prefer final versions of - Buildout, recipes, and extensions. This can be changed by using the - --accept-buildout-test-releases flag (or -t for short) when calling - bootstrap. This will hopefully allow beta releases of these items to - be more easily and safely made in the future. - - NOTE: dependencies of your own software are not affected by this new - behavior. Buildout continues to choose the newest available versions - of your dependencies regardless of whether they are final releases. To - prevent this, use the pre-existing switch ``prefer-final = true`` in - the [buildout] section of your configuration file (see - http://pypi.python.org/pypi/zc.buildout#preferring-final-releases) or - pin your versions using a versions section (see - http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used). - - Bugs fixed: - - - You can now again use virtualenv with Buildout. The new features to let - buildout be used with a system Python are disabled in this configuration, - and the previous script generation behavior (1.4.3) is used, even if - the new function ``zc.buildout.easy_install.sitepackage_safe_scripts`` - is used. - - 1.5.0b2 (2010-04-29) - ==================== - - This was a re-release of 1.4.3 in order to keep 1.5.0b1 release from hurting - workflows that combined virtualenv with zc.buildout. - - 1.5.0b1 (2010-04-29) - ==================== - - New Features: - - - Added buildout:socket-timout option so that socket timeout can be configured - both from command line and from config files. (gotcha) - - - Buildout can be safely used with a system Python (or any Python with code - in site-packages), as long as you use (1) A fresh checkout, (2) the - new bootstrap.py, and (3) recipes that use the new - ``zc.buildout.easy_install.sitepackage_safe_scripts`` function to generate - scripts and interpreters. Many recipes will need to be updated to use - this new function. The scripts and interpreters generated by - ``zc.recipe.egg`` will continue to use the older function, not safe - with system Pythons. Use the ``z3c.recipe.scripts`` as a replacement. - - zc.recipe.egg is still a fully supported, and simpler, way of - generating scripts and interpreters if you are using a "clean" Python, - without code installed in site-packages. It keeps its previous behavior in - order to provide backwards compatibility. - - The z3c.recipe.scripts recipe allows you to control how you use the - code in site-packages. You can exclude it entirely (preferred); allow - eggs in it to fulfill package dependencies declared in setup.py and - buildout configuration; allow it to be available but not used to - fulfill dependencies declared in setup.py or buildout configuration; - or only allow certain eggs in site-packages to fulfill dependencies. - - - Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``, - to generate scripts and interpreter. It produces a full-featured - interpreter (all command-line options supported) and the ability to - safely let scripts include site packages, such as with a system - Python. The ``z3c.recipe.scripts`` recipe uses this new function. - - - Improve bootstrap. - - * New options let you specify where to find ez_setup.py and where to find - a download cache. These options can keep bootstrap from going over the - network. - - * Another new option lets you specify where to put generated eggs. - - * The buildout script generated by bootstrap honors more of the settings - in the designated configuration file (e.g., buildout.cfg). - - * Correctly handle systems where pkg_resources is present but the rest of - setuptools is missing (like Ubuntu installs). - https://bugs.launchpad.net/zc.buildout/+bug/410528 - - - You can develop zc.buildout using Distribute instead of Setuptools. Use - the --distribute option on the dev.py script. (Releases should be tested - with both Distribute and Setuptools.) The tests for zc.buildout pass - with Setuptools and Python 2.4, 2.5, 2.6, and 2.7; and with Distribute and - Python 2.5, 2.6, and 2.7. Using zc.buildout with Distribute and Python 2.4 - is not recommended. - - - The ``distribute-version`` now works in the [buildout] section, mirroring - the ``setuptools-version`` option (this is for consistency; using the - general-purpose ``versions`` option is preferred). - - Bugs fixed: - - - Using Distribute with the ``allow-picked-versions = false`` buildout - option no longer causes an error. - - - The handling and documenting of default buildout options was normalized. - This means, among other things, that ``bin/buildout -vv`` and - ``bin/buildout annotate`` correctly list more of the options. - - - Installing a namespace package using a Python that already has a package - in the same namespace (e.g., in the Python's site-packages) failed in - some cases. It is now handled correctly. - - - Another variation of this error showed itself when at least two - dependencies were in a shared location like site-packages, and the - first one met the "versions" setting. The first dependency would be - added, but subsequent dependencies from the same location (e.g., - site-packages) would use the version of the package found in the - shared location, ignoring the version setting. This is also now - handled correctly. - - 1.4.3 (2009-12-10) - ================== - - Bugs fixed: - - - Using pre-detected setuptools version for easy_installing tgz files. This - prevents a recursion error when easy_installing an upgraded "distribute" - tgz. Note that setuptools did not have this recursion problem solely - because it was packaged as an ``.egg``, which does not have to go through - the easy_install step. - - - 1.4.2 (2009-11-01) - ================== - - New Feature: - - - Added a --distribute option to the bootstrap script, in order - to use Distribute rather than Setuptools. By default, Setuptools - is used. - - Bugs fixed: - - - While checking for new versions of setuptools and buildout itself, - compare requirement locations instead of requirement objects. - - - Incrementing didn't work properly when extending multiple files. - https://bugs.launchpad.net/zc.buildout/+bug/421022 - - - The download API computed MD5 checksums of text files wrong on Windows. - - 1.4.1 (2009-08-27) - ================== - - New Feature: - - - Added a debug built-in recipe to make writing some tests easier. - - Bugs fixed: - - - (introduced in 1.4.0) option incrementing (-=) and decrementing (-=) - didn't work in the buildout section. - https://bugs.launchpad.net/zc.buildout/+bug/420463 - - - Option incrementing and decrementing didn't work for options - specified on the command line. - - - Scripts generated with relative-paths enabled couldn't be - symbolically linked to other locations and still work. - - - Scripts run using generated interpreters didn't have __file__ set correctly. - - - The standard Python -m option didn't work for custom interpreters. - - 1.4.0 (2009-08-26) - ================== - - - When doing variable substitutions, you can omit the section name to - refer to a variable in the same section (e.g. ${:foo}). - - - When doing variable substitution, you can use the special option, - ``_buildout_section_name_`` to get the section name. This is most handy - for getting the current section name (e.g. ${:_buildout_section_name_}). - - - A new special option, ``<`` allows sections to be used as macros. - - - Added annotate command for annotated sections. Displays sections - key-value pairs along with the value origin. - - - Added a download API that handles the download cache, offline mode etc and - is meant to be reused by recipes. - - - Used the download API to allow caching of base configurations (specified by - the buildout section's 'extends' option). - - 1.3.1 (2009-08-12) - ================== - - - Bug fixed: extras were ignored in some cases when versions were specified. - - 1.3.0 (2009-06-22) - ================== - - - Better Windows compatibility in test infrastructure. - - - Now the bootstrap.py has an optional --version argument, - that can be used to force zc.buildout version to use. - - - ``zc.buildout.testing.buildoutSetUp`` installs a new handler in the - python root logging facility. This handler is now removed during - tear down as it might disturb other packages reusing buildout's - testing infrastructure. - - - fixed usage of 'relative_paths' keyword parameter on Windows - - - Added an unload entry point for extensions. - - - Fixed bug: when the relative paths option was used, relative paths - could be inserted into sys.path if a relative path was used to run - the generated script. - - 1.2.1 (2009-03-18) - ================== - - - Refactored generation of relative egg paths to generate simpler code. - - 1.2.0 (2009-03-17) - ================== - - - Added a relative_paths option to zc.buildout.easy_install.script to - generate egg paths relative to the script they're used in. - - 1.1.2 (2009-03-16) - ================== - - - Added Python 2.6 support. Removed Python 2.3 support. - - - Fixed remaining deprecation warnings under Python 2.6, both when running - our tests and when using the package. - - - Switched from using os.popen* to subprocess.Popen, to avoid a deprecation - warning in Python 2.6. See: - - http://docs.python.org/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3 - - - Made sure the 'redo_pyc' function and the doctest checkers work with Python - executable paths containing spaces. - - - Expand shell patterns when processing the list of paths in `develop`, e.g:: - - [buildout] - develop = ./local-checkouts/* - - - Conditionally import and use hashlib.md5 when it's available instead - of md5 module, which is deprecated in Python 2.6. - - - Added Jython support for bootstrap, development bootstrap - and zc.buildout support on Jython - - - Fixed a bug that would cause buildout to break while computing a - directory hash if it found a broken symlink (Launchpad #250573) - - 1.1.1 (2008-07-28) - ================== - - - Fixed a bug that caused buildouts to fail when variable - substitutions are used to name standard directories, as in:: - - [buildout] - eggs-directory = ${buildout:directory}/develop-eggs - - 1.1.0 (2008-07-19) - ================== - - - Added a buildout-level unzip option tp change the default policy for - unzipping zip-safe eggs. - - - Tracebacks are now printed for internal errors (as opposed to user - errors) even without the -D option. - - - pyc and pyo files are regenerated for installed eggs so that the - stored path in code objects matches the the install location. - - 1.0.6 (2008-06-13) - ================== - - - Manually reverted the changeset for the fix for - https://bugs.launchpad.net/zc.buildout/+bug/239212 to verify thet the test - actually fails with the changeset: - http://svn.zope.org/zc.buildout/trunk/src/zc/buildout/buildout.py?rev=87309&r1=87277&r2=87309 - Thanks tarek for pointing this out. (seletz) - - - fixed the test for the += -= syntax in buildout.txt as the test - was actually wronng. The original implementation did a split/join - on whitespace, and later on that was corrected to respect the original - EOL setting, the test was not updated, though. (seletz) - - - added a test to verify against https://bugs.launchpad.net/zc.buildout/+bug/239212 - in allowhosts.txt (seletz) - - - further fixes for """AttributeError: Buildout instance has no - attribute '_logger'""" by providing reasonable defaults - within the Buildout constructor (related to the new 'allow-hosts' option) - (patch by Gottfried Ganssauge) (ajung) - - - 1.0.5 (2008-06-10) - ================== - - - Fixed wrong split when using the += and -= syntax (mustapha) - - 1.0.4 (2008-06-10) - ================== - - - Added the `allow-hosts` option (tarek) - - - Quote the 'executable' argument when trying to detect the python - version using popen4. (sidnei) - - - Quote the 'spec' argument, as in the case of installing an egg from - the buildout-cache, if the filename contains spaces it would fail (sidnei) - - - Extended configuration syntax to allow -= and += operators (malthe, mustapha). - - 1.0.3 (2008-06-01) - ================== - - - fix for """AttributeError: Buildout instance has no attribute '_logger'""" - by providing reasonable defaults within the Buildout constructor. - (patch by Gottfried Ganssauge) (ajung) - - 1.0.2 (2008-05-13) - ================== - - - More fixes for Windows. A quoted sh-bang is now used on Windows to make the - .exe files work with a Python executable in 'program files'. - - - Added "-t <timeout_in_seconds>" option for specifying the socket timeout. - (ajung) - - 1.0.1 (2008-04-02) - ================== - - - Made easy_install.py's _get_version accept non-final releases of Python, - like 2.4.4c0. (hannosch) - - - Applied various patches for Windows (patch by Gottfried Ganssauge). (ajung) - - - Applied patch fixing rmtree issues on Windows (patch by - Gottfried Ganssauge). (ajung) - - 1.0.0 (2008-01-13) - ================== - - - Added a French translation of the buildout tutorial. - - 1.0.0b31 (2007-11-01) - ===================== - - Feature Changes - --------------- - - - Added a configuration option that allows buildouts to ignore - dependency_links metadata specified in setup. By default - dependency_links in setup are used in addition to buildout specified - find-links. This can make it hard to control where eggs come - from. Here's how to tell buildout to ignore URLs in - dependency_links:: - - [buildout] - use-dependency-links = false - - By default use-dependency-links is true, which matches the behavior - of previous versions of buildout. - - - Added a configuration option that causes buildout to error if a - version is picked. This is a nice safety belt when fixing all - versions is intended, especially when creating releases. - - Bugs Fixed - ---------- - - - 151820: Develop failed if the setup.py script imported modules in - the distribution directory. - - - Verbose logging of the develop command was omitting detailed - output. - - - The setup command wasn't documented. - - - The setup command failed if run in a directory without specifying a - configuration file. - - - The setup command raised a stupid exception if run without arguments. - - - When using a local find links or index, distributions weren't copied - to the download cache. - - - When installing from source releases, a version specification (via a - buildout versions section) for setuptools was ignored when deciding - which setuptools to use to build an egg from the source release. - - 1.0.0b30 (2007-08-20) - ===================== - - Feature Changes - --------------- - - - Changed the default policy back to what it was to avoid breakage in - existing buildouts. Use:: - - [buildout] - prefer-final = true - - to get the new policy. The new policy will go into effect in - buildout 2. - - 1.0.0b29 (2007-08-20) - ===================== - - Feature Changes - --------------- - - - Now, final distributions are prefered over non-final versions. If - both final and non-final versions satisfy a requirement, then the - final version will be used even if it is older. The normal way to - override this for specific packages is to specifically require a - non-final version, either specifically or via a lower bound. - - - There is a buildout prefer-final version that can be used with a - value of "false":: - - prefer-final = false - - To prefer newer versions, regardless of whether or not they are - final, buildout-wide. - - - The new simple Python index, http://cheeseshop.python.org/simple, is - used as the default index. This will provide better performance - than the human package index interface, - http://pypi.python.org/pypi. More importantly, it lists hidden - distributions, so buildouts with fixed distribution versions will be - able to find old distributions even if the distributions have been - hidden in the human PyPI interface. - - Bugs Fixed - ---------- - - - 126441: Look for default.cfg in the right place on Windows. - - 1.0.0b28 (2007-07-05) - ===================== - - Bugs Fixed - ---------- - - - When requiring a specific version, buildout looked for new versions - even if that single version was already installed. - - 1.0.0b27 (2007-06-20) - ===================== - - Bugs Fixed - ---------- - - - Scripts were generated incorrectly on Windows. This included the - buildout script itself, making buildout completely unusable. - - 1.0.0b26 (2007-06-19) - ===================== - - Feature Changes - --------------- - - - Thanks to recent fixes in setuptools, I was able to change buildout - to use find-link and index information when searching extensions. - - Sadly, this work, especially the timing, was motivated my the need - to use alternate indexes due to performance problems in the cheese - shop (http://www.python.org/pypi/). I really home we can address - these performance problems soon. - - 1.0.0b25 (2007-05-31) - ===================== - - Feature Changes - --------------- - - - buildout now changes to the buildout directory before running recipe - install and update methods. - - - Added a new init command for creating a new buildout. This creates - an empty configuration file and then bootstraps. - - - Except when using the new init command, it is now an error to run - buildout without a configuration file. - - - In verbose mode, when adding distributions to fulful requirements of - already-added distributions, we now show why the new distributions - are being added. - - - Changed the logging format to exclude the logger name for the - zc.buildout logger. This reduces noise in the output. - - - Clean up lots of messages, adding missing periods and adding quotes around - requirement strings and file paths. - - Bugs Fixed - ---------- - - - 114614: Buildouts could take a very long time if there were - dependency problems in large sets of pathologically interdependent - packages. - - - 59270: Buggy recipes can cause failures in later recipes via chdir - - - 61890: file:// urls don't seem to work in find-links - - setuptools requires that file urls that point to directories must - end in a "/". Added a workaround. - - - 75607: buildout should not run if it creates an empty buildout.cfg - - 1.0.0b24 (2007-05-09) - ===================== - - Feature Changes - --------------- - - - Improved error reporting by showing which packages require other - packages that can't be found or that cause version conflicts. - - - Added an API for use by recipe writers to clean up created files - when recipe errors occur. - - - Log installed scripts. - - Bugs Fixed - ---------- - - - 92891: bootstrap crashes with recipe option in buildout section. - - - 113085: Buildout exited with a zero exist status when internal errors - occurred. - - - 1.0.0b23 (2007-03-19) - ===================== - - Feature Changes - --------------- - - - Added support for download caches. A buildout can specify a cache - for distribution downloads. The cache can be shared among buildouts - to reduce network access and to support creating source - distributions for applications allowing install without network - access. - - - Log scripts created, as suggested in: - https://bugs.launchpad.net/zc.buildout/+bug/71353 - - Bugs Fixed - ---------- - - - It wasn't possible to give options on the command line for sections - not defined in a configuration file. - - 1.0.0b22 (2007-03-15) - ===================== - - Feature Changes - --------------- - - - Improved error reporting and debugging support: - - - Added "logical tracebacks" that show functionally what the buildout - was doing when an error occurs. Don't show a Python traceback - unless the -D option is used. - - - Added a -D option that causes the buildout to print a traceback and - start the pdb post-mortem debugger when an error occurs. - - - Warnings are printed for unused options in the buildout section and - installed-part sections. This should make it easier to catch option - misspellings. - - - Changed the way the installed database (.installed.cfg) is handled - to avoid database corruption when a user breaks out of a buildout - with control-c. - - - Don't save an installed database if there are no installed parts or - develop egg links. - - 1.0.0b21 (2007-03-06) - ===================== - - Feature Changes - --------------- - - - Added support for repeatable buildouts by allowing egg versions to - be specified in a versions section. - - - The easy_install module install and build functions now accept a - versions argument that supplied to mapping from project name to - version numbers. This can be used to fix version numbers for - required distributions and their depenencies. - - When a version isn't fixed, using either a versions option or using - a fixed version number in a requirement, then a debug log message is - emitted indicating the version picked. This is useful for setting - versions options. - - A default_versions function can be used to set a default value for - this option. - - - Adjusted the output for verbosity levels. Using a single -v option - no longer causes voluminous setuptools output. Uisng -vv and -vvv - now triggers extra setuptools output. - - - Added a remove testing helper function that removes files or directories. - - 1.0.0b20 (2007-02-08) - ===================== - - Feature Changes - --------------- - - - Added a buildout newest option, to control whether the newest - distributions should be sought to meet requirements. This might - also provide a hint to recipes that don't deal with - distributions. For example, a recipe that manages subversion - checkouts might not update a checkout if newest is set to "false". - - - Added a *newest* keyword parameter to the - zc.buildout.easy_install.install and zc.buildout.easy_install.build - functions to control whether the newest distributions that meed - given requirements should be sought. If a false value is provided - for this parameter and already installed eggs meet the given - requirements, then no attempt will be made to search for newer - distributions. - - - The recipe-testing support setUp function now adds the name - *buildout* to the test namespace with a value that is the path to - the buildout script in the sample buildout. This allows tests to - use - - >>> print system(buildout), - - rather than: - - >>> print system(join('bin', 'buildout')), - - - Bugs Fixed - ---------- - - - Paths returned from update methods replaced lists of installed files - rather than augmenting them. - - 1.0.0b19 (2007-01-24) - ===================== - - Bugs Fixed - ---------- - - - Explicitly specifying a Python executable failed if the output of - running Python with the -V option included a 2-digit (rather than a - 3-digit) version number. - - 1.0.0b18 (2007-01-22) - ===================== - - Feature Changes - --------------- - - - Added documentation for some previously undocumented features of the - easy_install APIs. - - - By popular demand, added a -o command-line option that is a short - hand for the assignment buildout:offline=true. - - Bugs Fixed - ---------- - - - When deciding whether recipe develop eggs had changed, buildout - incorrectly considered files in .svn and CVS directories. - - 1.0.0b17 (2006-12-07) - ===================== - - Feature Changes - --------------- - - - Configuration files can now be loaded from URLs. - - Bugs Fixed - ---------- - - - https://bugs.launchpad.net/products/zc.buildout/+bug/71246 - - Buildout extensions installed as eggs couldn't be loaded in offline - mode. - - - 1.0.0b16 (2006-12-07) - ===================== - - Feature Changes - --------------- - - - A new command-line argument, -U, suppresses reading user defaults. - - - You can now suppress use of an installed-part database - (e.g. .installed.cfg) by sprifying an empty value for the buildout - installed option. - - Bugs Fixed - ---------- - - - When the install command is used with a list of parts, only - those parts are supposed to be installed, but the buildout was also - building parts that those parts depended on. - - 1.0.0b15 (2006-12-06) - ===================== - - Bugs Fixed - ---------- - - - Uninstall recipes weren't loaded correctly in cases where - no parts in the (new) configuration used the recipe egg. - - 1.0.0b14 (2006-12-05) - ===================== - - Feature Changes - --------------- - - - Added uninstall recipes for dealing with complex uninstallation - scenarios. - - Bugs Fixed - ---------- - - - Automatic upgrades weren't performed on Windows due to a bug that - caused buildout to incorrectly determine that it wasn't running - locally in a buildout. - - - Fixed some spurious test failures on Windows. - - 1.0.0b13 (2006-12-04) - ===================== - - Feature Changes - --------------- - - - Variable substitutions now reflect option data written by recipes. - - - A part referenced by a part in a parts list is now added to the parts - list before the referencing part. This means that you can omit - parts from the parts list if they are referenced by other parts. - - - Added a develop function to the easy_install module to aid in - creating develop eggs with custom build_ext options. - - - The build and develop functions in the easy_install module now - return the path of the egg or egg link created. - - - Removed the limitation that parts named in the install command can - only name configured parts. - - - Removed support ConfigParser-style variable substitutions - (e.g. %(foo)s). Only the string-template style of variable - (e.g. ${section:option}) substitutions will be supported. - Supporting both violates "there's only one way to do it". - - - Deprecated the buildout-section extendedBy option. - - Bugs Fixed - ---------- - - - We treat setuptools as a dependency of any distribution that - (declares that it) uses namespace packages, whether it declares - setuptools as a dependency or not. This wasn't working for eggs - intalled by virtue of being dependencies. - - - 1.0.0b12 (2006-10-24) - ===================== - - Feature Changes - --------------- - - - Added an initialization argument to the - zc.buildout.easy_install.scripts function to include initialization - code in generated scripts. - - 1.0.0b11 (2006-10-24) - ===================== - - Bugs Fixed - ---------- - - `67737 <https://launchpad.net/products/zc.buildout/+bug/67737>`_ - Verbose and quite output options caused errors when the - develop buildout option was used to create develop eggs. - - `67871 <https://launchpad.net/products/zc.buildout/+bug/67871>`_ - Installation failed if the source was a (local) unzipped - egg. - - `67873 <https://launchpad.net/products/zc.buildout/+bug/67873>`_ - There was an error in producing an error message when part names - passed to the install command weren't included in the - configuration. - - 1.0.0b10 (2006-10-16) - ===================== - - Feature Changes - --------------- - - - Renamed the runsetup command to setup. (The old name still works.) - - - Added a recipe update method. Now install is only called when a part - is installed for the first time, or after an uninstall. Otherwise, - update is called. For backward compatibility, recipes that don't - define update methiods are still supported. - - - If a distribution defines namespace packages but fails to declare - setuptools as one of its dependencies, we now treat setuptools as an - implicit dependency. We generate a warning if the distribution - is a develop egg. - - - You can now create develop eggs for setup scripts that don't use setuptools. - - Bugs Fixed - ---------- - - - Egg links weren't removed when corresponding entries were removed - from develop sections. - - - Running a non-local buildout command (one not installed in the - buildout) ket to a hang if there were new versions of zc.buildout or - setuptools were available. Now we issue a warning and don't - upgrade. - - - When installing zip-safe eggs from local directories, the eggs were - moved, rather than copied, removing them from the source directory. - - 1.0.0b9 (2006-10-02) - ==================== - - Bugs Fixed - ---------- - - Non-zip-safe eggs were not unzipped when they were installed. - - 1.0.0b8 (2006-10-01) - ==================== - - Bugs Fixed - ---------- - - - Installing source distributions failed when using alternate Python - versions (depending on the versions of Python used.) - - - Installing eggs wasn't handled as efficiently as possible due to a - bug in egg URL parsing. - - - Fixed a bug in runsetup that caused setup scripts that introspected - __file__ to fail. - - 1.0.0b7 - ======= - - Added a documented testing framework for use by recipes. Refactored - the buildout tests to use it. - - Added a runsetup command run a setup script. This is handy if, like - me, you don't install setuptools in your system Python. - - 1.0.0b6 - ======= - - Fixed https://launchpad.net/products/zc.buildout/+bug/60582 - Use of extension options caused bootstrapping to fail if the eggs - directory didn't already exist. We no longer use extensions for - bootstrapping. There really isn't any reason to anyway. - - - 1.0.0b5 - ======= - - Refactored to do more work in buildout and less work in easy_install. - This makes things go a little faster, makes errors a little easier to - handle, and allows extensions (like the sftp extension) to influence - more of the process. This was done to fix a problem in using the sftp - support. - - 1.0.0b4 - ======= - - - Added an **experimental** extensions mechanism, mainly to support - adding sftp support to buildouts that need it. - - - Fixed buildout self-updating on Windows. - - 1.0.0b3 - ======= - - - Added a help option (-h, --help) - - - Increased the default level of verbosity. - - - Buildouts now automatically update themselves to new versions of - zc.buildout and setuptools. - - - Added Windows support. - - - Added a recipe API for generating user errors. - - - No-longer generate a py_zc.buildout script. - - - Fixed some bugs in variable substitutions. - - The characters "-", "." and " ", weren't allowed in section or - option names. - - Substitutions with invalid names were ignored, which caused - missleading failures downstream. - - - Improved error handling. No longer show tracebacks for user errors. - - - Now require a recipe option (and therefore a section) for every part. - - - Expanded the easy_install module API to: - - - Allow extra paths to be provided - - - Specify explicit entry points - - - Specify entry-point arguments - - 1.0.0b2 - ======= - - Added support for specifying some build_ext options when installing eggs - from source distributions. - - 1.0.0b1 - ======= - - - Changed the bootstrapping code to only install setuptools and - zc.buildout. The bootstrap code no-longer runs the buildout itself. - This was to fix a bug that caused parts to be recreated - unnecessarily because the recipe signature in the initial buildout - reflected temporary locations for setuptools and zc.buildout. - - - Now create a minimal setup.py if it doesn't exist and issue a - warning that it is being created. - - - Fixed bug in saving installed configuration data. %'s and extra - spaces weren't quoted. - - 1.0.0a1 - ======= - - Initial public version - - Download - ********************** - -Keywords: development build -Platform: UNKNOWN -Classifier: Intended Audience :: Developers -Classifier: License :: OSI Approved :: Zope Public License -Classifier: Topic :: Software Development :: Build Tools -Classifier: Topic :: Software Development :: Libraries :: Python Modules diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/SOURCES.txt b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/SOURCES.txt deleted file mode 100644 index ec4a9ff..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/SOURCES.txt +++ /dev/null @@ -1,76 +0,0 @@ -CHANGES.txt -DEVELOPERS.txt -README.txt -SYSTEM_PYTHON_HELP.txt -dev.py -setup.cfg -setup.py -test_all_pythons.cfg -todo.txt -bootstrap/bootstrap.py -doc/tutorial.fr.txt -doc/tutorial.txt -specifications/README.txt -specifications/repeatable.txt -src/zc/__init__.py -src/zc.buildout.egg-info/PKG-INFO -src/zc.buildout.egg-info/SOURCES.txt -src/zc.buildout.egg-info/dependency_links.txt -src/zc.buildout.egg-info/entry_points.txt -src/zc.buildout.egg-info/namespace_packages.txt -src/zc.buildout.egg-info/not-zip-safe -src/zc.buildout.egg-info/requires.txt -src/zc.buildout.egg-info/top_level.txt -src/zc/buildout/__init__.py -src/zc/buildout/allowhosts.txt -src/zc/buildout/bootstrap.txt -src/zc/buildout/buildout.py -src/zc/buildout/buildout.txt -src/zc/buildout/debugging.txt -src/zc/buildout/dependencylinks.txt -src/zc/buildout/distribute.txt -src/zc/buildout/download.py -src/zc/buildout/download.txt -src/zc/buildout/downloadcache.txt -src/zc/buildout/easy_install.py -src/zc/buildout/easy_install.txt -src/zc/buildout/extends-cache.txt -src/zc/buildout/repeatable.txt -src/zc/buildout/rmtree.py -src/zc/buildout/runsetup.txt -src/zc/buildout/setup.txt -src/zc/buildout/testing.py -src/zc/buildout/testing.txt -src/zc/buildout/testing_bugfix.txt -src/zc/buildout/testrecipes.py -src/zc/buildout/tests.py -src/zc/buildout/testselectingpython.py -src/zc/buildout/unzip.txt -src/zc/buildout/update.txt -src/zc/buildout/upgrading_distribute.txt -src/zc/buildout/virtualenv.txt -src/zc/buildout/windows.txt -z3c.recipe.scripts_/CHANGES.txt -z3c.recipe.scripts_/MANIFEST.in -z3c.recipe.scripts_/README.txt -z3c.recipe.scripts_/setup.py -z3c.recipe.scripts_/src/z3c/__init__.py -z3c.recipe.scripts_/src/z3c/recipe/__init__.py -z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt -z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py -z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py -z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py -zc.recipe.egg_/CHANGES.txt -zc.recipe.egg_/MANIFEST.in -zc.recipe.egg_/README.txt -zc.recipe.egg_/setup.py -zc.recipe.egg_/src/zc/__init__.py -zc.recipe.egg_/src/zc/recipe/__init__.py -zc.recipe.egg_/src/zc/recipe/egg/README.txt -zc.recipe.egg_/src/zc/recipe/egg/__init__.py -zc.recipe.egg_/src/zc/recipe/egg/api.txt -zc.recipe.egg_/src/zc/recipe/egg/custom.py -zc.recipe.egg_/src/zc/recipe/egg/custom.txt -zc.recipe.egg_/src/zc/recipe/egg/egg.py -zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt -zc.recipe.egg_/src/zc/recipe/egg/tests.py
\ No newline at end of file diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/dependency_links.txt b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/dependency_links.txt deleted file mode 100644 index 8b13789..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/dependency_links.txt +++ /dev/null @@ -1 +0,0 @@ - diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/entry_points.txt b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/entry_points.txt deleted file mode 100644 index 85beedf..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/entry_points.txt +++ /dev/null @@ -1,7 +0,0 @@ - -[console_scripts] -buildout = zc.buildout.buildout:main - -[zc.buildout] -debug = zc.buildout.testrecipes:Debug - diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/namespace_packages.txt b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/namespace_packages.txt deleted file mode 100644 index 7647cfa..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/namespace_packages.txt +++ /dev/null @@ -1 +0,0 @@ -zc diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/not-zip-safe b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/not-zip-safe deleted file mode 100644 index 8b13789..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/not-zip-safe +++ /dev/null @@ -1 +0,0 @@ - diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/requires.txt b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/requires.txt deleted file mode 100644 index 2d7b34d..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/requires.txt +++ /dev/null @@ -1,4 +0,0 @@ -setuptools - -[test] -zope.testing
\ No newline at end of file diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/top_level.txt b/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/top_level.txt deleted file mode 100644 index 7647cfa..0000000 --- a/eggs/zc.buildout-1.5.2-py2.6.egg/EGG-INFO/top_level.txt +++ /dev/null @@ -1 +0,0 @@ -zc |