diff options
Diffstat (limited to 'eggs/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.txt')
| -rw-r--r-- | eggs/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.txt | 1985 |
1 files changed, 1985 insertions, 0 deletions
diff --git a/eggs/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.txt b/eggs/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.txt new file mode 100644 index 0000000..ec654cd --- /dev/null +++ b/eggs/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.txt @@ -0,0 +1,1985 @@ +Python API for egg and script installation +========================================== + +The easy_install module provides some functions to provide support for +egg and script installation. It provides functionality at the python +level that is similar to easy_install, with a few exceptions: + +- By default, we look for new packages *and* the packages that + they depend on. This is somewhat like (and uses) the --upgrade + option of easy_install, except that we also upgrade required + packages. + +- If the highest-revision package satisfying a specification is + already present, then we don't try to get another one. This saves a + lot of search time in the common case that packages are pegged to + specific versions. + +- If there is a develop egg that satisfies a requirement, we don't + look for additional distributions. We always give preference to + develop eggs. + +- Distutils options for building extensions can be passed. + +Distribution installation +------------------------- + +The easy_install module provides a function, install, for installing one +or more packages and their dependencies. The install function takes 2 +positional arguments: + +- An iterable of setuptools requirement strings for the distributions + to be installed, and + +- A destination directory to install to and to satisfy requirements + from. The destination directory can be None, in which case, no new + distributions are downloaded and there will be an error if the + needed distributions can't be found among those already installed. + +It supports a number of optional keyword arguments: + +links + A sequence of URLs, file names, or directories to look for + links to distributions. + +index + The URL of an index server, or almost any other valid URL. :) + + If not specified, the Python Package Index, + http://pypi.python.org/simple/, is used. You can specify an + alternate index with this option. If you use the links option and + if the links point to the needed distributions, then the index can + be anything and will be largely ignored. In the examples, here, + we'll just point to an empty directory on our link server. This + will make our examples run a little bit faster. + +executable + A path to a Python executable. Distributions will be installed + using this executable and will be for the matching Python version. + +path + A list of additional directories to search for locally-installed + distributions. + +always_unzip + A flag indicating that newly-downloaded distributions should be + directories even if they could be installed as zip files. + +working_set + An existing working set to be augmented with additional + distributions, if necessary to satisfy requirements. This allows + you to call install multiple times, if necessary, to gather + multiple sets of requirements. + +newest + A boolean value indicating whether to search for new distributions + when already-installed distributions meet the requirement. When + this is true, the default, and when the destination directory is + not None, then the install function will search for the newest + distributions that satisfy the requirements. + +versions + A dictionary mapping project names to version numbers to be used + when selecting distributions. This can be used to specify a set of + distribution versions independent of other requirements. + +use_dependency_links + A flag indicating whether to search for dependencies using the + setup dependency_links metadata or not. If true, links are searched + for using dependency_links in preference to other + locations. Defaults to true. + +include_site_packages + A flag indicating whether Python's non-standard-library packages should + be available for finding dependencies. Defaults to true. + + Paths outside of Python's standard library--or more precisely, those that + are not included when Python is started with the -S argument--are loosely + referred to as "site-packages" here. + +relative_paths + Adjust egg paths so they are relative to the script path. This + allows scripts to work when scripts and eggs are moved, as long as + they are both moved in the same way. + +The install method returns a working set containing the distributions +needed to meet the given requirements. + +We have a link server that has a number of eggs: + + >>> print get(link_server), + <html><body> + <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> + <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> + <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> + <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> + <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> + <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> + <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> + <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> + <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> + <a href="index/">index/</a><br> + <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> + </body></html> + +Let's make a directory and install the demo egg to it, using the demo: + + >>> dest = tmpdir('sample-install') + >>> import zc.buildout.easy_install + >>> ws = zc.buildout.easy_install.install( + ... ['demo==0.2'], dest, + ... links=[link_server], index=link_server+'index/') + +We requested version 0.2 of the demo distribution to be installed into +the destination server. We specified that we should search for links +on the link server and that we should use the (empty) link server +index directory as a package index. + +The working set contains the distributions we retrieved. + + >>> for dist in ws: + ... print dist + demo 0.2 + demoneeded 1.1 + +We got demoneeded because it was a dependency of demo. + +And the actual eggs were added to the eggs directory. + + >>> ls(dest) + - demo-0.2-py2.4.egg + - demoneeded-1.1-py2.4.egg + +If we remove the version restriction on demo, but specify a false +value for newest, no new distributions will be installed: + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... newest=False) + >>> ls(dest) + - demo-0.2-py2.4.egg + - demoneeded-1.1-py2.4.egg + +If we leave off the newest option, we'll get an update for demo: + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/') + >>> ls(dest) + - demo-0.2-py2.4.egg + - demo-0.3-py2.4.egg + - demoneeded-1.1-py2.4.egg + +Note that we didn't get the newest versions available. There were +release candidates for newer versions of both packages. By default, +final releases are preferred. We can change this behavior using the +prefer_final function: + + >>> zc.buildout.easy_install.prefer_final(False) + True + +The old setting is returned. + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/') + >>> for dist in ws: + ... print dist + demo 0.4c1 + demoneeded 1.2c1 + + >>> ls(dest) + - demo-0.2-py2.4.egg + - demo-0.3-py2.4.egg + - demo-0.4c1-py2.4.egg + - demoneeded-1.1-py2.4.egg + - demoneeded-1.2c1-py2.4.egg + +Let's put the setting back to the default. + + >>> zc.buildout.easy_install.prefer_final(True) + False + +We can supply additional distributions. We can also supply +specifications for distributions that would normally be found via +dependencies. We might do this to specify a specific version. + + >>> ws = zc.buildout.easy_install.install( + ... ['demo', 'other', 'demoneeded==1.0'], dest, + ... links=[link_server], index=link_server+'index/') + + >>> for dist in ws: + ... print dist + demo 0.3 + other 1.0 + demoneeded 1.0 + + >>> ls(dest) + - demo-0.2-py2.4.egg + - demo-0.3-py2.4.egg + - demo-0.4c1-py2.4.egg + - demoneeded-1.0-py2.4.egg + - demoneeded-1.1-py2.4.egg + - demoneeded-1.2c1-py2.4.egg + d other-1.0-py2.4.egg + +We can request that eggs be unzipped even if they are zip safe. This +can be useful when debugging. (Note that Distribute will unzip eggs by +default, so if you are using Distribute, most or all eggs will already be +unzipped without this flag.) + + >>> rmdir(dest) + >>> dest = tmpdir('sample-install') + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... always_unzip=True) + + >>> ls(dest) + d demo-0.3-py2.4.egg + d demoneeded-1.1-py2.4.egg + + >>> rmdir(dest) + >>> dest = tmpdir('sample-install') + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... always_unzip=False) + + >>> ls(dest) + - demo-0.3-py2.4.egg + - demoneeded-1.1-py2.4.egg + +We can also set a default by calling the always_unzip function: + + >>> zc.buildout.easy_install.always_unzip(True) + False + +The old default is returned: + + >>> rmdir(dest) + >>> dest = tmpdir('sample-install') + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/') + + >>> ls(dest) + d demo-0.3-py2.4.egg + d demoneeded-1.1-py2.4.egg + + + >>> zc.buildout.easy_install.always_unzip(False) + True + + >>> rmdir(dest) + >>> dest = tmpdir('sample-install') + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/') + + >>> ls(dest) + - demo-0.3-py2.4.egg + - demoneeded-1.1-py2.4.egg + + >>> rmdir(dest) + >>> dest = tmpdir('sample-install') + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... always_unzip=True) + + >>> ls(dest) + d demo-0.3-py2.4.egg + d demoneeded-1.1-py2.4.egg + +Specifying version information independent of requirements +---------------------------------------------------------- + +Sometimes it's useful to specify version information independent of +normal requirements specifications. For example, a buildout may need +to lock down a set of versions, without having to put put version +numbers in setup files or part definitions. If a dictionary is passed +to the install function, mapping project names to version numbers, +then the versions numbers will be used. + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... versions = dict(demo='0.2', demoneeded='1.0')) + >>> [d.version for d in ws] + ['0.2', '1.0'] + +In this example, we specified a version for demoneeded, even though we +didn't define a requirement for it. The versions specified apply to +dependencies as well as the specified requirements. + +If we specify a version that's incompatible with a requirement, then +we'll get an error: + + >>> from zope.testing.loggingsupport import InstalledHandler + >>> handler = InstalledHandler('zc.buildout.easy_install') + >>> import logging + >>> logging.getLogger('zc.buildout.easy_install').propagate = False + + >>> ws = zc.buildout.easy_install.install( + ... ['demo >0.2'], dest, links=[link_server], + ... index=link_server+'index/', + ... versions = dict(demo='0.2', demoneeded='1.0')) + Traceback (most recent call last): + ... + IncompatibleVersionError: Bad version 0.2 + + >>> print handler + zc.buildout.easy_install DEBUG + Installing 'demo >0.2'. + zc.buildout.easy_install ERROR + The version, 0.2, is not consistent with the requirement, 'demo>0.2'. + + >>> handler.clear() + +If no versions are specified, a debugging message will be output +reporting that a version was picked automatically: + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... ) + + >>> print handler + zc.buildout.easy_install DEBUG + Installing 'demo'. + zc.buildout.easy_install DEBUG + We have the best distribution that satisfies 'demo'. + zc.buildout.easy_install DEBUG + Picked: demo = 0.3 + zc.buildout.easy_install DEBUG + Getting required 'demoneeded' + zc.buildout.easy_install DEBUG + required by demo 0.3. + zc.buildout.easy_install DEBUG + We have the best distribution that satisfies 'demoneeded'. + zc.buildout.easy_install DEBUG + Picked: demoneeded = 1.1 + + >>> handler.uninstall() + >>> logging.getLogger('zc.buildout.easy_install').propagate = True + +We can request that we get an error if versions are picked: + + >>> zc.buildout.easy_install.allow_picked_versions(False) + True + +(The old setting is returned.) + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... ) + Traceback (most recent call last): + ... + UserError: Picked: demo = 0.3 + + >>> zc.buildout.easy_install.allow_picked_versions(True) + False + +The function default_versions can be used to get and set default +version information to be used when no version information is passes. +If called with an argument, it sets the default versions: + + >>> zc.buildout.easy_install.default_versions(dict(demoneeded='1')) + {} + +It always returns the previous default versions. If called without an +argument, it simply returns the default versions without changing +them: + + >>> zc.buildout.easy_install.default_versions() + {'demoneeded': '1'} + +So with the default versions set, we'll get the requested version even +if the versions option isn't used: + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... ) + + >>> [d.version for d in ws] + ['0.3', '1.0'] + +Of course, we can unset the default versions by passing an empty +dictionary: + + >>> zc.buildout.easy_install.default_versions({}) + {'demoneeded': '1'} + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, links=[link_server], index=link_server+'index/', + ... ) + + >>> [d.version for d in ws] + ['0.3', '1.1'] + +Dependencies in Site Packages +----------------------------- + +Paths outside of Python's standard library--or more precisely, those that are +not included when Python is started with the -S argument--are loosely referred +to as "site-packages" here. These site-packages are searched by default for +distributions. This can be disabled, so that, for instance, a system Python +can be used with buildout, cleaned of any packages installed by a user or +system package manager. + +The default behavior can be controlled and introspected using +zc.buildout.easy_install.include_site_packages. + + >>> zc.buildout.easy_install.include_site_packages() + True + +Here's an example of using a Python executable that includes our dependencies. + +Our "py_path" will have the "demoneeded," and "demo" packages available. + We'll simply be asking for "demoneeded" here, but without any external + index or links. + + >>> from zc.buildout.tests import create_sample_sys_install + >>> py_path, site_packages_path = make_py() + >>> create_sample_sys_install(site_packages_path) + + >>> example_dest = tmpdir('site-packages-example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['demoneeded'], example_dest, links=[], executable=py_path, + ... index=None) + >>> [dist.project_name for dist in workingset] + ['demoneeded'] + +That worked fine. Let's try again with site packages not allowed. We'll +change the policy by changing the default. Notice that the function for +changing the default value returns the previous value. + + >>> zc.buildout.easy_install.include_site_packages(False) + True + + >>> zc.buildout.easy_install.include_site_packages() + False + + >>> zc.buildout.easy_install.clear_index_cache() + >>> rmdir(example_dest) + >>> example_dest = tmpdir('site-packages-example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['demoneeded'], example_dest, links=[], executable=py_path, + ... index=None) + Traceback (most recent call last): + ... + MissingDistribution: Couldn't find a distribution for 'demoneeded'. + >>> zc.buildout.easy_install.clear_index_cache() + +Now we'll reset the default. + + >>> zc.buildout.easy_install.include_site_packages(True) + False + + >>> zc.buildout.easy_install.include_site_packages() + True + +Dependency links +---------------- + +Setuptools allows metadata that describes where to search for package +dependencies. This option is called dependency_links. Buildout has its +own notion of where to look for dependencies, but it also uses the +setup tools dependency_links information if it's available. + +Let's demo this by creating an egg that specifies dependency_links. + +To begin, let's create a new egg repository. This repository hold a +newer version of the 'demoneeded' egg than the sample repository does. + + >>> repoloc = tmpdir('repo') + >>> from zc.buildout.tests import create_egg + >>> create_egg('demoneeded', '1.2', repoloc) + >>> link_server2 = start_server(repoloc) + +Turn on logging on this server so that we can see when eggs are pulled +from it. + + >>> get(link_server2 + 'enable_server_logging') + GET 200 /enable_server_logging + '' + +Now we can create an egg that specifies that its dependencies are +found on this server. + + >>> repoloc = tmpdir('repo2') + >>> create_egg('hasdeps', '1.0', repoloc, + ... install_requires = "'demoneeded'", + ... dependency_links = [link_server2]) + +Let's add the egg to another repository. + + >>> link_server3 = start_server(repoloc) + +Now let's install the egg. + + >>> example_dest = tmpdir('example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['hasdeps'], example_dest, + ... links=[link_server3], index=link_server3+'index/') + GET 200 / + GET 200 /demoneeded-1.2-pyN.N.egg + +The server logs show that the dependency was retrieved from the server +specified in the dependency_links. + +Now let's see what happens if we provide two different ways to retrieve +the dependencies. + + >>> rmdir(example_dest) + >>> example_dest = tmpdir('example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['hasdeps'], example_dest, index=link_server+'index/', + ... links=[link_server, link_server3]) + GET 200 / + GET 200 /demoneeded-1.2-pyN.N.egg + +Once again the dependency is fetched from the logging server even +though it is also available from the non-logging server. This is +because the version on the logging server is newer and buildout +normally chooses the newest egg available. + +If you wish to control where dependencies come from regardless of +dependency_links setup metadata use the 'use_dependency_links' option +to zc.buildout.easy_install.install(). + + >>> rmdir(example_dest) + >>> example_dest = tmpdir('example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['hasdeps'], example_dest, index=link_server+'index/', + ... links=[link_server, link_server3], + ... use_dependency_links=False) + +Notice that this time the dependency egg is not fetched from the +logging server. When you specify not to use dependency_links, eggs +will only be searched for using the links you explicitly provide. + +Another way to control this option is with the +zc.buildout.easy_install.use_dependency_links() function. This +function sets the default behavior for the zc.buildout.easy_install() +function. + + >>> zc.buildout.easy_install.use_dependency_links(False) + True + +The function returns its previous setting. + + >>> rmdir(example_dest) + >>> example_dest = tmpdir('example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['hasdeps'], example_dest, index=link_server+'index/', + ... links=[link_server, link_server3]) + +It can be overridden by passing a keyword argument to the install +function. + + >>> rmdir(example_dest) + >>> example_dest = tmpdir('example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['hasdeps'], example_dest, index=link_server+'index/', + ... links=[link_server, link_server3], + ... use_dependency_links=True) + GET 200 /demoneeded-1.2-pyN.N.egg + +To return the dependency_links behavior to normal call the function again. + + >>> zc.buildout.easy_install.use_dependency_links(True) + False + >>> rmdir(example_dest) + >>> example_dest = tmpdir('example-install') + >>> workingset = zc.buildout.easy_install.install( + ... ['hasdeps'], example_dest, index=link_server+'index/', + ... links=[link_server, link_server3]) + GET 200 /demoneeded-1.2-pyN.N.egg + + +Script generation +----------------- + +The easy_install module provides support for creating scripts from eggs. +It provides two competing functions. One, ``scripts``, is a +well-established approach to generating reliable scripts with a "clean" +Python--e.g., one that does not have any packages in its site-packages. +The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is +designed to work with a Python that has code in its site-packages, such +as a system Python. + +Both are similar to setuptools except that they provides facilities for +baking a script's path into the script. This has two advantages: + +- The eggs to be used by a script are not chosen at run time, making + startup faster and, more importantly, deterministic. + +- The script doesn't have to import pkg_resources because the logic that + pkg_resources would execute at run time is executed at script-creation + time. (There is an exception in ``sitepackage_safe_scripts`` if you + want to have your Python's site packages available, as discussed + below, but even in that case pkg_resources is only partially + activated, which can be a significant time savings.) + + +The ``scripts`` function +~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``scripts`` function is the first way to generate scripts that we'll +examine. It is the earlier approach that the package offered. Let's +create a destination directory for it to place them in: + + >>> bin = tmpdir('bin') + +Now, we'll use the scripts function to generate scripts in this directory +from the demo egg: + + >>> import sys + >>> scripts = zc.buildout.easy_install.scripts( + ... ['demo'], ws, sys.executable, bin) + +the four arguments we passed were: + +1. A sequence of distribution requirements. These are of the same + form as setuptools requirements. Here we passed a single + requirement, for the version 0.1 demo distribution. + +2. A working set, + +3. The Python executable to use, and + +3. The destination directory. + +The bin directory now contains a generated script: + + >>> ls(bin) + - demo + +The return value is a list of the scripts generated: + + >>> import os, sys + >>> if sys.platform == 'win32': + ... scripts == [os.path.join(bin, 'demo.exe'), + ... os.path.join(bin, 'demo-script.py')] + ... else: + ... scripts == [os.path.join(bin, 'demo')] + True + +Note that in Windows, 2 files are generated for each script. A script +file, ending in '-script.py', and an exe file that allows the script +to be invoked directly without having to specify the Python +interpreter and without having to provide a '.py' suffix. + +The demo script run the entry point defined in the demo egg: + + >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + <BLANKLINE> + import sys + sys.path[0:0] = [ + '/sample-install/demo-0.3-py2.4.egg', + '/sample-install/demoneeded-1.1-py2.4.egg', + ] + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main() + +Some things to note: + +- The demo and demoneeded eggs are added to the beginning of sys.path. + +- The module for the script entry point is imported and the entry + point, in this case, 'main', is run. + +Rather than requirement strings, you can pass tuples containing 3 +strings: + + - A script name, + + - A module, + + - An attribute expression for an entry point within the module. + +For example, we could have passed entry point information directly +rather than passing a requirement: + + >>> scripts = zc.buildout.easy_install.scripts( + ... [('demo', 'eggrecipedemo', 'main')], + ... ws, sys.executable, bin) + + >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + <BLANKLINE> + import sys + sys.path[0:0] = [ + '/sample-install/demo-0.3-py2.4.egg', + '/sample-install/demoneeded-1.1-py2.4.egg', + ] + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main() + +Passing entry-point information directly is handy when using eggs (or +distributions) that don't declare their entry points, such as +distributions that aren't based on setuptools. + +The interpreter keyword argument can be used to generate a script that can +be used to invoke the Python interactive interpreter with the path set +based on the working set. This generated script can also be used to +run other scripts with the path set on the working set: + + >>> scripts = zc.buildout.easy_install.scripts( + ... ['demo'], ws, sys.executable, bin, interpreter='py') + + + >>> ls(bin) + - demo + - py + + >>> if sys.platform == 'win32': + ... scripts == [os.path.join(bin, 'demo.exe'), + ... os.path.join(bin, 'demo-script.py'), + ... os.path.join(bin, 'py.exe'), + ... os.path.join(bin, 'py-script.py')] + ... else: + ... scripts == [os.path.join(bin, 'demo'), + ... os.path.join(bin, 'py')] + True + +The py script simply runs the Python interactive interpreter with +the path set: + + >>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + <BLANKLINE> + import sys + <BLANKLINE> + sys.path[0:0] = [ + '/sample-install/demo-0.3-pyN.N.egg', + '/sample-install/demoneeded-1.1-pyN.N.egg', + ] + <BLANKLINE> + _interactive = True + if len(sys.argv) > 1: + _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') + _interactive = False + for (_opt, _val) in _options: + if _opt == '-i': + _interactive = True + elif _opt == '-c': + exec _val + elif _opt == '-m': + sys.argv[1:] = _args + _args = [] + __import__("runpy").run_module( + _val, {}, "__main__", alter_sys=True) + <BLANKLINE> + if _args: + sys.argv[:] = _args + __file__ = _args[0] + del _options, _args + execfile(__file__) + <BLANKLINE> + if _interactive: + del _interactive + __import__("code").interact(banner="", local=globals()) + +If invoked with a script name and arguments, it will run that script, instead. + + >>> write('ascript', ''' + ... "demo doc" + ... print sys.argv + ... print (__name__, __file__, __doc__) + ... ''') + >>> print system(join(bin, 'py')+' ascript a b c'), + ['ascript', 'a', 'b', 'c'] + ('__main__', 'ascript', 'demo doc') + +For Python 2.5 and higher, you can also use the -m option to run a +module: + + >>> print system(join(bin, 'py')+' -m pdb'), + usage: pdb.py scriptfile [arg] ... + + >>> print system(join(bin, 'py')+' -m pdb what'), + Error: what does not exist + +An additional argument can be passed to define which scripts to install +and to provide script names. The argument is a dictionary mapping +original script names to new script names. + + >>> bin = tmpdir('bin2') + >>> scripts = zc.buildout.easy_install.scripts( + ... ['demo'], ws, sys.executable, bin, dict(demo='run')) + + >>> if sys.platform == 'win32': + ... scripts == [os.path.join(bin, 'run.exe'), + ... os.path.join(bin, 'run-script.py')] + ... else: + ... scripts == [os.path.join(bin, 'run')] + True + >>> ls(bin) + - run + + >>> print system(os.path.join(bin, 'run')), + 3 1 + +The ``scripts`` function: Including extra paths in scripts +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We can pass a keyword argument, extra paths, to cause additional paths +to be included in the a generated script: + + >>> foo = tmpdir('foo') + >>> scripts = zc.buildout.easy_install.scripts( + ... ['demo'], ws, sys.executable, bin, dict(demo='run'), + ... extra_paths=[foo]) + + >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + <BLANKLINE> + import sys + sys.path[0:0] = [ + '/sample-install/demo-0.3-py2.4.egg', + '/sample-install/demoneeded-1.1-py2.4.egg', + '/foo', + ] + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main() + +The ``scripts`` function: Providing script arguments +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An "argument" keyword argument can be used to pass arguments to an +entry point. The value passed is a source string to be placed between the +parentheses in the call: + + >>> scripts = zc.buildout.easy_install.scripts( + ... ['demo'], ws, sys.executable, bin, dict(demo='run'), + ... arguments='1, 2') + + >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + import sys + sys.path[0:0] = [ + '/sample-install/demo-0.3-py2.4.egg', + '/sample-install/demoneeded-1.1-py2.4.egg', + ] + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main(1, 2) + +The ``scripts`` function: Passing initialization code +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can also pass script initialization code: + + >>> scripts = zc.buildout.easy_install.scripts( + ... ['demo'], ws, sys.executable, bin, dict(demo='run'), + ... arguments='1, 2', + ... initialization='import os\nos.chdir("foo")') + + >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + import sys + sys.path[0:0] = [ + '/sample-install/demo-0.3-py2.4.egg', + '/sample-install/demoneeded-1.1-py2.4.egg', + ] + <BLANKLINE> + import os + os.chdir("foo") + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main(1, 2) + +The ``scripts`` function: Relative paths +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Sometimes, you want to be able to move a buildout directory around and +have scripts still work without having to rebuild them. We can +control this using the relative_paths option to install. You need +to pass a common base directory of the scripts and eggs: + + >>> bo = tmpdir('bo') + >>> ba = tmpdir('ba') + >>> mkdir(bo, 'eggs') + >>> mkdir(bo, 'bin') + >>> mkdir(bo, 'other') + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], join(bo, 'eggs'), links=[link_server], + ... index=link_server+'index/') + + >>> scripts = zc.buildout.easy_install.scripts( + ... ['demo'], ws, sys.executable, join(bo, 'bin'), dict(demo='run'), + ... extra_paths=[ba, join(bo, 'bar')], + ... interpreter='py', + ... relative_paths=bo) + + >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + <BLANKLINE> + import os + <BLANKLINE> + join = os.path.join + base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) + base = os.path.dirname(base) + <BLANKLINE> + import sys + sys.path[0:0] = [ + join(base, 'eggs/demo-0.3-pyN.N.egg'), + join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), + '/ba', + join(base, 'bar'), + ] + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main() + +Note that the extra path we specified that was outside the directory +passed as relative_paths wasn't converted to a relative path. + +Of course, running the script works: + + >>> print system(join(bo, 'bin', 'run')), + 3 1 + +We specified an interpreter and its paths are adjusted too: + + >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 + <BLANKLINE> + import os + <BLANKLINE> + join = os.path.join + base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) + base = os.path.dirname(base) + <BLANKLINE> + import sys + <BLANKLINE> + sys.path[0:0] = [ + join(base, 'eggs/demo-0.3-pyN.N.egg'), + join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), + '/ba', + join(base, 'bar'), + ] + <BLANKLINE> + _interactive = True + if len(sys.argv) > 1: + _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') + _interactive = False + for (_opt, _val) in _options: + if _opt == '-i': + _interactive = True + elif _opt == '-c': + exec _val + elif _opt == '-m': + sys.argv[1:] = _args + _args = [] + __import__("runpy").run_module( + _val, {}, "__main__", alter_sys=True) + <BLANKLINE> + if _args: + sys.argv[:] = _args + __file__ = _args[0] + del _options, _args + execfile(__file__) + <BLANKLINE> + if _interactive: + del _interactive + __import__("code").interact(banner="", local=globals()) + +The ``sitepackage_safe_scripts`` function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The newer function for creating scripts is ``sitepackage_safe_scripts``. +It has the same basic functionality as the ``scripts`` function: it can +create scripts to run arbitrary entry points, and to run a Python +interpreter. The following are the differences from a user's +perspective. + +- It can be used safely with a Python that has packages installed itself, + such as a system-installed Python. + +- In contrast to the interpreter generated by the ``scripts`` method, which + supports only a small subset of the usual Python executable's options, + the interpreter generated by ``sitepackage_safe_scripts`` supports all + of them. This makes it possible to use as full Python replacement for + scripts that need the distributions specified in your buildout. + +- Both the interpreter and the entry point scripts allow you to include the + site packages, and/or the sitecustomize, of the Python executable, if + desired. + +It works by creating site.py and sitecustomize.py files that set up the +desired paths and initialization. These must be placed within an otherwise +empty directory. Typically this is in a recipe's parts directory. + +Here's the simplest example, building an interpreter script. + + >>> interpreter_dir = tmpdir('interpreter') + >>> interpreter_parts_dir = os.path.join( + ... interpreter_dir, 'parts', 'interpreter') + >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin') + >>> mkdir(interpreter_bin_dir) + >>> mkdir(interpreter_dir, 'eggs') + >>> mkdir(interpreter_dir, 'parts') + >>> mkdir(interpreter_parts_dir) + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], + ... index=link_server+'index/') + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... interpreter='py') + +Depending on whether the machine being used is running Windows or not, this +produces either three or four files. In both cases, we have site.py and +sitecustomize.py generated in the parts/interpreter directory. For Windows, +we have py.exe and py-script.py; for other operating systems, we have py. + + >>> sitecustomize_path = os.path.join( + ... interpreter_parts_dir, 'sitecustomize.py') + >>> site_path = os.path.join(interpreter_parts_dir, 'site.py') + >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py') + >>> if sys.platform == 'win32': + ... py_path = os.path.join(interpreter_bin_dir, 'py-script.py') + ... expected = [sitecustomize_path, + ... site_path, + ... os.path.join(interpreter_bin_dir, 'py.exe'), + ... py_path] + ... else: + ... py_path = interpreter_path + ... expected = [sitecustomize_path, site_path, py_path] + ... + >>> assert generated == expected, repr((generated, expected)) + +We didn't ask for any initialization, and we didn't ask to use the underlying +sitecustomization, so sitecustomize.py is empty. + + >>> cat(sitecustomize_path) + +The interpreter script is simple. It puts the directory with the +site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python. + + >>> cat(py_path) + #!/usr/bin/python -S + import os + import sys + <BLANKLINE> + argv = [sys.executable] + sys.argv[1:] + environ = os.environ.copy() + path = '/interpreter/parts/interpreter' + if environ.get('PYTHONPATH'): + path = os.pathsep.join([path, environ['PYTHONPATH']]) + environ['PYTHONPATH'] = path + os.execve(sys.executable, argv, environ) + +The site.py file is a modified version of the underlying Python's site.py. +The most important modification is that it has a different version of the +addsitepackages function. It sets up the Python path, similarly to the +behavior of the function it replaces. The following shows the part that +buildout inserts, in the simplest case. + + >>> sys.stdout.write('#\n'); cat(site_path) + ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + #... + def addsitepackages(known_paths): + """Add site packages, as determined by zc.buildout. + <BLANKLINE> + See original_addsitepackages, below, for the original version.""" + buildout_paths = [ + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' + ] + for path in buildout_paths: + sitedir, sitedircase = makepath(path) + if not sitedircase in known_paths and os.path.exists(sitedir): + sys.path.append(sitedir) + known_paths.add(sitedircase) + return known_paths + <BLANKLINE> + def original_addsitepackages(known_paths):... + +Here are some examples of the interpreter in use. + + >>> print call_py(interpreter_path, "print 16+26") + 42 + <BLANKLINE> + >>> res = call_py(interpreter_path, "import sys; print sys.path") + >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + ['', + '/interpreter/parts/interpreter', + ..., + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] + <BLANKLINE> + >>> clean_paths = eval(res.strip()) # This is used later for comparison. + +If you provide initialization, it goes in sitecustomize.py. + + >>> def reset_interpreter(): + ... # This is necessary because, in our tests, the timestamps of the + ... # .pyc files are not outdated when we want them to be. + ... rmdir(interpreter_bin_dir) + ... mkdir(interpreter_bin_dir) + ... rmdir(interpreter_parts_dir) + ... mkdir(interpreter_parts_dir) + ... + >>> reset_interpreter() + + >>> initialization_string = """\ + ... import os + ... os.environ['FOO'] = 'bar baz bing shazam'""" + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... interpreter='py', initialization=initialization_string) + >>> cat(sitecustomize_path) + import os + os.environ['FOO'] = 'bar baz bing shazam' + >>> print call_py(interpreter_path, "import os; print os.environ['FOO']") + bar baz bing shazam + <BLANKLINE> + +If you use relative paths, this affects the interpreter and site.py. (This is +again the UNIX version; the Windows version uses subprocess instead of +os.execve.) + + >>> reset_interpreter() + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... interpreter='py', relative_paths=interpreter_dir) + >>> cat(py_path) + #!/usr/bin/python -S + import os + import sys + <BLANKLINE> + join = os.path.join + base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) + base = os.path.dirname(base) + <BLANKLINE> + argv = [sys.executable] + sys.argv[1:] + environ = os.environ.copy() + path = join(base, 'parts/interpreter') + if environ.get('PYTHONPATH'): + path = os.pathsep.join([path, environ['PYTHONPATH']]) + environ['PYTHONPATH'] = path + os.execve(sys.executable, argv, environ) + +For site.py, we again show only the pertinent parts. Notice that the egg +paths join a base to a path, as with the use of this argument in the +``scripts`` function. + + >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS + #... + def addsitepackages(known_paths): + """Add site packages, as determined by zc.buildout. + <BLANKLINE> + See original_addsitepackages, below, for the original version.""" + join = os.path.join + base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) + base = os.path.dirname(base) + base = os.path.dirname(base) + buildout_paths = [ + join(base, 'eggs/demo-0.3-pyN.N.egg'), + join(base, 'eggs/demoneeded-1.1-pyN.N.egg') + ]... + +The paths resolve in practice as you would expect. + + >>> print call_py(interpreter_path, + ... "import sys, pprint; pprint.pprint(sys.path)") + ... # doctest: +ELLIPSIS + ['', + '/interpreter/parts/interpreter', + ..., + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] + <BLANKLINE> + +The ``extra_paths`` argument affects the path in site.py. Notice that +/interpreter/other is added after the eggs. + + >>> reset_interpreter() + >>> mkdir(interpreter_dir, 'other') + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... interpreter='py', extra_paths=[join(interpreter_dir, 'other')]) + >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS + #... + def addsitepackages(known_paths): + """Add site packages, as determined by zc.buildout. + <BLANKLINE> + See original_addsitepackages, below, for the original version.""" + buildout_paths = [ + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', + '/interpreter/other' + ]... + + >>> print call_py(interpreter_path, + ... "import sys, pprint; pprint.pprint(sys.path)") + ... # doctest: +ELLIPSIS + ['', + '/interpreter/parts/interpreter', + ..., + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', + '/interpreter/other'] + <BLANKLINE> + +The ``sitepackage_safe_scripts`` function: using site-packages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``sitepackage_safe_scripts`` function supports including site +packages. This has some advantages and some serious dangers. + +A typical reason to include site-packages is that it is easier to +install one or more dependencies in your Python than it is with +buildout. Some packages, such as lxml or Python PostgreSQL integration, +have dependencies that can be much easier to build and/or install using +other mechanisms, such as your operating system's package manager. By +installing some core packages into your Python's site-packages, this can +significantly simplify some application installations. + +However, doing this has a significant danger. One of the primary goals +of buildout is to provide repeatability. Some packages (one of the +better known Python openid packages, for instance) change their behavior +depending on what packages are available. If Python curl bindings are +available, these may be preferred by the library. If a certain XML +package is installed, it may be preferred by the library. These hidden +choices may cause small or large behavior differences. The fact that +they can be rarely encountered can actually make it worse: you forget +that this might be a problem, and debugging the differences can be +difficult. If you allow site-packages to be included in your buildout, +and the Python you use is not managed precisely by your application (for +instance, it is a system Python), you open yourself up to these +possibilities. Don't be unaware of the dangers. + +That explained, let's see how it works. If you don't use namespace packages, +this is very straightforward. + + >>> reset_interpreter() + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... interpreter='py', include_site_packages=True) + >>> sys.stdout.write('#\n'); cat(site_path) + ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + #... + def addsitepackages(known_paths): + """Add site packages, as determined by zc.buildout. + <BLANKLINE> + See original_addsitepackages, below, for the original version.""" + setuptools_path = None + buildout_paths = [ + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' + ] + for path in buildout_paths: + sitedir, sitedircase = makepath(path) + if not sitedircase in known_paths and os.path.exists(sitedir): + sys.path.append(sitedir) + known_paths.add(sitedircase) + sys.__egginsert = len(buildout_paths) # Support distribute. + original_paths = [ + ... + ] + for path in original_paths: + if path == setuptools_path or path not in known_paths: + addsitedir(path, known_paths) + return known_paths + <BLANKLINE> + def original_addsitepackages(known_paths):... + +It simply adds the original paths using addsitedir after the code to add the +buildout paths. + +Here's an example of the new script in use. Other documents and tests in +this package give the feature a more thorough workout, but this should +give you an idea of the feature. + + >>> res = call_py(interpreter_path, "import sys; print sys.path") + >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + ['', + '/interpreter/parts/interpreter', + ..., + '/interpreter/eggs/demo-0.3-py2.4.egg', + '/interpreter/eggs/demoneeded-1.1-py2.4.egg', + ...] + <BLANKLINE> + +The clean_paths gathered earlier is a subset of this full list of paths. + + >>> full_paths = eval(res.strip()) + >>> len(clean_paths) < len(full_paths) + True + >>> set(os.path.normpath(p) for p in clean_paths).issubset( + ... os.path.normpath(p) for p in full_paths) + True + +Unfortunately, because of how setuptools namespace packages are implemented +differently for operating system packages (debs or rpms) as opposed to +standard setuptools installation, there's a slightly trickier dance if you +use them. To show this we'll needs some extra eggs that use namespaces. +We'll use the ``tellmy.fortune`` package, which we'll need to make an initial +call to another text fixture to create. + + >>> from zc.buildout.tests import create_sample_namespace_eggs + >>> namespace_eggs = tmpdir('namespace_eggs') + >>> create_sample_namespace_eggs(namespace_eggs) + + >>> reset_interpreter() + >>> ws = zc.buildout.easy_install.install( + ... ['demo', 'tellmy.fortune'], join(interpreter_dir, 'eggs'), + ... links=[link_server, namespace_eggs], index=link_server+'index/') + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... interpreter='py', include_site_packages=True) + >>> sys.stdout.write('#\n'); cat(site_path) + ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + #... + def addsitepackages(known_paths): + """Add site packages, as determined by zc.buildout. + <BLANKLINE> + See original_addsitepackages, below, for the original version.""" + setuptools_path = '...setuptools...' + sys.path.append(setuptools_path) + known_paths.add(os.path.normcase(setuptools_path)) + import pkg_resources + buildout_paths = [ + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', + '...setuptools...', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' + ] + for path in buildout_paths: + sitedir, sitedircase = makepath(path) + if not sitedircase in known_paths and os.path.exists(sitedir): + sys.path.append(sitedir) + known_paths.add(sitedircase) + pkg_resources.working_set.add_entry(sitedir) + sys.__egginsert = len(buildout_paths) # Support distribute. + original_paths = [ + ... + ] + for path in original_paths: + if path == setuptools_path or path not in known_paths: + addsitedir(path, known_paths) + return known_paths + <BLANKLINE> + def original_addsitepackages(known_paths):... + + >>> print call_py(interpreter_path, "import sys; print sys.path") + ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + ['', + '/interpreter/parts/interpreter', + ..., + '...setuptools...', + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', + ...] + +As you can see, the script now first imports pkg_resources. Then we +need to process egg files specially to look for namespace packages there +*before* we process process lines in .pth files that use the "import" +feature--lines that might be part of the setuptools namespace package +implementation for system packages, as mentioned above, and that must +come after processing egg namespaces. + +The most complex that this function gets is if you use namespace packages, +include site-packages, and use relative paths. For completeness, we'll look +at that result. + + >>> reset_interpreter() + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... interpreter='py', include_site_packages=True, + ... relative_paths=interpreter_dir) + >>> sys.stdout.write('#\n'); cat(site_path) + ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + #... + def addsitepackages(known_paths): + """Add site packages, as determined by zc.buildout. + <BLANKLINE> + See original_addsitepackages, below, for the original version.""" + join = os.path.join + base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) + base = os.path.dirname(base) + base = os.path.dirname(base) + setuptools_path = '...setuptools...' + sys.path.append(setuptools_path) + known_paths.add(os.path.normcase(setuptools_path)) + import pkg_resources + buildout_paths = [ + join(base, 'eggs/demo-0.3-pyN.N.egg'), + join(base, 'eggs/tellmy.fortune-1.0-pyN.N.egg'), + '...setuptools...', + join(base, 'eggs/demoneeded-1.1-pyN.N.egg') + ] + for path in buildout_paths: + sitedir, sitedircase = makepath(path) + if not sitedircase in known_paths and os.path.exists(sitedir): + sys.path.append(sitedir) + known_paths.add(sitedircase) + pkg_resources.working_set.add_entry(sitedir) + sys.__egginsert = len(buildout_paths) # Support distribute. + original_paths = [ + ... + ] + for path in original_paths: + if path == setuptools_path or path not in known_paths: + addsitedir(path, known_paths) + return known_paths + <BLANKLINE> + def original_addsitepackages(known_paths):... + + >>> print call_py(interpreter_path, "import sys; print sys.path") + ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + ['', + '/interpreter/parts/interpreter', + ..., + '...setuptools...', + '/interpreter/eggs/demo-0.3-pyN.N.egg', + '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', + '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', + ...] + +The ``exec_sitecustomize`` argument does the same thing for the +sitecustomize module--it allows you to include the code from the +sitecustomize module in the underlying Python if you set the argument to +True. The z3c.recipe.scripts package sets up the full environment necessary +to demonstrate this piece. + +The ``sitepackage_safe_scripts`` function: writing scripts for entry points +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +All of the examples so far for this function have been creating +interpreters. The function can also write scripts for entry +points. They are almost identical to the scripts that we saw for the +``scripts`` function except that they ``import site`` after setting the +sys.path to include our custom site.py and sitecustomize.py files. These +files then initialize the Python environment as we have already seen. Let's +see a simple example. + + >>> reset_interpreter() + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], + ... index=link_server+'index/') + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... reqs=['demo']) + +As before, in Windows, 2 files are generated for each script. A script +file, ending in '-script.py', and an exe file that allows the script +to be invoked directly without having to specify the Python +interpreter and without having to provide a '.py' suffix. This is in addition +to the site.py and sitecustomize.py files that are generated as with our +interpreter examples above. + + >>> if sys.platform == 'win32': + ... demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py') + ... expected = [sitecustomize_path, + ... site_path, + ... os.path.join(interpreter_bin_dir, 'demo.exe'), + ... demo_path] + ... else: + ... demo_path = os.path.join(interpreter_bin_dir, 'demo') + ... expected = [sitecustomize_path, site_path, demo_path] + ... + >>> assert generated == expected, repr((generated, expected)) + +The demo script runs the entry point defined in the demo egg: + + >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 -S + <BLANKLINE> + import sys + sys.path[0:0] = [ + '/interpreter/parts/interpreter', + ] + <BLANKLINE> + <BLANKLINE> + import os + path = sys.path[0] + if os.environ.get('PYTHONPATH'): + path = os.pathsep.join([path, os.environ['PYTHONPATH']]) + os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') + os.environ['PYTHONPATH'] = path + import site # imports custom buildout-generated site.py + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main() + + >>> demo_call = join(interpreter_bin_dir, 'demo') + >>> if sys.platform == 'win32': + ... demo_call = '"%s"' % demo_call + >>> print system(demo_call) + 3 1 + <BLANKLINE> + +There are a few differences from the ``scripts`` function. First, the +``reqs`` argument (an iterable of string requirements or entry point +tuples) is a keyword argument here. We see that in the example above. +Second, the ``arguments`` argument is now named ``script_arguments`` to +try and clarify that it does not affect interpreters. While the +``initialization`` argument continues to affect both the interpreters +and the entry point scripts, if you have initialization that is only +pertinent to the entry point scripts, you can use the +``script_initialization`` argument. + +Let's see ``script_arguments`` and ``script_initialization`` in action. + + >>> reset_interpreter() + >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( + ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, + ... reqs=['demo'], script_arguments='1, 2', + ... script_initialization='import os\nos.chdir("foo")') + + >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE + #!/usr/local/bin/python2.4 -S + import sys + sys.path[0:0] = [ + '/interpreter/parts/interpreter', + ] + <BLANKLINE> + import os + path = sys.path[0] + if os.environ.get('PYTHONPATH'): + path = os.pathsep.join([path, os.environ['PYTHONPATH']]) + os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') + os.environ['PYTHONPATH'] = path + import site # imports custom buildout-generated site.py + import os + os.chdir("foo") + <BLANKLINE> + import eggrecipedemo + <BLANKLINE> + if __name__ == '__main__': + eggrecipedemo.main(1, 2) + +Handling custom build options for extensions provided in source distributions +----------------------------------------------------------------------------- + +Sometimes, we need to control how extension modules are built. The +build function provides this level of control. It takes a single +package specification, downloads a source distribution, and builds it +with specified custom build options. + +The build function takes 3 positional arguments: + +spec + A package specification for a source distribution + +dest + A destination directory + +build_ext + A dictionary of options to be passed to the distutils build_ext + command when building extensions. + +It supports a number of optional keyword arguments: + +links + a sequence of URLs, file names, or directories to look for + links to distributions, + +index + The URL of an index server, or almost any other valid URL. :) + + If not specified, the Python Package Index, + http://pypi.python.org/simple/, is used. You can specify an + alternate index with this option. If you use the links option and + if the links point to the needed distributions, then the index can + be anything and will be largely ignored. In the examples, here, + we'll just point to an empty directory on our link server. This + will make our examples run a little bit faster. + +executable + A path to a Python executable. Distributions will be installed + using this executable and will be for the matching Python version. + +path + A list of additional directories to search for locally-installed + distributions. + +newest + A boolean value indicating whether to search for new distributions + when already-installed distributions meet the requirement. When + this is true, the default, and when the destination directory is + not None, then the install function will search for the newest + distributions that satisfy the requirements. + +versions + A dictionary mapping project names to version numbers to be used + when selecting distributions. This can be used to specify a set of + distribution versions independent of other requirements. + + +Our link server included a source distribution that includes a simple +extension, extdemo.c:: + + #include <Python.h> + #include <extdemo.h> + + static PyMethodDef methods[] = {}; + + PyMODINIT_FUNC + initextdemo(void) + { + PyObject *m; + m = Py_InitModule3("extdemo", methods, ""); + #ifdef TWO + PyModule_AddObject(m, "val", PyInt_FromLong(2)); + #else + PyModule_AddObject(m, "val", PyInt_FromLong(EXTDEMO)); + #endif + } + +The extension depends on a system-dependent include file, extdemo.h, +that defines a constant, EXTDEMO, that is exposed by the extension. + +We'll add an include directory to our sample buildout and add the +needed include file to it: + + >>> mkdir('include') + >>> write('include', 'extdemo.h', + ... """ + ... #define EXTDEMO 42 + ... """) + +Now, we can use the build function to create an egg from the source +distribution: + + >>> zc.buildout.easy_install.build( + ... 'extdemo', dest, + ... {'include-dirs': os.path.join(sample_buildout, 'include')}, + ... links=[link_server], index=link_server+'index/') + ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] + +The function returns the list of eggs + +Now if we look in our destination directory, we see we have an extdemo egg: + + >>> ls(dest) + - demo-0.2-py2.4.egg + d demo-0.3-py2.4.egg + - demoneeded-1.0-py2.4.egg + d demoneeded-1.1-py2.4.egg + d extdemo-1.4-py2.4-unix-i686.egg + +Let's update our link server with a new version of extdemo: + + >>> update_extdemo() + >>> print get(link_server), + <html><body> + <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> + <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> + <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> + <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> + <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> + <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> + <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> + <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> + <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> + <a href="extdemo-1.5.zip">extdemo-1.5.zip</a><br> + <a href="index/">index/</a><br> + <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> + </body></html> + +The easy_install caches information about servers to reduce network +access. To see the update, we have to call the clear_index_cache +function to clear the index cache: + + >>> zc.buildout.easy_install.clear_index_cache() + +If we run build with newest set to False, we won't get an update: + + >>> zc.buildout.easy_install.build( + ... 'extdemo', dest, + ... {'include-dirs': os.path.join(sample_buildout, 'include')}, + ... links=[link_server], index=link_server+'index/', + ... newest=False) + ['/sample-install/extdemo-1.4-py2.4-linux-i686.egg'] + + >>> ls(dest) + - demo-0.2-py2.4.egg + d demo-0.3-py2.4.egg + - demoneeded-1.0-py2.4.egg + d demoneeded-1.1-py2.4.egg + d extdemo-1.4-py2.4-unix-i686.egg + +But if we run it with the default True setting for newest, then we'll +get an updated egg: + + >>> zc.buildout.easy_install.build( + ... 'extdemo', dest, + ... {'include-dirs': os.path.join(sample_buildout, 'include')}, + ... links=[link_server], index=link_server+'index/') + ['/sample-install/extdemo-1.5-py2.4-unix-i686.egg'] + + >>> ls(dest) + - demo-0.2-py2.4.egg + d demo-0.3-py2.4.egg + - demoneeded-1.0-py2.4.egg + d demoneeded-1.1-py2.4.egg + d extdemo-1.4-py2.4-unix-i686.egg + d extdemo-1.5-py2.4-unix-i686.egg + +The versions option also influences the versions used. For example, +if we specify a version for extdemo, then that will be used, even +though it isn't the newest. Let's clean out the destination directory +first: + + >>> import os + >>> for name in os.listdir(dest): + ... remove(dest, name) + + >>> zc.buildout.easy_install.build( + ... 'extdemo', dest, + ... {'include-dirs': os.path.join(sample_buildout, 'include')}, + ... links=[link_server], index=link_server+'index/', + ... versions=dict(extdemo='1.4')) + ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] + + >>> ls(dest) + d extdemo-1.4-py2.4-unix-i686.egg + +Handling custom build options for extensions in develop eggs +------------------------------------------------------------ + +The develop function is similar to the build function, except that, +rather than building an egg from a source directory containing a +setup.py script. + +The develop function takes 2 positional arguments: + +setup + The path to a setup script, typically named "setup.py", or a + directory containing a setup.py script. + +dest + The directory to install the egg link to + +It supports some optional keyword argument: + +build_ext + A dictionary of options to be passed to the distutils build_ext + command when building extensions. + +executable + A path to a Python executable. Distributions will be installed + using this executable and will be for the matching Python version. + +We have a local directory containing the extdemo source: + + >>> ls(extdemo) + - MANIFEST + - MANIFEST.in + - README + - extdemo.c + - setup.py + +Now, we can use the develop function to create a develop egg from the source +distribution: + + >>> zc.buildout.easy_install.develop( + ... extdemo, dest, + ... {'include-dirs': os.path.join(sample_buildout, 'include')}) + '/sample-install/extdemo.egg-link' + +The name of the egg link created is returned. + +Now if we look in our destination directory, we see we have an extdemo +egg link: + + >>> ls(dest) + d extdemo-1.4-py2.4-unix-i686.egg + - extdemo.egg-link + +And that the source directory contains the compiled extension: + + >>> ls(extdemo) + - MANIFEST + - MANIFEST.in + - README + d build + - extdemo.c + d extdemo.egg-info + - extdemo.so + - setup.py + +Download cache +-------------- + +Normally, when distributions are installed, if any processing is +needed, they are downloaded from the internet to a temporary directory +and then installed from there. A download cache can be used to avoid +the download step. This can be useful to reduce network access and to +create source distributions of an entire buildout. + +A download cache is specified by calling the download_cache +function. The function always returns the previous setting. If no +argument is passed, then the setting is unchanged. If an argument is +passed, the download cache is set to the given path, which must point +to an existing directory. Passing None clears the cache setting. + +To see this work, we'll create a directory and set it as the cache +directory: + + >>> cache = tmpdir('cache') + >>> zc.buildout.easy_install.download_cache(cache) + +We'll recreate our destination directory: + + >>> remove(dest) + >>> dest = tmpdir('sample-install') + +We'd like to see what is being fetched from the server, so we'll +enable server logging: + + >>> get(link_server+'enable_server_logging') + GET 200 /enable_server_logging + '' + +Now, if we install demo, and extdemo: + + >>> ws = zc.buildout.easy_install.install( + ... ['demo==0.2'], dest, + ... links=[link_server], index=link_server+'index/', + ... always_unzip=True) + GET 200 / + GET 404 /index/demo/ + GET 200 /index/ + GET 200 /demo-0.2-py2.4.egg + GET 404 /index/demoneeded/ + GET 200 /demoneeded-1.1.zip + + >>> zc.buildout.easy_install.build( + ... 'extdemo', dest, + ... {'include-dirs': os.path.join(sample_buildout, 'include')}, + ... links=[link_server], index=link_server+'index/') + GET 404 /index/extdemo/ + GET 200 /extdemo-1.5.zip + ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] + +Not only will we get eggs in our destination directory: + + >>> ls(dest) + d demo-0.2-py2.4.egg + d demoneeded-1.1-py2.4.egg + d extdemo-1.5-py2.4-linux-i686.egg + +But we'll get distributions in the cache directory: + + >>> ls(cache) + - demo-0.2-py2.4.egg + - demoneeded-1.1.zip + - extdemo-1.5.zip + +The cache directory contains uninstalled distributions, such as zipped +eggs or source distributions. + +Let's recreate our destination directory and clear the index cache: + + >>> remove(dest) + >>> dest = tmpdir('sample-install') + >>> zc.buildout.easy_install.clear_index_cache() + +Now when we install the distributions: + + >>> ws = zc.buildout.easy_install.install( + ... ['demo==0.2'], dest, + ... links=[link_server], index=link_server+'index/', + ... always_unzip=True) + GET 200 / + GET 404 /index/demo/ + GET 200 /index/ + GET 404 /index/demoneeded/ + + >>> zc.buildout.easy_install.build( + ... 'extdemo', dest, + ... {'include-dirs': os.path.join(sample_buildout, 'include')}, + ... links=[link_server], index=link_server+'index/') + GET 404 /index/extdemo/ + ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] + + >>> ls(dest) + d demo-0.2-py2.4.egg + d demoneeded-1.1-py2.4.egg + d extdemo-1.5-py2.4-linux-i686.egg + +Note that we didn't download the distributions from the link server. + +If we remove the restriction on demo, we'll download a newer version +from the link server: + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, + ... links=[link_server], index=link_server+'index/', + ... always_unzip=True) + GET 200 /demo-0.3-py2.4.egg + +Normally, the download cache is the preferred source of downloads, but +not the only one. + +Installing solely from a download cache +--------------------------------------- + +A download cache can be used as the basis of application source +releases. In an application source release, we want to distribute an +application that can be built without making any network accesses. In +this case, we distribute a download cache and tell the easy_install +module to install from the download cache only, without making network +accesses. The install_from_cache function can be used to signal that +packages should be installed only from the download cache. The +function always returns the previous setting. Calling it with no +arguments returns the current setting without changing it: + + >>> zc.buildout.easy_install.install_from_cache() + False + +Calling it with a boolean value changes the setting and returns the +previous setting: + + >>> zc.buildout.easy_install.install_from_cache(True) + False + +Let's remove demo-0.3-py2.4.egg from the cache, clear the index cache, +recreate the destination directory, and reinstall demo: + + >>> for f in os.listdir(cache): + ... if f.startswith('demo-0.3-'): + ... remove(cache, f) + + >>> zc.buildout.easy_install.clear_index_cache() + >>> remove(dest) + >>> dest = tmpdir('sample-install') + + >>> ws = zc.buildout.easy_install.install( + ... ['demo'], dest, + ... links=[link_server], index=link_server+'index/', + ... always_unzip=True) + + >>> ls(dest) + d demo-0.2-py2.4.egg + d demoneeded-1.1-py2.4.egg + +This time, we didn't download from or even query the link server. + +.. Disable the download cache: + + >>> zc.buildout.easy_install.download_cache(None) + '/cache' + + >>> zc.buildout.easy_install.install_from_cache(False) + True |