Skip to main content
Warning: You are using the test version of PyPI. This is a pre-production deployment of Warehouse. Changes made here affect the production instance of TestPyPI (testpypi.python.org).
Help us improve Python packaging - Donate today!

Build utility to manage web resources

Project Description

Summary

PyBuildTool helps transform resources and run automated tests as you code, I usually have it watched the project files while editing with vim.

PyBuildTool is written in python as a higher level interface to Waf, a meta build system.

The configuration file you’d interact with: build.yml, is declarative, written in yaml. While the build tools which you could also write, were made to be simple, following a pattern.

The configuration file does not need to be named “build.yml”, just change the file wscript (this is something that waf needs). You need to create wscript file yourself, following the example here: http://raw.githubusercontent.com/dozymoe/PyBuildTool/master/pybuildtool/wscript.example

There are builtin tools, you can use them as examples to build your own build tools, see them here: http://github.com/dozymoe/PyBuildTool/tree/master/pybuildtool/tools. See how they were imported by the wscript file.

Since you control the wscript file, you control much of the aspect of the build process.

If you wanted to go deeper than just editing build.yml you could read introduction to waf and wscript, or a more technical waf project structure.

There’re three stages predefined: “dev”, “stage”, and “prod” (see the wscript file), but the functionality isn’t there yet, they have separate temporary build directory of their own but they still share the same build rules.

Currently stages were implemented like this: for clean-css tool, if the current stage was one of “dev” or “devel” or “development” it will do copy operation instead of css file minification.

This is an example of build.yml, pretend it’s a django project:

# Macro, reusable definition, defined once - used plenty.
# The word started with ampersand (&) is called yaml node anchor.
# The anchor is what matters, it's a yaml feature.
#
# Group name in all uppercase will be ignored.
#
# {_1} is a replacement pattern for parent group's name at level 1.
# It is being used to easily reproduce the directory structure.
JSHINT_EXCLUDES: &JSHINT_EXCLUDES
    - "{_1}/{_2}/js/jquery.js"
    - "{_1}/{_2}/js/require.js"
    - "{_1}/{_2}/js/underscore.js"


# Build group at level 1.
djangoprj:
    # This is special, not a group.
    options:
        # Lower level tool configurations can be defined at higher level.
        #
        # `config_file` of the tool `jshint`
        jshint_config_file: "etc/jshint.rc"
        # `config_file` of the tool `pylint`
        pylint_config_file: "etc/pylint.rc"

    # Build group at level 2
    blogapp:
        # The lowest level group is the tool being used, if no such tool
        # was found a fatal error will be raised.
        jshint:
            # Wildcards is a okay, see ant-glob.
            file_in: "{_1}/{_2}/js/**/*.js"
            # Tokens are virtual keys (not real files), can be used
            # as dependency of another tool
            token_out: "{_1}:{_2}:{_3}"
        concat:
            # `raw_file_in` will be monitored for changes by `waf watch`
            raw_file_in: "{_1}/{_2}/js/**/*.js"
            # `{_1}:{_2}` will be replaced with the groups' names, in this
            # case it will be read as `djangoprj:blogapp:jshint`
            token_in: "{_1}:{_2}:jshint"
            # Relative files are relative to the directory where you
            # run `waf configure`
            file_out: "js/blogapp.js"

    # Build group at level 2
    djangoprj:
        # Test javascript files syntax for errors
        jshint:
            options:
                # Example of macro usage, see yaml node anchor.
                # `_source_excluded_` is a special directive, excludes some
                # files listed in `file_in` (could be from wildcards) from
                # being processed.
                _source_excluded_: *JSHINT_EXCLUDES
            file_in: "{_1}/{_2}/js/**/*.js"
            token_out: "{_1}:{_2}:{_3}"
        # Test javascript files syntax for errors
        jscs:
            options:
                _source_excluded_: *JSHINT_EXCLUDES
                # This one is defined here, not in higher level group
                config_file: "etc/jscs.rc"
            file_in: "{_1}/{_2}/js/**/*.js"
            token_out: "{_1}:{_2}:{_3}"
        # Test python files syntax for errors
        pylint:
            raw_file_in: "{_1}/{_2}/**/*.py"
            token_out: "{_1}:{_2}:{_3}"
        # Concacenate javascript files into one file for production site
        concat:
            token_in:
                # Can has multiple items
                -   "@djangoprj:djangoprj:jshint"
                -   "{_1}:{_2}:jscs"
            # The `@` symbol means to use the files produced by other tools
            file_in: "@{_1}:blogapp:concat"
            raw_file_in: "{_1}/{_2}/js/**/*.js"
            file_out: "js/djangoprj.js"
        # Copy final javascript file to production directory
        cp:
            file_in: "@{_1}:{_2}:concat"
            # Files usually produced in sandbox directories, `raw_file_out`
            # directive made it produced in the real project directory
            raw_file_out: "dist/"
        # Copy compressed final javascript file to production
        # directory
        uglifyjs:
            file_in: "@{_1}:{_2}:concat"
            # Directory as target is a okay, directory must ends with `/`.
            raw_file_out: "dist/"

Several things to keep in mind:

  • “djangoprj”, “blogapp”, “jshint”, “concat” are group names.
  • JSHINT_EXCLUDES is not a group name (it matches all capital letters and underscore), pybuildtool will not recognize this entry, but yaml.load will. It can be used as yaml node anchor.
  • {_1} is string replacement thingy for a group name based on its level.
  • Because group name can be used as string replacement for file names, they can be used to represent directory structures.
  • “@djangoprj:djangoprj:jshint” is a reference to files or token generated by the rule “djangoprj”-“djangoprj”-“jshint”, that is, the combination of its file_out, raw_file_out and extra_out.
  • You can use ant glob like this **/*.js
  • You can use directory as output, they must end with path separator, for example: “minified_js/”
  • The child-most groups are special, they must match tool name like “jshint”, “concat”, “pylint”, “uglifyjs”, etc.
  • Rules are read in the order they are written, you can reference other rules generated output files as a input files but those rules must have been specified before. We don’t support lazy loading of rules yet.
  • The directive raw_file_in or raw_depend_in is used for waf watch to get list of files need to be monitored.
  • The directive depend_in can be used to force the tool to process file_in if files in depend_in changes.
  • The directive extra_out can be used to list auxiliary files produced by the tool, it can be used with combination of @group:group directive as inputs for other tools.
  • The option field: _source_excluded_ is list of files which will be excluded from inputs.
  • The directive raw_file_out means this rule’s outputs will be written in the actual file system, by default it’s generated inside ‘.BUILD/stage/’ directory.
  • The option field: config_file is configuration item provided by each tools, in this case it was provided by “pylint”, “jshint”, and “jscs”, and they happened to have used the same name. When option field is placed in higher group level, it’s prefixed with the tool name, for example: “jscs_config_file”

Warning

waf does not like it if the source and target existed in the same directory, see: Files are always built.

Install

  1. pip install pybuildtool

  2. Install waf as executable binary, download from Waf project.

    You could also pip install waftools and then run wafinstall, caution: this method will modify your ~/.bashrc adding WAFDIR=???.

  3. Copy and modify wscript in your project’s root directory, specify the build tools your are going to use.

  4. Create build.yml with content like our example, this will be your build rules.

  5. waf configure

  6. waf build_dev or waf watch_dev

Release History

Release History

This version
History Node

2.0.11

History Node

2.0.10

History Node

2.0.9

History Node

2.0.8

History Node

2.0.7

History Node

2.0.6

History Node

2.0.5

History Node

2.0.4

History Node

2.0.3

History Node

2.0.2

History Node

2.0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
pybuildtool-2.0.11.tar.gz (52.5 kB) Copy SHA256 Checksum SHA256 Source Jun 22, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting