Skip to main content
Warning: You are using the test version of PyPI. This is a pre-production deployment of Warehouse. Changes made here affect the production instance of TestPyPI (
Help us improve Python packaging - Donate today!

Tool for black-box testing command-line programs using STDIN, STDOUT and STDERR

Project Description

This is a tool for black-box testing command-line programs simply based on STDIN, STDOUT, and STDERR.


Test cases usually consist of pairs of input and output files, as well as an optional tests.json file specifying applicable command-line arguments. The input file is piped in via STDIN. If the command being tested emits the expected output file via STDOUT, the test case passes. A file representing an expected STDERR output can also optionally be specified.

The input/output file pairs must follow this naming pattern:  ->  test-A.out.txt  ->  test-B.out.txt, test-B.err.txt  ->  test-C.out.txt

The file extension (.txt here) can be anything. The file naming pattern is *.in* for input files, *.out* for expected output files, and *.err* for expected error files. The content of the *.in.* file is piped to the command being tested, and its STDOUT is compared against the *.out* file. If a *.err.* file has been provided, then the command STDERR is matched against it as well.

For an example of test_cmd in action, see the pypage project, particularly its tests folder.


usage: [-h] tests_dir cmd ...

Functional Testing Utility for Command-Line Applications

positional arguments:
  tests_dir   Path to the directory containing test cases
  cmd         Path to the command to be tested
  args        The command-line arguments with an ampersand character '@'
              markingwhere arguments from test.json should be injected

optional arguments:
  -h, --help  show this help message and exit

Command-line arguments for test cases can be specified by creating a special file named tests.json, and placing it in the directory containing your test cases. This tests.json file maps test cases to objects representing command-line arguments for that test case. If a command-line argument is a non-string value (e.g. a complex JavaScript object), the argument is stringified (with Python’s json.dumps), and passed in as JSON.

Release History

This version
History Node


Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Hash SHA256 Hash Help Version File Type Upload Date
(4.8 kB) Copy SHA256 Hash SHA256
Source Nov 3, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting