Skip to main content
Warning: You are using the test version of PyPI. This is a pre-production deployment of Warehouse. Changes made here affect the production instance of TestPyPI (testpypi.python.org).
Help us improve Python packaging - Donate today!

IronPort Coroutine/Threading Library

Project Description

This Python library was evolved at IronPort Systems and has been provided as open source by Cisco Systems under an MIT license.

Intro

Shrapnel is a library for high-performance concurrency. It uses coroutines to implement user threads on top of either kqueue (FreeBSD, OS X) or /dev/epoll (linux), and is written mostly in Pyrex/Cython, supporting both 32-bit and 64-bit platforms. It is the culmination of about 8 years of work at IronPort Systems, a provider of high-speed mail appliances. It was open-sourced by Cisco Systems in late 2011.

Status

Apr 18, 2013: I’ve recently merged in a long chain of branches for several important features:

  • Support for pure-cython servers (branch ‘pxdfix’)
  • Full DNS resolver implementation (branch ‘dns-cache’)
  • Updated postgres support (branch ‘postgres’)
  • Included OpenSSL support

Features

  • Lightweight threads, event-driven scheduler.
  • Underneath: non-blocking operations on descriptors, like sockets and pipes.
  • On top, synchronous API for straight-line, simple code.
  • Highly scalable - tens or hundreds of thousands of connections/threads.
  • Thread synchronization primitives, like mutexes, semaphores, etc…
  • with_timeout(): wrap any funcall with a timeout.
  • Wait on kqueue events like file/directory changes, signals, processes, etc… [kqueue only]
  • DNS resolver and cache
  • HTTP server and client (plus WebSocket, RFC6455 & hixie-76)
  • Support for TLS via tlslite and openssl (plus NPN for both)
  • other protocols/codecs: ldap, asn1, ftp, mysql, postgres, AMQP.
  • MIT License.

Advantages

Compared to other concurrency packages available for Python, Shrapnel gives you:

  • Speed and Efficiency: the entire scheduler, poller, socket layer, synchronization objects, etc… are written in Cython, with an emphasis on performance and low memory usage.
  • Stock Python: Shrapnel works with out-of-the-box CPython [2.X]. No special variants of Python are needed, it will even work with your OS’s OEM python installation. So you can use all the external libraries/modules you’ve come to rely on.
  • No Callbacks: no need to cuisinart your application into a thousand callbacks. No need to decompose every action into a state machine. Write simple, performant code now without having to send your programmers to class.
  • Drop to Cython for speed: all the capabilities of the system are available from Cython, so you can e.g. write a server entirely in Cython for speed. You can interface with external libraries, and do thread switches from Cython or C. It’s even possible to have external C code call back into shrapnel. This makes it easy to prototype your application in Python, and then push only the hot spots into Cython.
  • Timeouts: Shrapnel provides a general timeout mechanism that can be used to wrap any function call with a timeout.
  • Profiler: Thread-aware profiler generates HTML reports.

API Documentation

See http://ironport.github.com/shrapnel/

Release History

Release History

This version
History Node

1.0.5.2

History Node

1.0.5.1

History Node

1.0.5

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
coro-1.0.5.2.tar.gz (301.1 kB) Copy SHA256 Checksum SHA256 Source May 29, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting