Skip to main content
Warning: You are using the test version of PyPI. This is a pre-production deployment of Warehouse. Changes made here affect the production instance of TestPyPI (
Help us improve Python packaging - Donate today!

Fully compatible DynamoDB result backend for Celery

Project Description


Dyna-Veggie is the first fully compatible DynamoDB result backend for Celery. It supports both supports both DynamoDB in AWS and DynamoDB Local. Dyna-Veggie also supports these features out of the box:

  • Fully suports Native join operation in Celery just like the Redis backend, which bring the best performance for canvas operations like chord and group.
  • Uses batch get for retrieving large amount of items which provides much better performance than sequential gets.
  • Automatically creates the table if not presented
  • Allows customizable options for DynamoDB throttling retry and exponential backoff
  • Handles DynamoDB’s consistency issue appropriately


You can install Dyna-Veggie either via the Python Package Index (PyPI) or from source.

To install using pip:

$ pip install -U dyna-veggie

Get Started

To use Dyna-Veggie in your celery project is simple. All you need to do is:

  1. Install dyna-veggie through pip
  2. Add the following to your celery config file AT THE BEGINNING, and Enjoy!:
# maybe?


BACKEND_ALIASES['dynamodb'] = 'dyna_veggie.DynamoDBBackend'
# access_key - your aws account's access key, encoded
# secret_key - your aws account's secret key, encoded
# region - AWS region your DynamoDB table's in
# table - Optional. DynamoDB table for storing results. Default to celery
# port - Optional unless you use DynamoDBLocal
# read_throughput - Optional. Default to 5
# write_throughput - Optional. Default to 5
# retry_if_throttled - Optional. Default to True
# consistency_wait_time - Optional. Seconds to wait for data consistency. Default to 1
result_backend = 'dynamodb://encoded_aws_access_key_id:encoded_aws_secret_access_key@region:port/table?read_throughput=n&write_throughput=m'


access_key & secret_key

The credentials for accessing AWS API resources. These can also be resolved by the boto3 library from various sources, as described here.


The AWS region, e.g. us-east-1 or local for DynamoDB Local. See the boto3 library documentation for definition options.


The listening port of the local DynamoDB instance, if you are using the DynamoDB Local. If you have not specified the region parameter as local, setting this parameter has no effect.


Table name to use. Default is celery. See the DynamoDB Naming Rules for information on the allowed characters and length.

read_throughput & write_throughput

The Read & Write Capacity Units for the created DynamoDB table. Default is 5 for both read and write. More details can be found in the Provisioned Throughput documentation.


Whether to retry is the throttling exception is raised. Exponential backoff is used in the retry operations. More parameters for the exponential backoff behavior will be allowed to be customized in future version. Default to True.


Seconds to wait for data to become consistent before the backend thinks the data is not available. Why do we need this? Because of network latency and read consistency. Why can’t we just set ConsistentRead to True? Because the get request has the potential to reach to DynamoDB faster than a set request reaches it, if the set and get requests are issued on different machines/processes. Thus the only way to ensure we can get item in a distributed setting is to retry. Default to 1, because typically 1 second is enough for the data to present.


Apache License, Version 2.0

Release History

This version
History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Hash SHA256 Hash Help Version File Type Upload Date
(5.6 kB) Copy SHA256 Hash SHA256
Source Aug 8, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting