Welcome to arend’ documentation!

A simple producer-consumer library for the Beanstalkd queue!

Installation

Install it using pip:

pip install arend

Basic usage

Make sure you have the arend installed:

from arend import arend_task
from arend.backends.mongo import MongoSettings
from arend.brokers import BeanstalkdSettings
from arend.settings import ArendSettings
from arend.worker import consumer

settings = ArendSettings(
    beanstalkd=BeanstalkdSettings(host="beanstalkd", port=11300),
    backend=MongoSettings(
        mongo_connection="mongodb://user:pass@mongo:27017",
        mongo_db="db",
        mongo_collection="Tasks"
    ),
)

@arend_task(queue="my_queue", settings=settings)
def double(num: int):
    return 2 * num

double(2)  # returns 4
task = double.apply_async(args=(4,))  # It is sent to the queue

consumer(queue="my_queue", settings=settings)  # consume tasks from the queue

Task = settings.get_backend()  # you can check your backend for the result
task = Task.get(uuid=task.uuid)
assert task.result == 4

Backends

The available backends to store logs are Mongo and Redis.

Setting your backend with environment variables

You can set your backend by defining env vars. The AREND__ prefix indicates that it belongs to the Arend:

# Redis
AREND__REDIS_HOST='redis'
AREND__REDIS_DB='1'
AREND__REDIS_PASSWORD='pass'
...

# Mongo
AREND__MONGO_CONNECTION='mongodb://user:pass@mongo:27017'
AREND__MONGO_DB='db'
AREND__MONGO_COLLECTION='logs'
...

Now in your code:

from arend import arend_task
from arend.worker import consumer


@arend_task(queue="my_queue")
def double(num: int):
    return 2 * num

double.apply_async(args=(4,))  # It is sent to the queue

consumer(queue="my_queue")

Indices and tables