- Queue up APIs to run on demand with guaranteed order.
- Configure APIs to queue up periodically.
- Sink the results of the APIs to a destination such as a webhook.
- Extend the API payloads to dynamically queue up other APIs.
- Impose rate limits for the queued up API runs.
- Configure periods which the queue can pause execution.
- Add random delays between API runs.
Properties
Each queue requires to be configured upon creation, which will control various aspects of the queue, such as the rate limits and the destination of the results. This section will be a brief overview of the properties that can be configured for a queue. For detailed information about the types and shapes of the properties, please refer to the create queue API reference.Configuration
The configuration of a queue configures the retry policy, rate limits, schedule and random delays between API runs.Retry policy
The retry policy configures the queue-level maximum number of attempts to run a payload. This value can be overridden by the payload.The retries in queues are guaranteed to be executed in order. Before moving on to the next payload, the queue will keep retrying the same payload until it succeeds or the maximum attempts are reached.
Rate limits
The rate limits configuration controls the maximum number of payloads that can be queued up in a given time period. It consists of an array of rate limits, each with alimit
and a duration
. The limit
is the maximum number of payloads that can be executed in the duration
time frame. The duration is an ms-formatted string.
The union of the rate limits of the queue and the payload will be used. For example, if you have a limit of 10 items per hour and 2 items per minute, 2 items will be executed in the first minute and 2 more in the following minute until the 10 per hour limit is reached (assuming there are sufficent queued-up payloads).
Schedule
The schedule is an optional property to configure the queue to pause and resume execution periodically. This is useful if you want the queue to stop executing at specific time periods, lets say outside of business hours. It consists of two cron expressions, one for the start of the pause period (pause
) and one for the end of the pause period (resume
).
Random wait
The random wait configuration allows you to add a random delay between API runs. This is useful if you want to emulate a user’s interaction with a website, where the time between actions is not constant. It consists of a range of either milliseconds or ms-formatted strings.Sink
The sink configuration controls where the results of the API runs will be sent. Currently, the only supported sink for queues is a webhook. The webhook will receive a POST request with the results of the API run. Check out the API reference for sinks for more information about the configuration and output format of the sinks.Queue items
Queue items are the payloads that are queued up to be executed. They can override the queue’s retry policy.While the queue is running, payloads can be dynamically appended to the queue from the APIs it is running. This allows for running payloads conditionally based on the output of other APIs.