There may be times when the service behind the URL provided by the user for Cloud Elements to push events to, is down or undergoing maintenance. In this scenario, Cloud Elements has a mechanism to retry the event push and/or queue the event to be retrieved at a later time, say after the user's service endpoint is back up.
In such a scenario, the user can choose between two mechanisms to receive events after the user's event reception service is back up
- Retry the push of the events
- Queue the events for the user to pull/retrieve at a later time
Each account with Cloud Elements can be configured to either retry event notification upon failure or to queue the event for later retrieval. After creation of the account, the mechanism can be configured by using the POST /accounts/{accountId}/settings API. The payload posted to this API is -
{
"notification.webhook.failure.policy": "retry"
}The event retry/queueing configuration for a given account can be modified via the PATCH /accounts/{accountId}/settings API. Say that the account was configured use an event retry mechanism and needs to be modified to use event queueing. The payload for this API call is -
{
"notification.webhook.failure.policy": "queue"
}The default, if neither of the above APIs is invoked after the account is created is to retry the events notification.
Retry Logic
If an account is configured to retry event notifications, the web hook URL provided by the user (e.g., https://events.myservice.com/receiver) is retried up to 5 times. The interval uses the following equation: 2^3x, where x is the retry attempt.
Based on the above, the first attempt is retried after 8 seconds, the second attempt after 64 seconds and so on. The 5th and final attempt will be made a little under 9.5 hours after the original failed attempt.
As soon as a retry attempt is successful, the notification is deemed completed and all is good to go. In the case where all 9 attempts fail, then the event subscription is deactivated, and future events are queued for retrieval by the user until the subscription is reactivated.
Note: Even though the events are queued after 9 retry attempts, the account configuration is NOT changed to queue events.
If deactivated, the subscription can be reactivated and the queued events delivered via the PUT /notifications/subscriptions/deliveries?where=channel='webhook' and status='queued' API. As documented in the following section, an addition where clause parameter can be added to redeliver events only specific to a given subscription URL.
Queueing Logic
If the account is configured to queue events, or if the maximum number of retries (9) for a given push subscription is reached, then the event is queued for retrieval by the user.
If the user's web hook receiver url is https://events.myservice.com/receiver, then queued events for this endpoint can be retrieved by calling the GET /notifications/subscriptions/deliveries?where=channel='webhook' and status='queued' and config.url= 'https://events.myservice.com/receiver' API.
The above API requires the user to provide the Organization and User secrets, like most other Platform APIs, and will retrieved the queued events for a given URL, for the given account. Each returned delivery object consists among other attributes, the event payload, including the event ID and the event data.
