Atomicity of API Endpoint Actions

One critical aspect of maintaining data integrity is making API endpoint actions atomic. Atomicity refers to the property of an action that guarantees that the series of operations are completed successfully or none at all.

There are different challenges and solutions to the atomicity problem.

Action needs to write objects locally

This case is solved using traditional database transactions:

  DB.transaction do
    account = DB.find_and_lock!(params[:account_id])
    account.reservations.remove!
    account.settlements.create!
    account.update_state!(PROCESSED)
  end

The solution above ensures either all writes are saved or rolled back. It uses a row-level lock on a particular account preventing race-conditions, then updates dependent objects: reservations and settlements, then the account state and commits the transaction.

Note:
- select a proper transaction isolation level
- if a row lock is not enough, consider advisory(application-level) lock

Action needs to write to different systems

An example would be to call a set of HTTP endpoints, write to the DB and notify the MQ. Such situations have no standartized solution, and most of the times to make such action atomic would require the endpoint to be eventually consistent.

One approach is to introduce states on the object level and have a state machine. Say the endpoint creates an invoice, processes a payment, orders an item and sends an email, then the Invoice object can have the following states:

CREATED   # invoice is created
PAID      # invoice is paid
ORDERED   # item is ordered
COMPLETED # email confirmation is sent

The v1/invoices endpoint will only return a CREATED invoice and the processing will happen in the background. Ideally the action should also know how to rollback on errors where possible. For example, if the invoice is PAID, but ordering is not possible, since the last item is gone, the payment must be automatically REFUNDED(a new state). After the item is ORDERED, rollback is not possible, so sending an email must be retried until successful.

Another approach is to use a transactional outbox pattern. This approach involves creation of the separate outbox table. A request to v1/invoices in this case will open a database transaction that will:

1. Create a new invoice(invoices table)
2. Create a payment_action bound to the invoice(outbox table)
3. Create an order_action bound to the invoice(outbox table)
4. Create an send_email_action bound to the invoice(outbox table)

Again, a CREATED invoice will be returned to the API client, then actions from the outbox table will be picked up and done by corresponding services asynchronously.

Further read

Read on how to make actions idempotent.