AWS X-Ray Tracing In Ruby Lambda

Context

AWS X-Ray is a distributed tracing service offered by Amazon that helps developers analyze and debug applications in a microservice architecture. It allows to trace requests as they travel through various components, helping to identify bottlenecks, troubleshoot issues, and optimize performance.

Important note: X-Ray does not record every request, instead it uses so-called sampling rules, that capture 1st request plus 5% of any additional requests per second by default. It is not possible to configure X-Ray sampling rate for Lambdas.

API Gateway

Activate X-Ray tracing in your API Gateway:

resource "aws_api_gateway_stage", "default" {
  deployment_id        = aws_api_gateway_deployment.default.id
  rest_api_id          = aws_api_gateway_rest_api.default.id
  tags                 = var.tags
  xray_tracing_enabled = true
}

This covers Client ↔ API Gateway ↔ Lambda tracing.

Lambda

Add a Policy allowing your Lambda to access X-Ray:

resouce "aws_iam_role" "my_lambda_role" {
  name = "my_lambda_role"
  tags = var.tags

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}

resource "aws_iam_role_policy_attachment", "my_lambda_x_ray_policy_attachment" {
  policy_arn = "arn:aws:iam:aws:policy/AWSXRayDaemonWriteAccess"
  role       = "aws_iam_role.my_lambda_role.name
}

And enable Active tracing:

resource "aws_lambda_function" "my_lambda" {
  function_name = var.my_lambda_function_name
  role          = aws_iam_role.my_lambda_role.arn
  handler       = "handler.handler"

  tracing_config {
    mode = "Active"
  }
}

With this setting the Lambda will respect a tracing header it receives from an upstream service, in this case from the API Gateway.

Ruby Application

aws-xray-sdk gem takes care of application-level tracing. By default it traces all net/http and aws clients calls. Gem does it by patching all possible clients, example Aws::DynamoDB::Client. Thus all AWS SDK clients must be required before the X-Ray:

gem "aws-sdk-dynamodb"
gem "aws-xray-sdk"
gem "oj"

require "aws-sdk-dynamodb"
require "aws-xray-sdk/lambda"

aws-xray-sdk gem has a pre-configured XRay.recorder specifically for lambdas, where a segment represents a single function invocation. At the same time this recorder prohibits creation of any new segements.

You can check if a specific client was patched by calling .plugins:

Aws::DynamoDB::Client.plugins # => [..., XRay::AwsSDKPlugin]

This covers Lambda ↔ HTTP and Lambda ↔ DynamoDB tracing.

Localstack

X-Ray tracing does not work with localstack and will log errors cannot find the current context.