Skip to content

Application Load Balancing (ALB) Logs

ALB Access logs provide access logs that capture detailed information about requests sent to your load balancer. ALB publishes a log file for each load balancer node every 5 minutes.

You can create a log ingestion into Amazon OpenSearch Service or Light Engine either by using the Centralized Logging with OpenSearch console or by deploying a standalone CloudFormation stack.

Important

  • The ELB logging bucket's region must be the same as the Centralized Logging with OpenSearch solution.
  • The Amazon OpenSearch Service index is rotated on a daily basis by default, and you can adjust the index in the Additional Settings.

Create log ingestion (Amazon OpenSearch for log analytics)

Using the Console

  1. Sign in to the Centralized Logging with OpenSearch Console.
  2. In the navigation pane, under Log Analytics Pipelines, choose Service Log.
  3. Choose the Create a log ingestion button.
  4. In the AWS Services section, choose Elastic Load Balancer.
  5. Choose Amazon OpenSearch, Choose Next.
  6. Under Specify settings, choose Automatic or Manual.
    • For Automatic mode, choose an application load balancer in the dropdown list. (If the selected ALB access log is not enabled, click Enable to enable the ALB access log.)
    • For Manual mode, enter the Application Load Balancer identifier and Log location.
    • (Optional) If you are ingesting logs from another account, select a linked account from the Account dropdown first.
  7. Choose Next.
  8. In the Specify OpenSearch domain section, select an imported domain for Amazon OpenSearch domain.
  9. Choose Yes for Sample dashboard if you want to ingest an associated templated Amazon OpenSearch Service dashboard.
  10. You can change the Index Prefix of the target Amazon OpenSearch Service index if needed. The default prefix is the Load Balancer Name.
  11. In the Log Lifecycle section, input the number of days to manage the Amazon OpenSearch Service index lifecycle. The Centralized Logging with OpenSearch will create the associated Index State Management (ISM) policy automatically for this pipeline.
  12. In the Select log processor section, please choose the log processor.
    • (Optional) OSI as log processor is now supported in these regions. When OSI is selected, please type in the minimum and maximum number of OCU. See more information here.
  13. Choose Next.
  14. Add tags if needed.
  15. Choose Create.

Using the CloudFormation Stack

This automated AWS CloudFormation template deploys the Centralized Logging with OpenSearch - ELB Log Ingestion solution in the AWS Cloud.

Launch in AWS Console Download Template
AWS Regions Launch Stack Template
AWS China Regions Launch Stack Template

  1. Log in to the AWS Management Console and select above button to launch the AWS CloudFormation template. You can also download the template as a starting point for your own implementation.

  2. To launch the stack in a different AWS Region, use the Region selector in the console navigation bar.

  3. On the Create stack page, verify that the correct template URL shows in the Amazon S3 URL text box and choose Next.

  4. On the Specify stack details page, assign a name to your solution stack.

  5. Under Parameters, review the parameters for the template and modify them as necessary. This solution uses the following parameters.

    Parameter Default Description
    Log Bucket Name <Requires input> The S3 bucket name which stores the logs.
    Log Bucket Prefix <Requires input> The S3 bucket path prefix which stores the logs.
    Log Source Account ID <Optional input> The AWS Account ID of the S3 bucket. Required for cross-account log ingestion (Please add a member account first). By default, the Account ID you logged in at Step 1 will be used.
    Log Source Region <Optional> The AWS Region of the S3 bucket. By default, the Region you selected at Step 2 will be used.
    Log Source Account Assume Role <Optional> The IAM Role ARN used for cross-account log ingestion. Required for cross-account log ingestion (Please add a member account first).
    Engine Type OpenSearch The engine type of the OpenSearch. Select OpenSearch or Elasticsearch.
    OpenSearch Domain Name <Requires input> The domain name of the Amazon OpenSearch cluster.
    OpenSearch Endpoint <Requires input> The OpenSearch endpoint URL. For example, vpc-your_opensearch_domain_name-xcvgw6uu2o6zafsiefxubwuohe.us-east-1.es.amazonaws.com
    Index Prefix <Requires input> The common prefix of OpenSearch index for the log. The index name will be <Index Prefix>-<Log Type>-<Other Suffix>.
    Create Sample Dashboard Yes Whether to create a sample OpenSearch dashboard.
    VPC ID <Requires input> Select a VPC which has access to the OpenSearch domain. The log processing Lambda will reside in the selected VPC.
    Subnet IDs <Requires input> Select at least two subnets which have access to the OpenSearch domain. The log processing Lambda will reside in the subnets. Make sure the subnets have access to the Amazon S3 service.
    Security Group ID <Requires input> Select a Security Group which will be associated with the log processing Lambda. Make sure the Security Group has access to the OpenSearch domain.
    S3 Backup Bucket <Requires input> The S3 backup bucket name to store the failed ingestion logs.
    KMS-CMK ARN <Optional> The KMS-CMK ARN for encryption. Leave it blank to create a new KMS CMK.
    Number Of Shards 5 Number of shards to distribute the index evenly across all data nodes. Keep the size of each shard between 10-50 GB.
    Number of Replicas 1 Number of replicas for OpenSearch Index. Each replica is a full copy of an index.
    Age to Warm Storage <Optional> The age required to move the index into warm storage (e.g. 7d). Index age is the time between its creation and the present. Supported units are d (days) and h (hours). This is only effective when warm storage is enabled in OpenSearch.
    Age to Cold Storage <Optional> The age required to move the index into cold storage (e.g. 30d). Index age is the time between its creation and the present. Supported units are d (days) and h (hours). This is only effective when cold storage is enabled in OpenSearch.
    Age to Retain <Optional> The age to retain the index (e.g. 180d). Index age is the time between its creation and the present. Supported units are d (days) and h (hours). If value is "", the index will not be deleted.
    Rollover Index Size <Optional> The minimum size of the shard storage required to roll over the index (e.g. 30GB).
    Index Suffix yyyy-MM-dd The common suffix format of OpenSearch index for the log(Example: yyyy-MM-dd, yyyy-MM-dd-HH). The index name will be <Index Prefix>-<Log Type>-<Index Suffix>-000001.
    Compression type best_compression The compression type to use to compress stored data. Available values are best_compression and default.
    Refresh Interval 1s How often the index should refresh, which publishes its most recent changes and makes them available for searching. Can be set to -1 to disable refreshing. Default is 1s.
    Plugins <Optional> List of plugins delimited by comma. Leave it blank if there are no available plugins to use. Valid inputs are user_agent, geo_ip.

  6. Choose Next.

  7. On the Configure stack options page, choose Next.

  8. On the Review page, review and confirm the settings. Check the box acknowledging that the template creates AWS Identity and Access Management (IAM) resources.

  9. Choose Create stack to deploy the stack.

You can view the status of the stack in the AWS CloudFormation console in the Status column. You should receive a CREATE_COMPLETE status in approximately 10 minutes.

View dashboard

The dashboard includes the following visualizations.

Visualization Name Source Field Description
Total Requests
  • log event
Displays aggregated events based on a specified time interval.
Request History
  • log event
Presents a bar chart that displays the distribution of events over time.
Request By Target
  • log event
  • target_ip
Presents a bar chart that displays the distribution of events over time and IP.
Unique Visitors
  • client_ip
Displays unique visitors identified by client IP address.
Status Code
  • elb_status_code
 Displays the count of requests made to the ALB, grouped by HTTP status codes (e.g., 200, 404, 403, etc.).
Status History
  • elb_status_code
Shows the historical trend of HTTP status codes returned by the ALB over a specific period of time.
Status Code Pipe
  • elb_status_code
 Represents the distribution of requests based on different HTTP status codes using a pie chart.
Average Processing Time
  • request_processing_time
  • response_processing_time
  • target_processing_time
 This visualization calculates and presents the average time taken for various operations in the ALB.
Avg. Processing Time History
  • request_processing_time
  • response_processing_time
  • target_processing_time
 Displays the historical trend of the average time-consuming of each operation returned by the ALB within a specific period of time.
Request Verb
  • request_verb
 Displays the count of requests made to the ALB using a pie chart, grouped by http request method names (e.g., POST, GET, HEAD, etc.).
Total Bytes
  • received_bytes
  • sent_bytes
 Provides insights into data transfer activities, including the total bytes transferred.
Sent and Received Bytes History
  • received_bytes
  • sent_bytes
 Displays the historical trend of the the received bytes, send bytes
SSL Protocol
  • ssl_protocol
 Displays the count of requests made to the ALB, grouped by SSL Protocol
Top Request URLs
  • request_url
 The web requests view enables you to analyze the top web requests.
Top Client IPs
  • client_ip
 Provides the top 10 IP address accessing your ALB.
Top User Agents
  • user_agent
 Provides the top 10 user agents accessing your ALB.
Target Status
  • target_ip
  • target_status_code
 Displays the http status code request count for targets in ALB target group.
Abnormal Requests
  • @timestamp
  • client_ip
  • target_ip
  • elb_status_code
  • error_reason
  • request_verb
  • target_status_code
  • target_status_code_list
  • request_url
  • request_proto
  • trace_id
 Provides a detailed list of log events, including timestamps, client ip, target ip, etc.
Requests by OS
  • ua_os
 Displays the count of requests made to the ALB, grouped by user agent OS
Request by Device
  • ua_device
 Displays the count of requests made to the ALB, grouped by user agent device.
Request by Browser
  • ua_browser
 Displays the count of requests made to the ALB, grouped by user agent browser.
Request by Category
  • ua_category
 Displays the count of category made to the ALB, grouped by user agent category (e.g., PC, Mobile, Tablet, etc.).
Requests by Countries or Regions
  • geo_iso_code
 Displays the count of requests made to the ALB (grouped by the corresponding country or region resolved by the client IP).
Top Countries or Regions
  • geo_country
 Top 10 countries with the ALB Access.
Top Cities
  • geo_city
 Top 10 cities with ALB Access

Sample Dashboard

Create log ingestion (Light Engine for log analytics)

Using the Console

  1. Sign in to the Centralized Logging with OpenSearch Console.
  2. In the navigation pane, under Log Analytics Pipelines, choose Service Log.
  3. Choose the Create a log ingestion button.
  4. In the AWS Services section, choose Elastic Load Balancer.
  5. Choose Light Engine, Choose Next.
  6. Under Specify settings, choose Automatic or Manual.
    • For Automatic mode, choose an application load balancer in the dropdown list. (If the selected ALB access log is not enabled, click Enable to enable the ALB access log.)
    • For Manual mode, enter the Application Load Balancer identifier and Log location.
    • (Optional) If you are ingesting logs from another account, select a linked account from the Account dropdown first.
  7. Choose Next.
  8. Log Processing Enriched fields，The available plugins to choose from are location and OS/User Agent. Enabling rich fields will increase data processing latency and processing costs, it is not selected by default.
  9. In the Specify Light Engine Configuration section, if you want to ingest associated templated Grafana dashboards, select Yes for the sample dashboard.
  10. You can choose an existing Grafana, or if you need to import a new one, you can go to Grafana for configuration.
  11. Select an S3 bucket to store partitioned logs and define a name for the log table. We have provided a predefined table name, but you can modify it according to your business needs.
  12. The log processing frequency is set to 5 minutes by default, with a minimum processing frequency of 1 minute.
  13. In the Log Lifecycle section, enter the log merge time and log archive time. We have provided default values, but you can adjust them based on your business requirements.
  14. Select Next.
  15. If desired, add tags.
  16. Select Create.

Using the CloudFormation Stack

This automated AWS CloudFormation template deploys the Centralized Logging with OpenSearch - ELB Log Ingestion solution in the AWS Cloud.

Launch in AWS Console Download Template
AWS Regions Launch Stack Template
AWS China Regions Launch Stack Template

  1. Log in to the AWS Management Console and select the button to launch the AWS CloudFormation template. You can also download the template as a starting point for your own implementation.

  2. To launch the stack in a different AWS Region, use the Region selector in the console navigation bar.

  3. On the Create stack page, verify that the correct template URL shows in the Amazon S3 URL text box and choose Next.

  4. On the Specify stack details page, assign a name to your solution stack.

  5. Under Parameters, review the parameters for the template and modify them as necessary. This solution uses the following parameters.

    • Parameters for Pipeline settings
    Parameter Defaul Description
    Pipeline Id <Requires input> The unique identifier for the pipeline is essential if you need to create multiple ALB pipelines and write different ALB logs into separate tables. To ensure uniqueness, you can generate a unique pipeline identifier using uuidgenerator.
    Staging Bucket Prefix AWSLogs/ALBLogs The storage directory for logs in the temporary storage area should ensure the uniqueness and non-overlapping of the Prefix for different pipelines.
    • Parameters for Destination settings
    Parameters Default Description
    Centralized Bucket Name <Requires input> Input centralized s3 bucket name，for expample:centralized-logging-bucket。
    Centralized Bucket Prefix datalake Input centralized bucket prefix，default is datalake which means your data base's location is s3://{Centralized Bucket Name}/{Centralized Bucket Prefix}/amazon_cl_centralized。
    Centralized Table Name ALB Table name for writing data to the centralized database, can be defined as needed, default value is 'ALB'.
    Enrichment Plugins <Optional input> The available plugins to choose from are location and OS/User Agent. Enabling rich fields will increase data processing latency and processing costs, it is not selected by default.
    • Parameters for Scheduler settings
    Parameters Default Description
    LogProcessor Schedule Expression rate(5 minutes) Task scheduling expression for performing log processing, with a default value of executing the LogProcessor every 5 minutes. Configuration for reference。
    LogMerger Schedule Expression cron(0 1 * ? ) 执ask scheduling expression for performing log merging, with a default value of executing the LogMerger at 1 AM every day. Configuration for reference。
    LogArchive Schedule Expression cron(0 2 * ? ) Task scheduling expression for performing log archiving, with a default value of executing the LogArchive at 2 AM every day. Configuration for reference。
    Age to Merge 7 Small file retention days, with a default value of 7, indicates that logs older than 7 days will be merged into small files. It can be adjusted as needed.
    Age to Archive 30 Log retention days, with a default value of 30, indicates that data older than 30 days will be archived and deleted. It can be adjusted as needed.
    • Parameters for Notification settings
    Parameters Default Description
    Notification Service SNS Notification method for alerts. If your main stack is using China, you can only choose the SNS method. If your main stack is using Global, you can choose either the SNS or SES method.
    Recipients <Requires Input> Alert notification: If the Notification Service is SNS, enter the SNS Topic ARN here, ensuring that you have the necessary permissions. If the Notification Service is SES, enter the email addresses separated by commas here, ensuring that the email addresses are already Verified Identities in SES. The adminEmail provided during the creation of the main stack will receive a verification email by default.
    • Parameters for Dashboard settings
    Parameters Default Description
    Import Dashboards FALSE Whether to import the Dashboard into Grafana, with a default value of false. If set to true, you must provide the Grafana URL and Grafana Service Account Token.。
    Grafana URL <Requires Input> Grafana access URL，for example: https://alb-72277319.us-west-2.elb.amazonaws.com。
    Grafana Service Account Token <Requires Input> Grafana Service Account Token：Service Account Token created in Grafana。

  6. Choose Next.

  7. On the Configure stack options page, choose Next.

  8. On the Review page, review and confirm the settings. Check the box acknowledging that the template creates AWS Identity and Access Management (IAM) resources.

  9. Choose Create stack to deploy the stack.

You can view the status of the stack in the AWS CloudFormation console in the Status column. You should receive a CREATE_COMPLETE status in approximately 10 minutes.

View Dashboard

Visualization Name Source Field Description
Filters Filters The following data can be filtered by query filter conditions.
Total Requests log event Displays aggregated events based on a specified time interval.
Unique Visitors client_ip Displays unique visitors identified by client IP address.
Requests History log event Presents a bar chart that displays the distribution of events over time.
Request By Target log event
target_ip		 Presents a bar chart that displays the distribution of events over time and IP.
HTTP Status Code elb_status_code Displays the count of requests made to the ALB, grouped by HTTP status codes (e.g., 200, 404, 403, etc.).
Status Code History elb_status_code Shows the historical trend of HTTP status codes returned by the ALB over a specific period of time.
Status Code Pie elb_status_code Represents the distribution of requests based on different HTTP status codes using a pie chart.
Average Processing Time request_processing_time
response_processing_time
target_processing_time		 This visualization calculates and presents the average time taken for various operations in the ALB.
Avg. Processing Time History request_processing_time
response_processing_time
target_processing_time		 Displays the historical trend of the average time-consuming of each operation returned by the ALB within a specific period of time.
HTTP Method request_verb Displays the count of requests made to the ALB using a pie chart, grouped by HTTP request method names (e.g., POST, GET, HEAD, etc.).
Total Bytes received_bytes
sent_bytes		 Provides insights into data transfer activities, including the total bytes transferred.
Sent and Received Bytes History received_bytes
sent_bytes		 Displays the historical trend of the received bytes, send bytes.
SSL Protocol ssl_protocol Displays the count of requests made to the ALB, grouped by SSL Protocol.
Top Request URLs request_url The web requests view enables you to analyze the top web requests.
Top Client IPs client_ip Provides the top 10 IP addresses accessing your ALB.
Bad Requests type client_ip
target_group_arn
target_ip elb_status_code
request_verb
request_url ssl_protocol
received_bytes
sent_bytes		 Provides a detailed list of log events, including timestamps, client IP, target IP, etc.
Requests by OS ua_os Displays the count of requests made to the ALB, grouped by user agent OS.
Requests by Device ua_device Displays the count of requests made to the ALB, grouped by user agent device.
Requests by Browser ua_browser Displays the count of requests made to the ALB, grouped by user agent browser.
Requests by Category ua_category Displays the count of category made to the ALB, grouped by user agent category (e.g., PC, Mobile, Tablet, etc.).
Requests by Countries or Regions geo_iso_code Displays the count of requests made to the ALB (grouped by the corresponding country or region resolved by the client IP).
Top Countries or Regions geo_country Top 10 countries with the ALB Access.
Top Cities geo_city Top 10 cities with ALB Access.