Configuration and Metrics Collection on Avi Vantage for AWS Server Autoscaling

Overview

Cloud server autoscaling is a new feature in Avi Vantage Platform that enables infrastructure admins, operators, and application developers to use the Avi server autoscaling solution in concert with cloud Auto Scaling groups. Auto Scaling groups in the AWS are referred as external autoscaling groups in Avi Vantage since they are an external entity to Avi Vantage. With this feature, more fine-grained scaling policies can be applied, based on Avi Controller-collected metrics and AWS CloudWatch metrics.

Metrics used for server autoscaling

Metrics collected from Avi Service Engine, and AWS cloud enable Avi Vantage to take action regarding scale-out or scale-in events. Metrics for services hosted on Avi Service Engine are collected by Avi SEs. Server instance (VM) level metrics are collected from AWS CloudWatch.

Metrics collected by Avi Service Engines

  • Level 4 metrics
  • Level 7 metrics
  • Service Engine metrics
  • Insight metrics

For the complete list of metrics collected by Avi Vantage, refer to Metrics List.

Metrics collected by AWS

Infrastructure-related metrics for server instances such as CPU usage, network usage, etc., are fetched from CloudWatch by Avi Vantage. The metrics collected from AWS are:

  • vm_stats.avg_cpu_usage
  • vm_stats.avg_disk_read
  • vm_stats.avg_disk_write
  • vm_stats.avg_disk_io
  • vm_stats.avg_net_usage

Autoscale policy and autoscale launch configuration

The above-mentioned metrics are fetched for a pool when the pool is enabled for autoscale orchestration. The metrics mentioned in the previous section, are fetched for a pool when the pool is enabled for autoscaling. For server autoscaling to work, the pool must be enabled with an autoscale policy and an autoscale launch configuration.

Autoscale policy

An Autoscale policy is a set of rules to configure and trigger an alert using the above-mentioned metrics. To create or choose an existing autoscale policy, navigate to Applications > Pools and click on the edit icon for the desired pool. Click on AutoScale Policy option under Settings to add a new autoscale policy or to use an existing one.

pool-edit autoscale-policy

Autoscale launch configuration

An autoscale launch configuration must be associated with the pool for server autoscaling to work.

Setting the value for use-external-asg to true instructs the Autoscale Manager to start orchestrating scale-in or scale-out activities for the associated pool. The value for use-external-asg flag is set to true for the default autoscale launch configuration (default-autoscalelaunchconfig).

autoscale-launch-config

To enable the checkbox for Use External ASG, navigate to Applications > Pools, and click on the edit icon for the desired pool. The option for AutoScale Launch Config is available under Settings tab. Select the drop-down menu for Autoscale Launch Config, provide the desired name, and check the Use External ASG box.

drop-down-menu

use-external-asg

AZ Aware Autoscaling

Starting with Avi Vantage version 20.1.4, While scaling-in, Avi Vantage autoscale will ensure balance of servers across different AWS availability zones. For instance, if there are four servers in a pool (two servers each in AZ1 and AZ2), and scale-in happens for two servers, you will left with two Servers (one in each AZ).

Note: This feature is available only for AWS.

Configuration

Perform the following steps to enable a pool for server autoscaling:

  • Navigate to Applications > Pools. Click on Create Pool and select the required cloud.

    create-pool select-pool

  • Click the drop-down menu for Autoscale Policy and select Create Autoscale Policy.

    autoscale-policy create-autoscale-policy

  • On the new Autoscale policy page, provide the desired name, and minimum and maximum instances for the pool.

    new-policy

    To create a new alert for server autoscaling, select Create from the drop-down option in the Scale Out section.

    create-alert

  • Provide the name, select Metrics as Source, and check the AutoScale Alert box. For Metrics Occurs option, select the autoscaling metrics to be used by the pool. In the below example, average CPU usage is selected as the metric.

    scaleout-alert-name

    Select option for Comparator from the drop-down menu and duration for which the metrics statistics is compared with given value for raising an alert for autoscale. In this example, the autoscale alerts will be generated when average CPU usage exceeds 50% for a duration of 300 seconds.

    alert-configuration

  • Specify alert action. An existing alert action can be selected, for example, generating a medium level system event. Click on Save.

    alert-action

  • Navigate again to the autoscale policy page, specify a value for Cooldown Period and Adjustment Step.

    Cooldown Period ** – During this period no new scale-out event is triggered to allow the previous scaleout to complete.

    Adjustment Step – Maximum number of servers to scalein simultaneously. This number is chosen such that target number of servers is always more than or equal to the min_size.

    cooldown-adjustment

  • To create a policy for scalein, follow similar steps as mentioned previously for the scale-out policy.

  • Navigate to Applications>Pools and select the drop-down menu for AutoScale Launch Config to create a new autoscale launch configuration.

  • Navigate to Servers tab of Edit Pool page, select AutoScaling Groups button and from the drop-down select the autoscaling group for the pool.

    select-asg

  • Click Next and perform configuration for the Advanced section if required, click Save.

  • Create a virtual service for the configured pool with autoscaling group.

Autoscale activities and events

Autoscale alerts are generated and pool members scaleout when configured metrics exceed the threshold values. Following are the events generated by the pool for a scale-in and scale-out activity.

Events generated for scale-out activity

  1. SERVER_AUTOSCALE_OUT – An autoscale scale-out alert is generated by alerts manager.
  2. SERVER_AUTOSCALE_OUT_TRIGGERED – The autoscale manager triggers scale-out activity on pool.
  3. CONFIG_UPDATE – The pool is updated with the new member.
  4. AWS_ASG_NOTIFICATION_INSTANCE_ADDED – An instance is added to AWS Auto Scaling Group.
  5. SERVER_AUTOSCALE_OUT_COMPLETE – The autoscale manager triggers this event when the scale out activity is complete. Indicates scaleout is successfully complete.
  6. SERVER_UP – Indicates the newly added server is ready to serve traffic.

Navigate to Templates > Events to check alerts generated for scale-out or scale-in event. Below screenshot is a sample for reference purpose.

select-asg

The alert AWS_ASG_NOTIFICATION_INSTANCE_ADDED has details of the newly added instance. In the below screenshot, a new instance having IP 10.148.87.23 is added.

instance-added

Events generated for scale-in activity

  1. SERVER_AUTOSCALE_IN – An autoscale scale-out alert is generated by alerts manager.
  2. SERVER_AUTOSCALE_IN_TRIGGERED – The autoscale manager triggers scale in activity on pool.
  3. CONFIG_UPDATE – The pool was updated and the scaled-in pool member is deleted.
  4. AWS_ASG_NOTIFICATION_INSTANCE_REMOVED – Indicates an instance has been removed from AWS Auto Scaling Group.
  5. SERVER_AUTOSCALE_IN_COMPLETE – The autoscale manager triggers this event when the scale-in activity is complete. Indicates scale-in activity is successfully complete.

Refer to the below screenshot for reference purpose.

scalein-alerts

SERVER_AUTOSCALE_IN_TRIGGERED event has details of pool member (AWS server instance) that have been terminated and removed from the pool.

autoscalein-trigerred

AWS_ASG_NOTIFICATION_INSTANCE_REMOVED – This indicates that a corresponding AWS instance (10.148.177.204) has been removed.

instance-removed

Note: Burstable Performance Instance types are not supported for CPU Utilization based autoscaling. Burstable Performance Instance types are AWS instance types with their names starting with T2, i.e., T2.micro, T2.large, etc.

### Document Revision History

Date Change Summary
February 15, 2021 Added AZ Aware Autoscaling details