Virtual Service Scaling

This article covers the following virtual service optimization topics:

  • Scaling out a virtual service to an additional Avi Service Engine (SE)
  • Scaling in a virtual service back to fewer SEs
  • Migrating a virtual service from one SE to another SE

Avi Vantage supports scaling virtual services, which distributes the virtual service workload across multiple SEs to provide increased capacity on demand, thus extending the throughput capacity of the virtual service and increasing the level of high availability.

  • Scaling out a virtual service distributes that virtual service to an additional SE. By default, Avi Vantage supports a maximum of four SEs per virtual service when native load balancing of SEs is in play. In BGP environments the maximum can be increased to 64.
  • Scaling in a virtual service reduces the number of SEs over which its load is distributed. A virtual service will always require a minimum of one SE.

Operational Notes

apps_vsscaling_1 Native load balancing of SEs. The primary SE (in the middle) is flanked by two secondary SEs.

This section provides additional information for specific infrastructures.

Scaling Avi Virtual Services in VMware / OpenStack with Nuage

For VMware deployments and OpenStack deployments with Nuage, the scaled out traffic behaves as follows:

  • The virtual service IP is GARPed by the primary SE. All inbound traffic from clients will arrive at this SE.
  • The primary SE will handle a percentage of traffic as expected.
  • At Layer 2, excess traffic is forwarded to the MAC address of the additional secondary Service Engine(s).
  • The scaled-out traffic to the secondary SEs is processed as normal. The SEs will change the source IP address of the connection to their own IP address within the server network.
  • The servers will respond to the source IP address of the traffic, which could be the primary or one of the secondary SEs.
  • Secondary SEs will forward the response traffic back to the client, bypassing the primary SE.

Scaling Avi Virtual Services in OpenStack with Neutron

For OpenStack deployments with native Neutron, server response traffic sent to the secondary SEs will be forwarded back to and through the primary SE before returning to the origin client.

Avi Vantage will issue an Alert if the average CPU utilization of an SE exceeds the designated limit during a five-minute polling period. Alerts for additional thresholds can be configured for a virtual service. The process of scaling in or scaling out must be initiated by an administrator. The CPU Threshold field of the SE Group > High Availability tab defines the minimum and maximum CPU percentages.

Scaling Avi Virtual Services in Amazon Web Services (AWS)

For deployments in AWS, the scaled out traffic behaviour is as follows:

  • The virtual service IP is GARPed by the primary SE. All inbound traffic from clients will arrive at this SE.
  • The primary SE will handle a percentage of traffic as expected.
  • At Layer 2, excess traffic is forwarded to the MAC address of the additional secondary Service Engine(s).
  • The scaled-out traffic to the secondary SEs is processed as normal. The SEs will change the source IP address of the connection to their own IP address within the server network.
  • The servers will respond to the source IP address of the traffic, which could be the primary or one of the secondary SEs.
  • Secondary SEs will forward the response traffic back to the client origin, bypassing the primary SE.

Scaling Avi Virtual Services in Microsoft Azure Deployments

Avi Vantage deployments in Microsoft Azure leverage the Azure Load Balancer to provide an ECMP-like, layer 3 scale-out architecture. In this case, the traffic flow is as follows:

  • The virtual service IP resides on the Azure Load Balancer. All inbound traffic from clients will arrive at the Azure LB.
  • The Azure LB has a backend pool consisting of the Avi Service Engines.
  • The Azure LB load balances the traffic to one of the Avi Service Engines associated with the virtual service IP.
  • The traffic to the SEs is processed. The SEs will change the source IP address of the connection to their own IP address within the server network.
  • The servers will respond to the source IP address of the traffic, which could be the primary or one of the secondary SEs.
  • The SEs forward their response traffic directly back to the origin client, bypassing the Azure LB.

Scaling Process

The process used to scale out will depend on the level of access, write access or read/no Access, that Avi Vantage has to the hypervisor orchestrator:

  • If Avi Vantage is in write access mode with write privileges to the virtualization orchestrator, then Avi Vantage will automatically create additional Service Engines when required to share the load. If the Controller runs into an issue while creating a new Service Engine, it will wait a few minutes and then try again on a different host. With native load balancing of SEs in play, the original Service Engine (primary SE) owns and ARPs for the virtual service IP address to process as much traffic as it can. Some percentage of traffic arriving to it will be forwarded via Layer 2 to the additional (secondary) Service Engines. When traffic decreases, the virtual service automatically scales in back to the original primary Service Engine.
  • If Avi Vantage is in read access or no access mode, an administrator must manually create and configure new Service Engines in the virtualization orchestrator. The virtual service can only be scaled out once the Service Engine is both properly configured for the network and connected to the Avi Vantage Controller. > <p style="text-align: center;">Note: Existing Service Engines with spare capacity and appropriate network settings may be used for the scale out; otherwise, scaling out may require either modifying existing Service Engines or creating new Service Engines.

Manual Scaling of Virtual ServicesVS scale in out migrate

Virtual services inherit from their SE group the values for the minimum and maximum number of SEs on which they can be instantiated. [Note: A virtual service’s maximum instantiation count may be well below the maximum number of SEs in its group.] Between the virtual service min/max values, the user can manually scale the virtual service out or in from the UI, CLI, or REST API. Also, current virtual service instantiations on SEs can be migrated to other SEs with the same SE group. The mouse-over popup at right shows how these three actions can be accomplished from within the UI.

Note: For information related to the SE group settings min_scaleout_per_vs and max_scaleout_per_vs, refer to Impact of Changes to Min/Max Scaleout per Virtual Service.

Automatic Scaling of Virtual Services

Virtual services likewise inherit from their SE group the value set for automatic rebalancing of VS instantiations. [Note: Auto-rebalancing applies only if elastic HA has been selected for the SE group]. As shown above, this setting can be checked in the Virtual Service Placement Policy section of the SE group editor. With auto-rebalance in play, and based upon the CPU utilizations of SEs with the group, Avi Vantage will migrate virtual services and – if need be – scale out/in the number of SEs deployed. As a result of an auto-rebalance operation, one or more virtual services in the group may be migrated to alternative SEs and/or their instantiation count adjusted to best serve the current client load.

To configure auto-rebalancing for an SE group, please follow the instructions here.

Scaling Out

To manually scale a virtual service out when Avi Vantage is operating in Write Access mode:

  1. Open the Virtual Service Details page for the virtual service that you want to scale.
  2. Hover the cursor over the name of the virtual service to open the Virtual Service Quick Info popup.
  3. Click the Scale Out button to scale the Virtual Service out to an additional Service Engine per click, up to a maximum of four Service Engines.
  4. If available, Avi Vantage will attempt to use an existing Service Engine. If none is available or matches reachability criteria, it may create a new SE.
  5. In some environments, Avi Vantage may prompt for additional information in order to create a new Service Engine, such as additional IP addresses.

The prompt “Currently scaling out” displays the progress while the operation is taking place.

Note: If avirtual service scales out across multiple Service Engines, then each Service Engine will independently perform server health monitoring to the pool’s servers.

Note: Scaling out does not interrupt existing client connections.

Scaling out a virtual service may take anywhere from a few seconds to a few minutes. The scale out timing depends whether an additional Service Engine exists or if a new one needs to be created, as well as network and disk speeds if creating a new SE.

Scaling In

To manually scale in a virtual service in when Avi Vantage is operating in Write Access mode:

  1. Open the Virtual Service Details page for the virtual service that you want to scale.
  2. Hover the cursor over the name of the virtual service to open the Virtual Service Quick Info popup.
  3. Click the Scale In button to open the Scale In popup window.
  4. Select Service Engine to scale in. In other words, which SE should be removed from supporting this Virtual Service.
  5. Scale the virtual service in by one Service Engine per SE selection, down to a minimum of one Service Engine.

The prompt “Currently scaling in” displays the progress while the operation is taking place.

Note: When Scaling In, existing connections are given thirty seconds to complete. Remaining connections to the SE are closed and must restart.

Migrating

The Migrate option allows graceful migration from one Service Engine to another. During this process, the primary SE will scale out to the new SE and begin sending it new connections. After thirty seconds, the old SE will be deprovisioned from supporting the virtual service.

Note: Existing connections to the migration’s source SE will be given thirty seconds to complete prior to the SE being deprovisioned for the virtual service. Remaining connections to the SE are closed and must restart.