Avi Vantage Integration with GCP Instance Groups
Overview
Starting with Avi Vantage version 18.2.9 GCP instance groups can be configured in the Avi pools for a virtual service.
Avi Controller periodically polls GCP instance groups configured in Avi pool and update the pool servers with the instances in the GCP instance groups. In addition to polling GCP pub/ sub notification service is also used to get the notifications of the instances created and deleted in a GCP instance group.
If a new instance is added to a GCP instance group which is configured in an Avi pool, Avi Vantage will update the pool membership to include the newly provisioned instance. Conversely, upon deletion of the instance from GCP instance group, Avi Vantage will delete this server from its pool membership. This enables seamless, elastic and automated management of backend server resources without any operator intervention or configuration updates.
Avi Vantage supports both GCP managed and un-managed instance groups. Instance groups can be in any GCP project but should have required permissions as stated below.
Refer to Roles and Permissions (GCP Full Access) for details on the permissions required in GCP. You can configure the service account to have the permissions for GCP Instance Group Auto Scaling feature in Service Engine project and for instance group (server) project.
Configuring Pool
Configuring via Avi CLI
The following are the steps to configure GCP instance groups via Avi CLI:
-
Set the external_autoscale_groups field in the Avi pool with the list of GCP instance group in
InstanceGroupName@InstanceGroupProjectIDformat. -
The instance groups can be shared across multiple Avi pools.
Example
You can add two GCP instance groups to a pool and both of them belongs to different GCP projects. The following are the CLI details:
[admin:controller]: > configure pool pool-1
[admin:controller]: pool> cloud_ref gcp-cloud
[admin:controller]: pool> external_autoscale_groups instance-group-name-1@instance-group-project-1
[admin:controller]: pool> external_autoscale_groups instance-group-name-2@instance-group-project-2
[admin:controller]: pool> save
+---------------------------------------+-----------------------------------------------------+
| Field | Value |
+---------------------------------------+-----------------------------------------------------+
| uuid | pool-ea2ee84d-a51e-451f-b59e-4906a4a0a4e2 |
| name | pool-1 |
| default_server_port | 80 |
| graceful_disable_timeout | 1 min |
| connection_ramp_duration | 10 min |
| max_concurrent_connections_per_server | 0 |
| lb_algorithm | LB_ALGORITHM_LEAST_CONNECTIONS |
| lb_algorithm_hash | LB_ALGORITHM_CONSISTENT_HASH_SOURCE_IP_ADDRESS |
| inline_health_monitor | True |
| use_service_port | False |
| capacity_estimation | False |
| capacity_estimation_ttfb_thresh | 0 milliseconds |
| vrf_ref | global |
| fewest_tasks_feedback_delay | 10 sec |
| enabled | True |
| request_queue_enabled | False |
| request_queue_depth | 128 |
| host_check_enabled | False |
| sni_enabled | True |
| rewrite_host_header_to_sni | False |
| rewrite_host_header_to_server_name | False |
| external_autoscale_groups[1] | instance-group-name-1@instance-group-project-1 |
| external_autoscale_groups[2] | instance-group-name-2@instance-group-project-2 |
| lb_algorithm_core_nonaffinity | 2 |
| lookup_server_by_name | False |
| analytics_profile_ref | System-Analytics-Profile |
| tenant_ref | admin |
| cloud_ref | gcp-cloud |
| server_timeout | 0 milliseconds |
| delete_server_on_dns_refresh | True |
| enable_http2 | False |
| ignore_server_port | False |
| routing_pool | False |
+---------------------------------------+-----------------------------------------------------+
Refer to Roles and Permissions (GCP Full Access) guide for roles and permissions of auto-scaling Service Engine project.
Configuring Cloud
The following are the steps to configure cloud:
-
You can configure the polling interval by using autoscale_polling_interval field in the cloud configuration.
-
It is recommended to increase the polling interval to 5 minutes if GCP pub/sub is configured for the instance group notification. This is required for the reconciliation of the Avi pool configuration after every periodic interval.
Configuring Cloud via Avi CLI
The following are the CLI details to configure cloud via Avi CLI:
[admin:controller]: > configure cloud gcp-cloud
Updating an existing object.
[admin:controller]: cloud> autoscale_polling_interval 300
Overwriting the previously entered value for autoscale_polling_interval
[admin:controller]: cloud> save
+------------------------------+--------------------------------------------+
| Field | Value |
+------------------------------+--------------------------------------------+
| uuid | cloud-32cd1a1e-bfc0-40f9-940b-1b37408ffa67 |
| name | gcp-cloud |
| vtype | CLOUD_GCP |
| apic_mode | False |
| gcp_configuration | |
| cloud_credentials_ref | gcp-service-account |
| region_name | us-central1 |
| zones[1] | us-central1-a |
| zones[2] | us-central1-b |
| se_project_id | se-project-id |
| network_config | |
| config | INBAND_MANAGEMENT |
| inband | |
| vpc_subnet_name | subnet-1 |
| vpc_project_id | network-project-id |
| vpc_network_name | dev-net-1 |
| vip_allocation_strategy | |
| mode | ROUTES |
| dhcp_enabled | True |
| mtu | 1500 bytes |
| prefer_static_routes | False |
| enable_vip_static_routes | False |
| license_type | LIC_CORES |
| state_based_dns_registration | True |
| ip6_autocfg_enabled | False |
| dns_resolution_on_se | False |
| enable_vip_on_all_interfaces | False |
| tenant_ref | admin |
| license_tier | ENTERPRISE |
| autoscale_polling_interval | 300 seconds |
+------------------------------+--------------------------------------------+
Tracking of Instances in GCP Instance Group
For tracking instance in GCP instance group, the polling of instance groups and notifications from GCP StackDriver logging are used.
Server Updates using GCP Stackdriver Logging and GCP Pub/Sub
The following are the server updates using GCP StackDriver logging and GCP pub/sub:
-
The Avi Controller creates one GCP pub/sub topic and one GCP pub/sub subscription in the Service Engine project for each cloud.
-
The Avi Controller creates a Stackdriver log sink for each instance group in the server project
-
Whenever a instance is added or removed from a GCP instance group, a log entry is created in GCP StackDriver logging.
-
The log entry gets matched with the query of the configured instance groups sink and if matches then it will be exported to the pub/sub topic in Service Engine project.
-
The Avi Controller gets the notification from GCP pub/sub whenever instance is added or removed from the instance group and it updates all the Avi pools with the configured instance group.
Server Updates through Polling
The periodic polling of the GCP instance groups is also done which syncs the topic, subscriptions, sinks and servers in case some updates were missed.
Servers in AVI Pool
The Avi pool will be updated with the servers once the GCP pub/sub notifications are processed by the Avi Controller.
[admin:10-138-10-50]: > show pool pool1
+---------------------------------------+----------------------------------------------------------------------------------+
| Field | Value |
+---------------------------------------+----------------------------------------------------------------------------------+
| uuid | pool-ea2ee84d-a51e-451f-b59e-4906a4a0a4e2 |
| name | pool1 |
| default_server_port | 80 |
| graceful_disable_timeout | 1 min |
| connection_ramp_duration | 10 min |
| max_concurrent_connections_per_server | 0 |
| servers[1] | |
| ip | 10.20.0.8 |
| hostname | instance-group-1-bf52 |
| enabled | True |
| ratio | 1 |
| external_uuid | https://www.googleapis.com/compute/v1/projects/instance-group-project-1/zones/us |
| | -central1-c/instances/instance-group-1-bf52 |
| verify_network | False |
| resolve_server_by_dns | False |
| static | False |
| rewrite_host_header | False |
| autoscaling_group_name | instance-group-1@instance-group-project-1 |
| servers[2] | |
| ip | 10.20.0.9 |
| hostname | instance-group-1-9phd |
| enabled | True |
| ratio | 1 |
| external_uuid | https://www.googleapis.com/compute/v1/projects/instance-group-project-1/zones/us |
| | -central1-b/instances/instance-group-1-9phd |
| verify_network | False |
| resolve_server_by_dns | False |
| static | False |
| rewrite_host_header | False |
| autoscaling_group_name | instance-group-1@instance-group-project-1 |
| servers[3] | |
| ip | 10.20.0.7 |
| hostname | instance-group-1-s078 |
| enabled | True |
| ratio | 1 |
| external_uuid | https://www.googleapis.com/compute/v1/projects/instance-group-project-1/zones/us |
| | -central1-f/instances/instance-group-1-s078 |
| verify_network | False |
| resolve_server_by_dns | False |
| static | False |
| rewrite_host_header | False |
| autoscaling_group_name | instance-group-1@instance-group-project-1 |
| lb_algorithm | LB_ALGORITHM_LEAST_CONNECTIONS |
| lb_algorithm_hash | LB_ALGORITHM_CONSISTENT_HASH_SOURCE_IP_ADDRESS |
| inline_health_monitor | True |
| use_service_port | False |
| capacity_estimation | False |
| capacity_estimation_ttfb_thresh | 0 milliseconds |
| vrf_ref | global |
| fewest_tasks_feedback_delay | 10 sec |
| enabled | True |
| request_queue_enabled | False |
| request_queue_depth | 128 |
| host_check_enabled | False |
| sni_enabled | True |
| rewrite_host_header_to_sni | False |
| rewrite_host_header_to_server_name | False |
| external_autoscale_groups[1] | instance-group-1@instance-group-project-1 |
| lb_algorithm_core_nonaffinity | 2 |
| lookup_server_by_name | False |
| analytics_profile_ref | System-Analytics-Profile |
| tenant_ref | admin |
| cloud_ref | gcp-cloud |
| server_timeout | 0 milliseconds |
| delete_server_on_dns_refresh | True |
| enable_http2 | False |
| ignore_server_port | False |
| routing_pool | False |
+---------------------------------------+----------------------------------------------------------------------------------+