Troubleshooting GCP Cloud

Overview

This guide explains the troubleshooting techniques for GCP cloud.

Cloud Configuration Issues

The following are the troubleshooting techniques of cloud configuration issues:

Check the cloud status in Avi UI. The status should have the reason for the cloud not being up.

Invalid Configuration

Ensure that the values in the GCP cloud configuration are correct. Avi GCP cloud will be in failed state if it fails to validate cloud configuration.

Permission Issues

Ensure that the service account configured either in the Controller virtual machine or as part of the Avi GCP cloud configuration has the required permissions in all the required projects.

Refer to GCP Full Access Roles and Permissions for more details.

Service Engine Image

After a GCP cloud is successfully created, a Service Engine image should be present in Service Engine GCP project. The prefix of image name is avi-se and image will have a label named cloudid: <avi-cloud-uuid>.

Service Engine Issues

The following are the troubleshooting techniques of Service Engine issues:

Bootup Issues

If the Service Engine VM is failing to boot up, then check the Machine Type set in the Service Engine group. If the disk allocated is too low, then VM would fail to boot up.
Check the GCP console logs to narrow down the issue.

Service Engine failing to connect to Avi Controller

Ensure that Service Engine VMs have connectivity to the Controller. Check the VPC settings in network configuration of the Avi GCP cloud.
Check the firewall rules on the Service Engine VM and the Controller. The firewall rules should allow Avi Controller to SE Communication and vice versa. Refer to Firewall Configuration guide.

Virtual Service Issues

The following are the troubleshooting techniques of virtual service issues:

Reachability Issues

The firewall rules for Service Engine VMs should allow traffic from the clients.
The backend servers firewall rules should allow traffic from the Service Engines. Refer to Firewall Configuration guide.
If static routes are configured for the backend server’s reachability from Service Engines, then ensure that the next hop is configured as the gateway of the Service Engine subnet.

VIP as Routes - applicable if VIP allocation strategy mode is Routes

Ensure that the VIP subnet (Avi Internal Network) does not overlap with any subnet in GCP data VPCs.
VIP subnets (Avi internal network) should be unique across Avi clusters.
Verify that there are no out of band or stale overlapping routes for the VIP in GCP data VPCs. This can prevent the traffic from reaching the Service Engines for the VIP. Avi Controller does not manage routes which are not created by it.

VIP as ILB - applicable if VIP allocation strategy mode is ILB

In a VPC, ILB VIP subnet cannot be same as the Service Engine subnet. Ensure that VIP subnet is different than the Service Engine subnet for that VPC.
For ILB healthchecks, firewall rules for Service Engine VMs should allow traffic from the following source IP ranges:
- 35.191.0.0/16
- 130.211.0.0/22
Ensure that the cloud routers (if configured) are not shared across Avi clusters.

Logging

Check the following logs for errors at the following locations in Avi Controller:

/opt/avi/log/cc_agent_.log</code> — These logs are created for each Avi Cloud. All the operations being performed for the specific cloud are logged here.
/opt/avi/log/glog/cloudconnectorgo — All the cloudconnectorgo service specific logs are logged in these log files. Errors relating to RPC handling and other common infrastructure components are logged here.
/var/log/upstart/cloudconnectorgo.log — All the crashes/panics are logged in this log file.

Common Errors

The following are the common errors:

projects/project-id/zones/us-central1-b/machineTypes/n2-standard-4 was not found — Ensure that the instance_flavor added in the Service Engine group is available in all the zones configured in the Avi GCP cloud.

Refer to Available regions and zones to check flavor (machine-type) availability in GCP regions and zones.