Preserve Client IP for NSX-T Overlay

Overview

Load balancing solutions for some of the applications (particularly in Layer 4 mode) require Client IP address to be presented as the source IP address when the packet lands on the backend pool server member. NSX Advanced Load Balancer (formerly known as Avi Vantage) supports this functionality as Preserve Client IP.

This solution is deployed generally by making the default gateway of the backend server to point to a floating IP. The floating IP is hosted on the active Service Engine’s backend interface. However, in this mode, the servers’ default gateway has to be modified/updated to point to the floating IP. The NSX-T overlay deployment model being in layer 3 mode has complications for preserving the client IP.

Service Insertion Framework

VMware NSX-T provides the service insertion, framework which has the ability to redirect traffic. The NSX-T service insertion framework has been utilized to achieve the return traffic from backend server to the floating IP of Active Service Engine without needing to change the default gateway of the backend server.

Preserve Client IP

Configuring Preserve Client IP

Preserve client IP for NSX-T overlay deployments with respect to configuration at virtual service, Service Engine group, and network service will remain the same as per other supported clouds. There are additional pre-requisites and limitations which are discussed in this article.

Pre-requisites

  1. The Service Engine HA mode should be Legacy HA mode (Active/Standby).

    Preserve Client IP

  2. The NSX-T user for configuring NSX-T cloud should have additional permissions of Netx Partner Admin and Security Admin for the preserve client IP functionality apart from the Network Admin requirement for other use cases.

    Preserve Client IP

  3. Set URPF Mode to None for the VIP data segments in which the preserve client IP feature will be enabled.

    Preserve Client IP

    Note: If uRPF Mode is not set to None, then an event with error will be generated, though the status of the virtual service is up, as shown below:

    Preserve Client IP

  4. Configure the virtual service (for which preserve client IP has to configured), and pool server as Network Security Groups. Individual IP address or range, DNS Name, and IP group are not allowed.

  5. Configure the Floating Interface IP(FIP) in the network service and attach it to the appropriate Avi Service Engine group, VRF, and cloud reference which is hosting the virtual service that require the preserve client IP feature.

    
 [admin:10-170-67-140]: > show nsxt segment London_ALB_DATA_SEGMENT
 +-------------------+-----------------------------------------+
 | Field             | Value                                   |
 +-------------------+-----------------------------------------+
 | uuid              | segmentruntime-ab75a213243b             |
 | segment_id        | /infra/segments/London_ALB_DATA_SEGMENT |
 | name              | London_ALB_DATA_SEGMENT                 |
 | subnet            | 192.168.100.0/24                        |
 | dhcp_enabled      | True                                    |
 | nw_ref            | London_ALB_DATA_SEGMENT                 |
 | nw_name           | London_ALB_DATA_SEGMENT                 |
 | vrf_context_ref   | London_Tier1Gateway1                    |
 | tier1_id          | /infra/tier-1s/London_Tier1Gateway1     |
 | opaque_network_id | 9cbf6823-3bb8-4935-a675-e07872e7935f    |
 | segment_gw        | 192.168.100.1/24                        |
 | dhcp_ranges[1]    | 192.168.100.170-192.168.100.180         |
 | segname           | London_ALB_DATA_SEGMENT                 |
 | tenant_ref        | admin                                   |
 | cloud_ref         | nsxt_cloud_overlay                      |
 +-------------------+-----------------------------------------+

    From the configuration, note that the DHCP range is 192.168.100.170-192.168.100.180 Preserve Client IP

    The preserve client IP is configured as shown below:

    
 [admin:10-170-67-140]: > show networkservice nsxt_preserveIP_ns
 +--------------------------------+-----------------------------------------------------+
 | Field                          | Value                                               |
 +--------------------------------+-----------------------------------------------------+
 | uuid                           | networkservice-55e0f033-02e1-4a6b-99a1-b3a0f674f380 |
 | name                           | nsxt_preserveIP_ns                                  |
 | se_group_ref                   | Default-Group                                       |
 | vrf_ref                        | London_Tier1Gateway1                                |
 | service_type                   | ROUTING_SERVICE                                     |
 | routing_service                |                                                     |
 |   enable_routing               | False                                               |
 |   routing_by_linux_ipstack     | False                                               |
 |   floating_intf_ip[1]          | 192.168.100.150                                     |
 |   enable_vmac                  | False                                               |
 |   enable_vip_on_all_interfaces | True                                                |
 |   advertise_backend_networks   | False                                               |
 |   graceful_restart             | False                                               |
 |   enable_auto_gateway          | False                                               |
 | tenant_ref                     | admin                                               |
 | cloud_ref                      | nsxt_cloud_overlay                                  |
 +--------------------------------+-----------------------------------------------------+

    The floating IP is 192.168.100.150, which is outside the DHCP range:

    Preserve Client IP

    Notes:

    • Ensure that the FIP is from the same segment where the Service Engine’s data segment is configured
    • Ensure that the FIP does not fall in the DHCP/static range of the data segment

Configuring Preserve Client IP (L7 Virtual Service)

To configure preserve client IP for the L7 virtual service,

  1. In the HTTP profile, enable Preserve Client IP as shown below:

    Preserve Client IP

    Note: Connection Multiplex cannot be enabled while using Preserve Client IP.

  2. Bind the Application Profile to the L7 virtual service as shown below:

    Preserve Client IP

Configuring Preserve Client IP (L4 Virtual Service)

To configure preserve client IP for the L4 virtual service,

  1. In the L4 Application Profile, enable Preserve Client IP as shown below:

    Preserve Client IP

  2. Bind the L4 Application Profile to the L7 virtual service as shown below:

    Preserve Client IP

To configure a non-default customized Tier-1 locale-service for hosting the redirect service, configure the field locale_service.

Note: This option is available in the CLI (Configure cloud > nsxt_configuration > data_network_config > tier1_segment_config > manual > tier1_lrs).


[admin:10-170-67-140]: cloud:nsxt_configuration:data_network_config:tier1_segment_config:manual:tier1_lrs>
[admin:10-170-67-140]: cloud:nsxt_configuration:data_network_config:tier1_segment_config:manual:tier1_lrs> locale_service /infra/tier-1s/London_Tier1Gateway1/locale-services/London_Tier1LocalServices-1
[admin:10-170-67-140]: cloud:nsxt_configuration:data_network_config:tier1_segment_config:manual:tier1_lrs> where
Tenant: admin
Cloud: Default-Cloud
+----------------+---------------------------------------------------------------------------------+
| Field          | Value                                                                           |
+----------------+---------------------------------------------------------------------------------+
| tier1_lr_id    | London_Tier1Gateway1                                                            |
| segment_id     | /infra/segments/London_ALB_DATA_SEGMENT                                         |
| locale_service | /infra/tier-1s/London_Tier1Gateway1/locale-services/London_Tier1LocalServices-1 |
+----------------+---------------------------------------------------------------------------------+
[admin:10-170-67-140]: cloud:nsxt_configuration:data_network_config:tier1_segment_config:manual:tier1_lrs>

Preserve Client IP

Caveats

  • Only Inline Topologies are supported for preserving the client IP. One-Arm Topology is not supported, similar to legacy NSX-T load balancer

  • IPv6 addressing is not supported when using the preserve client IP feature

  • If the same application NSG is used in more than one virtual services (across SE group/across the cloud), each pool must have a different service port.

  • The Tier-1, hosting the VIP data segment,should have the NSX Edge cluster selected as redirect service framework requires Tier-1’s Service Router component

    Note: Ensure that the sizing of the Edge cluster considers the traffic needs of the preserve client IP-enabled virtual service

  • Updating the redirect rule (change of port number of the pool or changing the FIP IP in the SE group) will cause traffic loss for around 90 seconds

  • Using the same pool server and port for preserve client IP and non-preserve client IP virtual service across SE groups causes the non-preserve client IP virtual service to be marked down due to health monitor traffic failing

  • Distributing load & auto-redistributing load properties of SE-group are not supported

Document Revision History

Date Change Summary
April 11, 2022 Published the feature KB for for Preseve client IP un NSX-T Overlay