ICAP

Overview

ICAP (Internet Content Adaptation Protocol) is a lightweight HTTP-like protocol to transport HTTP messages to 3rd party services. The server runs its transformation service on messages and sends back responses to the client, usually with modified messages. For more information on ICAP, refer to RFC3507.

ICAP is supported for HTTP requests processing through NSX Advanced Load Balancer. With the implementation of the ICAP client functionality within the NSX Advanced Load Balancer, the following use-cases are supported:

Antivirus scanning - via 3rd party antivirus scan engine
Content sanitization - via 3rd party content sanitization service
Other request modification options via ICAP services, for example, URL filtering

Starting with NSX Advanced Load Balancer version 21.1.3, ICAPs is supported.

Configuring NSX Advanced Load Balancer for ICAP

NSX Advanced Load Balancer, as an ICAP client, supports the followings:

Preview functionality
Streaming of payload
Content rewrite

The followings are the main configuration components for enabling ICAP for a virtual service on an NSX Advanced Load Balancer:

Configuring an ICAP pool group
Configuring an ICAP profile (attached to the virtual service)
Configuring an HTTP Policy for the virtual service with the action set as Enable ICAP
Associating the ICAP profile to the virtual service
Configuring HTTP security policy for ICAP

Configuring ICAP Pool Group
Navigate to Application > Pool Group, create a pool group, as shown below. The field for the Fail Action under the Pool Group Failure Settings needs to be empty.
Configuring ICAP Pool
Navigate to create an ICAP pool, as shown below. Configure the default port as 1344. Multiple Servers can be added as Pool members.

Configuring ICAP Profile

Refer to the following table for the various attributes used in the ICAP profile configuration.

Config Item	Description	Example
Name	Name of this profile	ICAP-APPX
Cloud	Defines, which cloud object this is associated with	Default-Cloud
Pool Group	Pool group of all ICAP server pools	ICAP-Pool-Group
Vendor	Vendor specific configuration if a vendor is supported	OPSWAT Generic-ICAP
Service URL	ICAP Server service URL	When using OPSWAT: /OMSScanReq-AV
Request Buffer Size	Maximum buffer size for request body	Default: 51200 (50 MB)
Enable ICAP Preview	Enable ICAP Preview functionality, where the ICAP server can make decisions by examining the preview size payload	Default: Enabled (Boolean)
Preview Size	Payload size for ICAP preview	Default: 5000 (5 MB)
Response Timeout	When this threshold is hit, the request will be handled as an error and the failure action will be run	Default: 60000 (60 seconds)
Slow Response warning Threshold	When this threshold is hit, the request will cause a significant log entry, but will still be served	Default: 10000 (10 seconds)
Actions
Failure Action	Handling of error with ICAP server. If failed closed, a 503 will be sent if an error is occurring.	Fail Closed/ Fail Open
Large Upload Failure Action	Handling of size exceeded error. If fail closed, a 413 will be sent.	Fail Closed/ Fail Open

Navigate to Virtual Service > Edit > ICAP Profile or Templates > Profiles > ICAP Profile to create an ICAP profile.

Navigate to the Application > Virtual Service, select the required virtual service, and select the ICAP profile (created in the previous step).
Creating HTTP Security Policy
Create a security policy to define the rules based on which the ICAP scanning should be performed. Navigate to Application > Virtual Service, select the desired virtual service, and click on Edit. Select Policies > HTTP Security, and create a new rule with the following options:
- Select a match criteria for the ICAP requests
- Select Enable ICAP as the action
Note: The rule name configured in this step will appear in the logs, so it is recommended to make it self-explanatory for ease of troubleshooting.
Save the virtual service configuration.

With these steps, the ICAP configuration for the virtual service is complete. Incoming requests on the virtual service that match the rule or the match criteria of the HTTP security policy will use ICAP.

NSX Advanced Load Balancer supports the following ICAP servers (3rd party AV-Malware/CDR vendors):

OPSWAT
MetaDefender ICAP Server (with MetaDefender Core)

To set up an OPWSWAT server for ICAP scanning, refer to the OPSWAT documentation.

Limitations

The followings are the limitations for ICAP support on NSX Advanced Load Balancer:

ICAP is not supported for HTTP/2 virtual services.
ICAP client does not work in the response context.

ICAP Support for NSX Defender

Starting with NSX Advanced Load Balancer release 21.1.1, ICAP support is available for NSX Defender’s server for preventing malicious file uploads. When compared with OPSWAT, there are minor difference in the method using which the NSX Defender sends file back to the NSX Advanced Load Balancer. This section covers the followings:

NSX Defender ICAP configuration
NSX Advanced Load Balancer integration with NSX Defender
Required visibility changes for NSX Defender reported information.

Configuring NSX Defender for ICAP

Select the ENABLED option for the following knobs on the UI:

ICAP Server
INLINE ANALYSIS

Under the Blocking pages section enable the follwoings:

BLOCKED PAGE DETAILS
X-LASTLINE HEADER
LASTLINE LOGO

The followings are the blocking types available on the NSX Defender. For more information, refer to the NSX Defender documentation.

PASSIVE – No blocking is attempted on this type of file, but any relevant content will be analyzed.
SENSOR-KNOWN – Block all artifacts known to be malicious by the Sensor (listed in its local cache). This method offers the lowest levels of protection but ensures minimal lag.
MANAGER-KNOWN – Block all artifacts known to be malicious by the NSX Defender Manager. These data are listed in the Manager cache and shared across all managed appliances.
FULL – — This mode allows the proxy to stall an ICAP request for as long as necessary to provide a verdict on the file, within the limits set by the ICAP timeout. Depending on the client implementation, this may cause the transaction to appear as unresponsive for long periods of time (in the order of minutes in some cases).
This blocking mode is particularly suitable for the integration with third party proxies that implement mechanisms to improve the user experience. Such mechanisms may include data trickling or “patience pages”, providing feedback to the user.
FULL WITH FEEDBACK – — This mode will generate “patience pages” that provide feedback to the user on the analysis progress. These mechanisms have been tested exclusively with the squid proxy. They may lead to unwanted side-effects when using third-party proxies, which may implement caching mechanisms that disrupt the NSX Defender operation. Such third party proxies often implement their own mechanisms to improve user experience, and therefore may perform better with the Full blocking mode.

Configuring NSX Advanced Load Balancer for NSX Defender

The following configuration ICAP server specific options are required to enable ICAP scanning using NSX Defender.

Service URI – This needs to be set to /lastline to use NSX Defender service.
ICAP Pool – ICAP pool needs to point to NSX Defender ICAP server:port.
Status URL – Only applicable to NSX Defender and has default value of https://user.lastline.com/portal#/analyst/task/$uuid/overview

Rest all the configuration options are generic and not tied to any particular ICAP server.

Visibility Changes for NSX Defender Reported Information

X-Lastline HTTP headers Pages analyzed by the ICAP instance may contain additional information on the analysis status by means of additional HTTP headers. The presence of such headers can be disabled from the ICAP configuration.

X-Lastline-Status – Provides information on the state of the object at the time of analysis. The following values are possible:
- new — The specific file hash has not been recently analyzed by NSX Defender and a score is not currently available.
- known — The specific file is known and a score is associated to it.
- blacklist — The contacted remote endpoint has low reputation.
- timeout — The process reached its timeout while waiting for the analysis of the file.
- error — An error is preventing the analysis of the file.
X-Lastline-Score — The score currently associated with the file, if known, expressed as a value between 0 and 100.
X-Lastline-Task — The NSX Defender task UUID associated with the analysis of the file. It is possible to use this UUID in order to access the analysis details from the NSX Defender Portal/Manager Web UI. The following is the REST API to access information about any upload using UUID:
*https://user.lastline.com/portal#/analyst/task//overview*

Following is the snapshot of Analysis overview based on the task UUID:

ICAP Response Header

NSX Defender may also send following ICAP headers as part of ICAP response as per the ICAP extensions draft.

X-Infection-Found:Type=0;Resolution=1;Threat=LastlineArtifact(score=XX;md5=;uuid=)
X-Virus-ID: LastlineArtifact(score=100;md5=;task_uuid=)

Logs and Troubleshooting

This section discusses the various logs and troubleshooting options available for ICAP on NSX Advanced Load Balancer. The NSX Advanced Load Balancer UI and NSX Advanced Load Balancer CLI can be used to check logs and error messages for analytics and troubleshooting.

Log for the requests that were handled by the ICAP server has an icap_log section populated.

If the ICAP server blocks or modifies a request, the consequent log entry is significant. The following example shows details of the available logs on NSX Advanced Load Balancer. As shown under the Response Information, the overall request is blocked, and a 403 response code is sent back to the client.

The following log exhibits ICAP scan detects an infection (JSON log file):


"icap_log": {
    "action": "ICAP_BLOCKED",
    "request_logs": [
        {
        "icap_response_code": 200,
        "icap_method": "ICAP_METHOD_REQMOD",
        "http_response_code": 403,
        "http_method": "HTTP_METHOD_POST",
        "icap_absolute_uri": "icap://100.64.3.15:1344/OMSScanReq-AV ",
        "complete_body_sent": true,
        "pool_name": {
        "val": "ICAP-POOL-GROUP",
        "crc32": 1799851903
        },
        "pool_uuid": "poolgroup-c7dd3b93-60c1-4190-b6d6-26c22d55dc30",
        "latency": "1275",
        "icap_headers_sent_to_server": "Host: 100.64.3.15:1344\r\nConnection: close\r\nPreview: 653\r\nAllow: 204\r\nEncapsulated: req-hdr=0, req-body=661\r\n",
        "icap_headers_received_from_server": "Date: Thu, 19 Nov 2020 13:55:00 G11T\r\nServer: Metadefender Core V4\r\nISTag: \"001605794100\"\r\nX-ICAP-Profile: File process\r\nX-Response-Info: Blocked\r\nX-Response-Desc: Infected\r\nX-Blocked-Reason: Infected\r\nX-Infection-Found: Type=0",
        "action": "ICAP_BLOCKED",
        "reason": "Infected",
        "threat_id": "EICAR-Test-File (not a virus)"
        }]
},

The following is the log entry when the ICAP server modifies the ICAP request:
The following log shows that the ICAP scan is performed successfully. The action field for the icap_log exhibits the value as ICAP_PASSED.


{"icap_log": 
    {"action": "ICAP_PASSED", "request_logs": 
        [{
          "icap_response_code": 204, 
          "icap_method": "ICAP_METHOD_REQMOD",
          "http_method": "HTTP_METHOD_POST",
          "icap_absolute_uri": 
          "icap://100.64.3.15:1344/OMSScanReq-AV ", 
          "complete_body_sent": true, 
          "pool_name": {"val": "ICAP-POOL-GROUP", "crc32": 1799851903}, 
          "pool_uuid": "poolgroup-c7dd3b93-60c1-4190-b6d6-26c22d55dc30", 
          "latency": "456", 
          "icap_headers_sent_to_server": "Host: 100.64.3.15:1344\r\nConnection: close\r\nPreview: 0\r\nAllow: 204\r\nEncapsulated: req-hdr=0, null-body=661\r\n", 
          "icap_headers_received_from_server": "Date: Wed, 18 Nov 2020 12:54:06 G11T\r\nServer: Metadefender Core V4\r\nISTag: \"000000000096\"\r\nX-Response-Info: Allowed\r\nEncapsulated: null-body=0\r\n", "action": "ICAP_PASSED"}]}

The log entries will show the action for icap_log as ICAP_DISABLED if ICAP feature is not enabled.

 
"icap_log": {"action": "ICAP_DISABLED"}

Log Analytics

When ICAP is enabled, the log analytics on NSX Advanced Load Balancer provides an additional overview. All data items are clickable and allow the quick addition of filters for a detailed log view.

Troubleshooting

ICAP Server Connection Failed
The following example shows a log error message for a failed ICAP server connection. The ICAP Error is logged against as the Significance field. To solve this issue, check the direct connectivity from the Service Engines to the ICAP servers.

ICAP Server Error The following example shows the ICAP Rrequest is blockedmisconfiguration of the ICAP server will exhibit the action for the ICAP log as ICAP_BLOCKED. The reason for the action is No security rule matched as available in the ICAP header.


"icap_log":
    {"action": "ICAP_BLOCKED", 
    "request_logs":
      [{
        "icap_response_code": 200, 
        "icap_method": "ICAP_METHOD_REQMOD", 
        "http_response_code": 403, 
        "http_method": "HTTP_METHOD_POST", 
        "icap_absolute_uri": "icap://100.64.3.15:1344/OMSScanReq-AV ", 
        "complete_body_sent": true, "pool_name": {"val": "ICAP-POOL-GROUP", "crc32": 1799851903}, "pool_uuid": "poolgroup-c7dd3b93-60c1-4190-b6d6-26c22d55dc30", "latency": "17", 
        "icap_headers_sent_to_server": "Host: 100.64.3.15:1344\r\nConnection: close\r\nPreview: 0\r\nAllow: 204\r\nEncapsulated: req-hdr=0, null-body=661\r\n", 
        "icap_headers_received_from_server": "Date: Thu, 19 Nov 2020 13:25:15 G11T\r\nServer: Metadefender Core V4\r\nISTag: \"001605792300\"\r\nX-Response-Info: Blocked\r\nX-Response-Desc: No security rule matched\r\nEncapsulated: res-hdr=0, res-body=91\r\n", "action": "ICAP_BLOCKED"}]}

To solve this issue, refer to the documentation of the ICAP server used for the deployment.

ICAPs

Starting with NSX Advanced Load Balancer version 21.1.3, ICAPs is supported. ICAP traffic can now be encrypted using SSL.

The following are the configuration components for enabling ICAPs:

To configure ICAPs on NSX Defender, enable Secure ICAP in Proxy configurations as shown below:
To configure ICAPs on OPSWAT, click here.
In NSX Advanced Load Balancer, when configuring a pool for ICAPs, ensure SSL is enabled in the Pool, that is referred in the ICAP profile (has IPs of ICAP servers) and configure the default port as 11344 as shown below:

Starting with Avi Vantage version 21.1.3, the ICAP supports HTTP2 traffic to the virtual service. If the virtual service has HTTP2 enabled for any port, and ICAP is configured, then the HTTP2 traffic will be subjected to the ICAP server.

Document Revision History

Date	Change Summary
December 20, 2021	Added ICAPs section for 21.1.3