Masking and Removing Personally Identifiable Information (PII) in Applications Logs on Avi Vantage
Overview
Avi Vantage collects different types of logs for troubleshooting various performance or outage issues, end-user experience, and success of any application. Avi Controller collects HTTP request header and response header information while establishing connections between the incoming client requests and the back-end servers. Avi Vantage supports removing or masking Personally Identifiable Information (PII) in HTTP request headers and response header fields in application logs. The actual value of the required HTTP header field can be removed or masked to X. HTTP request header is masked in the following type of requests:
- HTTP header received from the client.
- HTTP header sent to the back-end server.
The following type of requests are considered as HTTP response header on Avi Vantage:
- Responses coming from the back-end server
- Response header sent to the client
Note: Any of the HTTP headers in the application logs on Avi Vantage can be masked or removed. The feature is not just only limited to masking off the header, but also both the header name and their corresponding value from the application logs can be removed too.
Use case
Exposure of user-related sensitive information, for example, HTTP cookie, authorization information , e.t.c is avoided. Any security threat that may happen because of the exposed user or server-related information is avoided too.
Hiding PII in Information Logs using Avi CLI
Based on the requirement, any of the HTTP request headers and response headers can be masked or removed in the virtual service logs on Avi Vantage.
In the configuration snippet mentioned below, HTTP headers are selected by using the match_str: "HTTP-Header-name" command and their corresponding actions (to mask the original value ) are set by using the LOG_FIELD_MASKOFF command in the analytics profile configuration of a virtual service:
Refer to the following configuration snippet for reference:
analyticsprofile_object {
uuid: "l7-analytics-profile-3"
name: "l7-analytics-profile-3"
tenant_uuid: "admin"
sensitive_log_profile {
header_field_rules {
index: 1
name: "hdr_rule_1"
enabled: true
match {
match_criteria: EQUALS
string_group_uuids: "stringgroup-2"
}
action: LOG_FIELD_MASKOFF
}
header_field_rules {
index: 2
name: "hdr_rule_2"
enabled: true
match {
match_criteria: CONTAINS
match_str: "X-Forwarded-For"
}
action: LOG_FIELD_MASKOFF
}
}
}
stringgroup_object {
uuid: "stringgroup-2"
name: "stringgroup-2"
kv {
key: "Authorization"
}
kv {
key: "Cookie"
}
kv {
key: "Set-Cookie"
}
tenant_uuid: "admin"
}
Changes in HTTP request and HTTP response header:
The following screenshots exhibit the masked value of various HTTP headers in the logs on Avi Vantage.
Changes in HTTP request header
Details of an HTTP request header before the change in the analytic profile:
Details of an HTTP request header after the change in the analytic profile:
In the example mentioned above, the value for the following HTTP request headers are masked to X:
- X-Forwarded-For
- Authorization HTTP header
- Cookie field
Changes in HTTP response header
Details of an HTTP response header in the application logs before the change in the analytic profile:
Details of an HTTP response header in the application logs after the change in the analytic profile:
In the example mentioned above, the value for the HTTP response header, Set-Cookie is masked to X in the application logs.
Hiding PII in Information Logs using Avi UI
Starting with Avi Vantage release 18.1.4, Sensitive Log Profile option is available on Avi UI. This feature is used to hide sensitive information in the application logs for the selected virtual service.
Note: Regular expressions (Regex) are not supported for rules used for Sensitive Log Profile.
Following are the steps to add a new sensitive log profile or edit the existing profile.
-
Navigate to Applications > Virtual Services, select the desired virtual service, and click on the edit icon.
-
Navigate to the Analytics tab, click on the edit option for Analytic Profile as shown below.
-
Scroll down to the Sensitive Log Information section to add a new rule or edit the existing rules for capturing logs for the selected virtual service.
-
To add a new rule, click on the create option (the plus sign).
To edit the existing rule, click on the edit option as shown below.
-
Provide the desired name, matching rule, and the desired action for the match.
-
For Enable Action, the following actions are available:
- Remove the matched fields from the logs
- Mask the matched fields from the logs
Masking URI
Starting with NSX Load Balancer 21.1.3, you can mask URI query parameters in logs using the field uri_query_field_rules.
The values are masked if the keys in match_str[1] match the defined match_criteria.
A sample configuration is shown below:
[admin]: > configure analyticsprofile example_profile
[admin]: analyticsprofile> sensitive_log_profile
[admin]: analyticsprofile:sensitive_log_profile> uri_query_field_rules
[admin]: analyticsprofile:sensitive_log_profile:uri_query_field_rules> enabled
[admin]: analyticsprofile:sensitive_log_profile:uri_query_field_rules> name "example_mask_rule"
[admin]: analyticsprofile:sensitive_log_profile:uri_query_field_rules> action log_field_maskoff
[admin]: analyticsprofile:sensitive_log_profile:uri_query_field_rules> match
[admin]: analyticsprofile:sensitive_log_profile:uri_query_field_rules:match> match_str "password"
[admin]: analyticsprofile:sensitive_log_profile:uri_query_field_rules:match> match_criteria contains
With this configuration, if you send a request to the virtual service including a query parameter that matches the match rule, the URI will be masked, for example curl "VIP/?passwordfield=somepassword" in the application logs as shown below:
Masking WAF Rules
When the Core Rule Set (CRS) is matching with REQUEST_COOKIES, this can be specified as the match element in the waf_field_rules in the Sensitive Log Profile (Templates > Profiles > Analytics).
For example, a common analytics profile can be maintained for all WAF virtual services that include the rules that hide sensitive matches like Cookies, Args:Password, etc.
Configure Masking Via CLI
Configuring the analytics profile for masking WAF rules is as shown below:
configure analyticsprofile 'some-common-analytics-profile'
sensitive_log_profile
name mask-waf-match-cookies
match match_criteria equals match_str REQUEST_COOKIES
action log_field_maskoff
enabled true
The sensitive_log_profile will appear as below:
+---------------------------------------+--------------------------------------------+
| Field | Value |
+---------------------------------------+--------------------------------------------+
|sensitive_log_profile | |
|header_field_rules[1] | |
|name |1 |
|enabled |True |
|match | |
|match_criteria |Equals |
|string_group_refs[1] |cookie-group |
|action |LOG_FIELD_MASKOFF |
|waf_field_rules[1] | |
|name |mask-waf-match-cookies |
|enabled |True |
|match | |
|match_criteria |CONTAINS |
|match_str[1] |REQUEST_Cookies |
|action |LOG_FIELD_MASKOFF |
+---------------------------------------+--------------------------------------------+
The profile is masking the headers from App Log (non-WAF) as well as the WAF matches.
Note: Different set of rules have to be defined for masking WAF and non-WAF virtual services.
The test request and App Log before and after applying the policy are as shown below:
curl -vvv -k -H 'Host: foobar' -H 'Cookie: k=*script*alert(42);*script*' VIP
Before applying the sensitive policy, the cookies are seen in the WAF match as well as App Log Headers
After applying the sensitive policy, the cookie is masked in both WAF Log and App Log Headers.
