Status
Current state: Under Discussion
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
The current Connect REST server only sets a few default HTTP response headers. It's missing many headers, including most headers related to security. The Connect REST server uses an embedded Jetty server as the Java HTTP server and Java Servlet container, so users have no way to configure HTTP response headers for Connect REST server. Many customers using Connect REST server are demanding some headers related to security in the HTTP response. Some examples of headers are X-XSS-Protection
, Content-Security-Policy
, Strict-Transport-Security
and X-Content-Type-Options
. Some resources need more protected than others due to security required. So we will allow site administrators to control which resources the user agent is allowed or is not allowed to load for given page. For this case, the sensitive resources need use Content-Security-Policy in response header. Some resources can only be accessed by HTTPS instead of HTTP, then Strict-Transport-Security response header (often abbreviated as HSTS) need be set to let a web site tell browsers that it should only be accessed using HTTPS. So that is why we provide feature to support configuring different headers for different resources.
Public Interfaces
There is no any changes on public interfaces. We follow same pattern like configuring different kafka listeners. We define a new property response.http.headers which define how many headers will be configured. The default value for response.http.headers is empty string which mean there is no any header configured for HTTP response. We define a new prefix "response.http.headers.{name}.", then followed by a set of properties which define rules for header. The {name} will be one defined in response.http.headers. The following section has detailed description.
Proposed Changes
Adding Properties
We will add a set of new properties in the org.apache.kafka.connect.runtime.WorkerConfig
class. It will allow the REST server administrator to configure headers based on their security policies. We borrow and take advantage of the Jetty HeaderFilter
class and use the same format of headerConfig
, includedPaths
, excludedPaths
, includedMimeTypes
, excludedMimeTypes
, includedHttpMethods
, and excludedHttpMethods
init parameters.
Description of Properties
Property Name | Type | Default | Importance | Description | Example for Value |
---|---|---|---|---|---|
response.http.headers | string | "" | medium | Defines names of headers which will be separated by comma. The name could be any string which uniquely identify header. | default, header1, header2 |
response.http.headers.{name}.header.config | string | "" | low | Define a set of HTTP headers for header defined by {name} which will be one of names defined in property response.http.headers. Detailed explanation see Detailed Explanation section. | set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0 |
response.http.headers.{name}. included.paths | string | "" | low | It is a comma separated values of included path specs applied to HTTP headers. See path spec rules section. | /connectors/connector1/topics/* |
response.http.headers.{name}.excluded.paths | string | "" | low | It is a comma separated values of excluded path specs applied to HTTP headers. See path spec rules section. | /connectors/connector1/status |
response.http.headers.{name}.included.mime.types | string | "" | low | It is a comma separated values of included mime types applied to HTTP headers | application/json |
response.http.headers.{name}.excluded.mime.types | string | "" | low | It is a comma separated values of excluded mime types applied to HTTP headers. | application/xml |
response.http.headers.{name}.included.http.methods | string | "" | low | It is a comma separated values of included http methods applied to HTTP headers | POST,PUT |
response.http.headers.{name}.excluded.http.methods | string | "" | low | It is a comma separated values of excluded http methods applied to HTTP headers | GET |
Detailed Explanation
response.http.headers.{name}.header.config
The name will be one of names defined in property response.http.headers
The format will be "[[action] [header]:[header value],..." which is a list of [action] [header]:[value] separated by comma ",". So it is a CSV of actions to perform on headers with the following syntax:
[action] [header name]: [header value],
[action] can be one of "set, add, setDate, or addDate" which specify an action will perform on header.
set
action is the same as thesetHeader
function inHttpServletResponse
, it will set a response header with the given name and value. If the header had already been set, the new value overwrites the previous one.add
action is the same as theaddHeader
function inHttpServletResponse
, it will add a new value to the header. Responses headers could have multiple values.setDate
action is the same as thesetDateHeader
function inHttpServletResponse
. It will set a HTTP header with a date value. Such as "setDate Expires: 31540000000
" which indicates the header will be expired approximately one year in the future.addDate
action is the same as theaddDateHeader
function inHttpServletResponse
. It will add a response header with the given name and date-value. Such as "addDate Last-Modified: 0
" which indicates the Last-Modified date is same as current system date.
[header name] specify name of header.
[header value] specify value for the header. We need to put double quotes around the value if the value contains commas because we use comma as separator for different headers.
Path Spec Rules:
- If the spec starts with
^
, the spec is assumed to be a regex based path spec and will match with normal Java regex rules. - If the spec starts with
/
, the spec is assumed to be a Servlet url-pattern rules path spec for either an exact match or prefix based match. - If the spec starts with
*.
, the spec is assumed to be a Servlet url-pattern rules path spec for a suffix based match. - All other syntaxes are unsupported.
Multiple Headers Configuration
We use Jetty HeaderFilter to implement HTTP response header configuration. We need support multiple rules applied to multiple different response headers.
We define a prefix response.http.headers.<name>. to allow multiple configurations, so that different paths have different headers. The "<name>" will be the name of header and that will be used to uniquely identify the header config which could only apply to different Servlet path or HTTP methods.
Example:
response.http.headers.default.header.config=set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0 response.http.headers.default.header.included.paths=/connectors/connector1/topics/* response.http.headers.default.excluded.paths=/connectors/connector1/status response.http.headers.default.included.mime.types=application/json response.http.headers.default.excluded.mime.types=application/xml response.http.headers.default.included.http.methods=POST,PUT response.http.headers.default.excluded.http.methods=GET response.http.headers.header1.header.config=set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0 response.http.headers.header1.header.included.paths=^/test2/0$ response.http.headers.header1.excluded.paths=^/test3/0$ response.http.headers.header1.included.mime.types=application/json response.http.headers.header1.excluded.mime.types=application/xml response.http.headers.header1.included.http.methods=POST,PUT response.http.headers.header1.excluded.http.methods=GET
Implementation
Implementation will use the Jetty HeaderFilter
class. We need to update org.apache.kafka.connect.runtime.rest.RestServer
class. During initialization the Connect REST server will read all header configurations from the property with prefix response.http.headers., then create a list of FilterHolder
with HeaderFilter
class and add the list of filter holders to the Servlet context handler based on the name of the filter. Implementation is similar to how we handle the header access.control.allow.origin
in the Connect REST server.
Pseudocode
public void configureHttpResponsHeaderFilters(ServletContextHandler context) {
Map<String, Map<String, String>> headersConfigs = extractHttpResponseHeaderConfig();
FilterHolder headerFilterHolder = null;
for (Map.Entry<String, Map<String, String>> entry : headersConfigs.entrySet()) {
headerFilterHolder = new FilterHolder(HeaderFilter.class);
headerFilterHolder.setName(entry.getKey());
Map<String, String> oneHeaderConfig = entry.getValue();
for (Map.Entry<String, String> oneHeader : oneHeaderConfig.entrySet()) {
switch (oneHeader.getKey().toUpperCase()) {
case "HEADER.CONFIG":
headerFilterHolder.setInitParameter("headerConfig", oneHeader.getValue());
break;
case "INCLUDED.PATHS":
headerFilterHolder.setInitParameter("includedPaths", oneHeader.getValue());
break;
case "EXCLUDED.PATHS":
headerFilterHolder.setInitParameter("excludedPaths", oneHeader.getValue());
break;
case "INCLUDED.MIME.TYPES":
headerFilterHolder.setInitParameter("includedMimeTypes", oneHeader.getValue());
break;
case "ENCLUDED.MIME.TYPES":
headerFilterHolder.setInitParameter("excludedMimeTypes", oneHeader.getValue());
break;
case "INCLUDED.HTTP.METHODS":
headerFilterHolder.setInitParameter("includedHttpMethods", oneHeader.getValue());
break;
case "ENCLUDED.HTTP.METHODS":
headerFilterHolder.setInitParameter("excludedHttpMethods", oneHeader.getValue());
break;
default:
}
}
context.addFilter(headerFilterHolder, "/*", EnumSet.of(DispatcherType.REQUEST));
}
}
References
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://www.eclipse.org/jetty/documentation/current/header-filter.html
https://www.eclipse.org/jetty/javadoc/9.4.24.v20191120/org/eclipse/jetty/servlets/HeaderFilter.html
Compatibility, Deprecation, and Migration Plan
Since we just add a new property and the default value for new property is empty string, existing use cases and behavior will be unaffected.
Rejected Alternatives
Another implementation would be writing a customized filter extension to intercept and set HTTP response headers. Ultimately the purpose of this KIP will allow users to set HTTP response headers, using this alternative make implementation much complex and doesn't gain any benefits.