Table of Contents |
---|
Status
Current state: [Under Under Discussion"]
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: here [Change the link from KAFKA-1 to your own ticket]
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Current The current Connect REST server only set sets a few default HTTP response headers, missed . It's missing many headers, specially including most headers relate related to security missed. The Connect REST server uses an embedded Jetty server as the Java HTTP server and Java Servlet container, so users have no way to configure HTTP response headers for Connect REST server. Many customers using Connect REST server are demanding some headers relate related to security set in the HTTP response. Some examples of headers are are X-XSS-Protection
, Content Content-Security-Policy
, Strict Strict-Transport-Security
and X and X-Content-Type-Options
.
Public Interfaces
There is no any changes on public interfaces. We define a new prefix "filter.name.<name>.", then followed by a set of properties which define rules for header. The following section has detailed description.
Proposed Changes
Adding Properties
We will add a set of new properties in the org.apache.kafka.connect.runtime.WorkerConfig
class. It will allow will allow the REST server administrator to configure headers based on their security policies. We borrow and take advantage of the Jetty HeaderFilter
class and use the same format of headerConfig
, includedPaths
, excludedPaths
, includedMimeTypes
, excludedMimeTypes excludedMimeTypes
, includedHttpMethods includedHttpMethods
, excludedHttpMethodsand excludedHttpMethods
init parameters.
Description of Properties
header.config
The format for response.http.headers
will be "[[action] [header]:[header value],..." which is a list of [action] [header]:[value] separated by comma ",". So it is a CSV of actions to perform on headers with the following syntax:
[action] [header name]: [header value],
[action] can be one of of "set, add, setDate, or addDate" which specify an action will perform on header.
set
action is the same as thesetHeader
function inHttpServletResponse
, it will set a response header with the given name and value. If the header had already been set, the new value overwrites the previous one.add
action is the same as theaddHeader
function inHttpServletResponse
, it will add a new value to the header. Responses headers could have multiple values.setDate
action is same as setDateHeader the same as thesetDateHeader
function inHttpServletResponse
. It will set a HTTP header need with a date value. Such as "setDate Expires: 31540000000
" which indicates the header will be expired approximately one year in the future.addDate
action is the same as addDateHeader theaddDateHeader
function inHttpServletResponse
. It will add a response header with the given name and date-value. Such as "addDate Last-Modified: 0
" which indicates the Last-Modified date is same as current system date.
[header name] specify name of header.
[header value] specify value for the header. We need to put double quotes around the value if the value contains commas due to because we use comma as separator for different headers.
Example:
Code Block | ||
---|---|---|
| ||
header.config= |
...
set X-Frame-Options: DENY, " |
...
add Cache-Control: no-cache, no-store, must-revalidate", |
...
setDate Expires: 31540000000, |
...
addDate Last-Modified: 0 |
...
included.
...
paths
It is optional. it is a comma separated values of included path specs applied to headers.config. See path spec rules section.
Example:header.includedpaths
Code Block | ||
---|---|---|
| ||
included.paths=^/test/0$ |
headerexcluded.
excludedathspaths
It is optional. it is a comma separated values of excluded path specs applied to headers.config. See path spec rules section.
Example:header.excludedpaths
Code Block | ||
---|---|---|
| ||
excluded.paths=^/test/0$ |
headerincluded.mime.
includedmimetypestypes
It is optional. it is a comma separated values of included mime types applied to headers.config.
Example:header.includedmimetypes
Code Block | ||
---|---|---|
| ||
included.mime.types=application/json |
header.excludedmimetypesexcluded.mime.types
It is optional. it is a comma separated values of excluded mime types applied to headers.config.
Example:header.excludedmimetypes
Code Block | ||
---|---|---|
| ||
excluded.mime.types=application/xml |
headerincluded.http.
includedhttpmethodsmethods
It is optional. it is a comma separated values of included http methods applied to headers.config.
Example:header.includedhttpmethods
Code Block | ||
---|---|---|
| ||
included.http.methods=POST,PUT |
header.ex
cludedhttpmethodscluded.http.methods
It is optional. it is a comma separated values of excluded http methods applied to headers.config.
Example:header.excludedhttpmethods
Code Block | ||
---|---|---|
| ||
excluded.http.methods=GET |
Path Spec Rules:
- If the spec starts with
^
, the spec is assumed to be a regex based path spec and will match with normal Java regex rules. - If the spec starts with
/
, the spec is assumed to be a Servlet url-pattern rules path spec for either an exact match or prefix based match. - If the spec starts with
*.
, the spec is assumed to be a Servlet url-pattern rules path spec for a suffix based match. - All other syntaxes are unsupported.
...
We use Jetty HeaderFilter to implement HTTP response header configuration. We need support multiple rules applied to multiple different response headers.
We define a prefix filterresponse.http.nameheaders.<name>. to allow multiple configurations, so that different paths have different headers.
Example:
Code Block |
---|
...
response.http. |
...
headers.header1.header.config= |
...
set X-Frame-Options: DENY, " |
...
add Cache-Control: no-cache, no-store, must-revalidate", |
...
setDate Expires: 31540000000, |
...
addDate Last-Modified: |
...
0 response.http.headers.header1.header.included. |
...
paths=^/test/0$ |
...
response.http.headers.header1. |
...
excluded. |
...
paths=^/test1/0$ |
...
response.http.headers.header1.included. |
...
mime. |
...
types=application/jsonheader |
...
response.http.headers.header1.excluded. |
...
mime. |
...
types=application/xml |
...
response.http.headers.header1.included. |
...
http. |
...
methods=POST,PUT |
...
response.http.headers.header1.excluded. |
...
http. |
...
methods=GET |
Implementation
Implementation will use the Jetty HeaderFilter
class. We need to update org.apache.kafka.connect.runtime.rest.RestServer
class. During initializing process initialization the Connect REST server will read all header configurations from the property with prefix filter.name
., then create a list of FilterHoder FilterHolder
with HeaderFilter
class and add the list of filter holders to the Servlet context handler based on the name of the filter. Implementation is similar as to how we handle the header access.control.allow.origin
in the Connect REST server.
Pseudocode
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
//Transfer WorkConfig to map which map filter name to a map which map name of init parameter to value of init parameter for the filter.
Map<String, Map<String, String>String>> headerFilterConfigs = extractHeaderFilterConfig(workerConfig);
FilterHolder headersFilterHolder = null;
for (String nameEntry<String, Map<String, String>> oneFilter : headerFilterConfigs.getkeySetgetEntrySet()) {
StringheadersFilterHolder filterName= = extract
String responseHeaders = config.getString("response.http.headers");
FilterHolder headersFilterHolder = new FilterHolder(HeaderFilter.class);
headersFilterHolder.setName("headerConfig");
headersFilterHolder.setInitParameter("headerConfig", responseHeaders);
context.addFilter(headersFilterHolder, "/*", EnumSet.of(DispatcherType.REQUEST));new FilterHolder(HeaderFilter.class);
context.addFilter(headersFilterHolder, "/*", EnumSet.of(DispatcherType.REQUEST));
headersFilterHolder.setName(oneFilter.getName());
Map<String, String> oneHeaderConfig = oneFilter.getValue());
oneHeaderConfig.forEach((k,v) -> {
switch (k.toUpperCase()) {
"HEADER.CONFIG":
headersFilterHolder.setInitParameter("headerConfig", v);
break;
"INCLUDEED.PATHS":
headersFilterHolder.setInitParameter("includedPaths", v);
break;
"EXCLUDEED.PATHS":
headersFilterHolder.setInitParameter("excludedPaths", v);
break;
"INCLUDED.MIMETYPES":
headersFilterHolder.setInitParameter("includedMimeTypes", v);
break;
"ENCLUDED.MIMETYPES":
headersFilterHolder.setInitParameter("excludedMimeTypes", v);
break;
"INCLUDED.HTTP.METHODS":
headersFilterHolder.setInitParameter("includedHttpMeethods", v);
break;
"ENCLUDED.HTTP.METHODS":
headersFilterHolder.setInitParameter("excludedHttpMeethods", v);
break;
}
});
}
References
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://www.eclipse.org/jetty/documentation/current/header-filter.html
https://www.eclipse.org/jetty/javadoc/9.4.24.v20191120/org/eclipse/jetty/servlets/HeaderFilter.html
Compatibility, Deprecation, and Migration Plan
Since we just add a new property and the default value for new property is empty string, existing use cases and behavior will be unaffected.
Rejected Alternatives
Another implementation would be writing a customized filter extension to intercept and set HTTP response headers. Ultimately the purpose of this KIP will allow users to set HTTP response headers, using this alternative make implementation much complex and doesn't gain any benefits.
...