Status
Current state: Under Discussion
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
The current Connect REST server only sets a few default HTTP response headers. It's missing many headers, including most headers related to security. The Connect REST server uses an embedded Jetty server as the Java HTTP server and Java Servlet container, so users have no way to configure HTTP response headers for Connect REST server. Many customers using Connect REST server are demanding some headers related to security in the HTTP response. Some examples of headers are X-XSS-Protection
, Content-Security-Policy
, Strict-Transport-Security
and X-Content-Type-Options
.
Public Interfaces
There is no any changes on public interfaces. We define a new prefix "response.http.headers.<name>.", then followed by a set of properties which define rules for header. The following section has detailed description.
Proposed Changes
Adding Properties
We will add a set of new properties in the org.apache.kafka.connect.runtime.WorkerConfig
class. It will allow the REST server administrator to configure headers based on their security policies. We borrow and take advantage of the Jetty HeaderFilter
class and use the same format of headerConfig
, includedPaths
, excludedPaths
, includedMimeTypes
, excludedMimeTypes
, includedHttpMethods
, and excludedHttpMethods
init parameters.
Description of Properties
header.config
The format for response.http.headers
will be "[[action] [header]:[header value],..." which is a list of [action] [header]:[value] separated by comma ",". So it is a CSV of actions to perform on headers with the following syntax:
[action] [header name]: [header value],
[action] can be one of "set, add, setDate, or addDate" which specify an action will perform on header.
set
action is the same as thesetHeader
function inHttpServletResponse
, it will set a response header with the given name and value. If the header had already been set, the new value overwrites the previous one.add
action is the same as theaddHeader
function inHttpServletResponse
, it will add a new value to the header. Responses headers could have multiple values.setDate
action is the same as thesetDateHeader
function inHttpServletResponse
. It will set a HTTP header with a date value. Such as "setDate Expires: 31540000000
" which indicates the header will be expired approximately one year in the future.addDate
action is the same as theaddDateHeader
function inHttpServletResponse
. It will add a response header with the given name and date-value. Such as "addDate Last-Modified: 0
" which indicates the Last-Modified date is same as current system date.
[header name] specify name of header.
[header value] specify value for the header. We need to put double quotes around the value if the value contains commas because we use comma as separator for different headers.
Example:
header.config=set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0
included.paths
It is optional. it is a comma separated values of included path specs applied to headers.config. See path spec rules section.
Example:
included.paths=^/test/0$
excluded.paths
It is optional. it is a comma separated values of excluded path specs applied to headers.config. See path spec rules section.
Example:
excluded.paths=^/test/0$
included.mime.types
It is optional. it is a comma separated values of included mime types applied to headers.config.
Example:
included.mime.types=application/json
excluded.mime.types
It is optional. it is a comma separated values of excluded mime types applied to headers.config.
Example:
excluded.mime.types=application/xml
included.http.methods
It is optional. it is a comma separated values of included http methods applied to headers.config.
Example:
included.http.methods=POST,PUT
header.excluded.http.methods
It is optional. it is a comma separated values of excluded http methods applied to headers.config.
Example:
excluded.http.methods=GET
Path Spec Rules:
- If the spec starts with
^
, the spec is assumed to be a regex based path spec and will match with normal Java regex rules. - If the spec starts with
/
, the spec is assumed to be a Servlet url-pattern rules path spec for either an exact match or prefix based match. - If the spec starts with
*.
, the spec is assumed to be a Servlet url-pattern rules path spec for a suffix based match. - All other syntaxes are unsupported.
Multiple Headers Configuration
We use Jetty HeaderFilter to implement HTTP response header configuration. We need support multiple rules applied to multiple different response headers.
We define a prefix response.http.headers.<name>. to allow multiple configurations, so that different paths have different headers.
Example:
response.http.headers.header1.header.config=set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0 response.http.headers.header1.header.included.paths=^/test/0$ response.http.headers.header1.excluded.paths=^/test1/0$ response.http.headers.header1.included.mime.types=application/jsonheader response.http.headers.header1.excluded.mime.types=application/xml response.http.headers.header1.included.http.methods=POST,PUT response.http.headers.header1.excluded.http.methods=GET
Implementation
Implementation will use the Jetty HeaderFilter
class. We need to update org.apache.kafka.connect.runtime.rest.RestServer
class. During initialization the Connect REST server will read all header configurations from the property with prefix response.http.headers., then create a list of FilterHolder
with HeaderFilter
class and add the list of filter holders to the Servlet context handler based on the name of the filter. Implementation is similar to how we handle the header access.control.allow.origin
in the Connect REST server.
Pseudocode
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
//Transfer WorkConfig to map which map filter name to a map which map name of init parameter to value of init parameter for the filter.
Map<String, Map<String, String>> headerFilterConfigs = extractHeaderFilterConfig(workerConfig);
FilterHolder headersFilterHolder = null;
for (Entry<String, Map<String, String>> oneFilter : headerFilterConfigs.getEntrySet()) {
headersFilterHolder = new FilterHolder(HeaderFilter.class);
context.addFilter(headersFilterHolder, "/*", EnumSet.of(DispatcherType.REQUEST));
headersFilterHolder.setName(oneFilter.getName());
Map<String, String> oneHeaderConfig = oneFilter.getValue());
oneHeaderConfig.forEach((k,v) -> {
switch (k.toUpperCase()) {
"HEADER.CONFIG":
headersFilterHolder.setInitParameter("headerConfig", v);
break;
"INCLUDEED.PATHS":
headersFilterHolder.setInitParameter("includedPaths", v);
break;
"EXCLUDEED.PATHS":
headersFilterHolder.setInitParameter("excludedPaths", v);
break;
"INCLUDED.MIME.TYPES":
headersFilterHolder.setInitParameter("includedMimeTypes", v);
break;
"ENCLUDED.MIME.TYPES":
headersFilterHolder.setInitParameter("excludedMimeTypes", v);
break;
"INCLUDED.HTTP.METHODS":
headersFilterHolder.setInitParameter("includedHttpMeethods", v);
break;
"ENCLUDED.HTTP.METHODS":
headersFilterHolder.setInitParameter("excludedHttpMeethods", v);
break;
}
});
}
References
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://www.eclipse.org/jetty/documentation/current/header-filter.html
https://www.eclipse.org/jetty/javadoc/9.4.24.v20191120/org/eclipse/jetty/servlets/HeaderFilter.html
Compatibility, Deprecation, and Migration Plan
Since we just add a new property and the default value for new property is empty string, existing use cases and behavior will be unaffected.
Rejected Alternatives
Another implementation would be writing a customized filter extension to intercept and set HTTP response headers. Ultimately the purpose of this KIP will allow users to set HTTP response headers, using this alternative make implementation much complex and doesn't gain any benefits.