Table of Contents |
---|
Status
Current state: Under Accepted
Discussion thread: here
Discussion Vote thread: here
JIRA: here
Jira | ||||||
---|---|---|---|---|---|---|
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
...
Kafka users have reported that their security scanners have identified some missing HTTP headers from Connect REST API response. Specific headers that customers have asked about include:
...
...
...
...
...
Public Interfaces
There is no any changes on public interfaces. We define a new prefix "response.http.headers.<name>.", then followed by a set of properties which define rules for header. The following section has detailed description.
Proposed Changes
Adding Properties
We will add a set of new properties in the org.apache.kafka.connect.runtime.WorkerConfig
class. It will allow the REST server administrator to configure headers based on their security policies. We borrow and take advantage of the Jetty HeaderFilter
class and use the same format of headerConfig
, includedPaths
, excludedPaths
, includedMimeTypes
, excludedMimeTypes
, includedHttpMethods
, and excludedHttpMethods
init parameters.
Description of Properties
header.config
...
We should provide a mechanism to return the desired headers. We can't, in general, anticipate what specific headers their scanners will demand now, much less in the future, so we must make this mechanism configurable. Also we don't foresee the requirement to set different headers for different paths or mime types (since Connect API only return application/json).
Public Interfaces
This proposal adds a new configuration property to customize HTTP response headers. The following section has detailed description.
response.http.headers.config
Type: Comma separated string
Valid Values: Values must use same format as headerConfig defined in Jetty's HeaderFilter documentation:
Code Block |
---|
[action] [header name]: [header value] |
...
[action]
can be one of "of set
, add
, setDate
, or addDate
" which which specify an action will perform on header.
set
action is the same as the setHeader function in HttpServletResponse, it will set a response header with the given name and value. If the header had already been set, the new value overwrites the previous one.add
action is the same as the addHeader function in HttpServletResponse, it will add a new value to the header. Responses headers could have multiple values.setDate
action is the same as thesetDateHeader
same as setDateHeader function in HttpServletResponse. It will set a HTTP header with a need date value. Such as "setDate Expires: 31540000000
" which which indicates the header will be expired approximately one year in the future.addDate
action is the same as theaddDateHeader
addDateHeader function in HttpServletResponse. It will add a response header with the given name and date-value. Such as "addDate Last-Modified: 0
" which indicates the Last-Modified date is same as current system date.
[header name]
specify name of header (For examples, see: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers)
[header value]
specify value for the header. We need to put double quotes around the value if the value contains commas because due to we use comma as separator for different headers. Default Value: ""
Example:
...
language | text |
---|
...
|
...
|
...
|
...
|
...
|
...
|
Output of Response Header:
|
Proposed Changes
Since we're using Jetty to serve this, we can take advantage of the Jetty HeaderFilter class to allow the configuration of these headers. We add configuration property response.http.headers.config
in the org.apache.kafka.connect.runtime.WorkerConfig
class and update org
...
It is optional. it is a comma separated values of included path specs applied to headers.config. See path spec rules section.
Example:
Code Block | ||
---|---|---|
| ||
included.paths=^/test/0$ |
excluded.paths
It is optional. it is a comma separated values of excluded path specs applied to headers.config. See path spec rules section.
Example:
Code Block | ||
---|---|---|
| ||
excluded.paths=^/test/0$ |
included.mime.types
It is optional. it is a comma separated values of included mime types applied to headers.config.
Example:
Code Block | ||
---|---|---|
| ||
included.mime.types=application/json |
excluded.mime.types
It is optional. it is a comma separated values of excluded mime types applied to headers.config.
Example:
Code Block | ||
---|---|---|
| ||
excluded.mime.types=application/xml |
included.http.methods
It is optional. it is a comma separated values of included http methods applied to headers.config.
Example:
Code Block | ||
---|---|---|
| ||
included.http.methods=POST,PUT |
...
It is optional. it is a comma separated values of excluded http methods applied to headers.config.
Example:
Code Block | ||
---|---|---|
| ||
excluded.http.methods=GET |
...
- If the spec starts with
^
, the spec is assumed to be a regex based path spec and will match with normal Java regex rules. - If the spec starts with
/
, the spec is assumed to be a Servlet url-pattern rules path spec for either an exact match or prefix based match. - If the spec starts with
*.
, the spec is assumed to be a Servlet url-pattern rules path spec for a suffix based match. - All other syntaxes are unsupported.
Multiple Headers Configuration
We use Jetty HeaderFilter to implement HTTP response header configuration. We need support multiple rules applied to multiple different response headers.
We define a prefix response.http.headers.<name>. to allow multiple configurations, so that different paths have different headers.
Example:
Code Block |
---|
response.http.headers.header1.header.config=set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0
response.http.headers.header1.header.included.paths=^/test/0$
response.http.headers.header1.excluded.paths=^/test1/0$
response.http.headers.header1.included.mime.types=application/jsonheader
response.http.headers.header1.excluded.mime.types=application/xml
response.http.headers.header1.included.http.methods=POST,PUT
response.http.headers.header1.excluded.http.methods=GET |
Implementation
Implementation will use the Jetty HeaderFilter
class. We need to update org.apache.kafka.connect.runtime.rest.RestServer class. During initialization the Connect process, Connect REST server will will read all header configurations from the property with prefix property response.http.headers.config
, then and create a list of FilterHolder with HeaderFilter
with HeaderFilter class and add the list of filter holders holder to the Servlet context handler based on the name of the filter. Implementation is similar to how we handle the header access.control.allow.origin
in the Connect REST server.
Pseudocode
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
//Transfer WorkConfig to map which map filter name to a map which map name of init parameter to value of init parameter for the filter.
Map<String, Map<String, String>> headerFilterConfigs = extractHeaderFilterConfig(workerConfig);
...
. We only need to provide a single HeaderFilter for the entire server, because that HeaderFilter can set as many headers as the customer needs. Both set
and add
action will add a header and value to the response header if the header is not already in the response. When the header is there, set
action overwrites the existing value, whereas add
action adds an additional value to the header value. So it is the Kafka user's applications and server administrator responsibility to manage headers configured and existing headers.
The following is flow how this will be implemented in RestServer class:
- loads configuration file
- parses the property
response.http.headers.config
in configuration file sing WorkerConfig - creates FilterHolder with HeadFilter class.
- adds the FilterHolder into ServletContextHandler
Pseudocode
private void configureHttpResponsHeaderFilter(ServletContextHandler context) |
Resources
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://www.eclipse.org/jetty/documentation/current/header-filter.html
https://www.eclipse.org/jetty/javadoc/9.4.24.v20191120/org/eclipse/jetty/servlets/HeaderFilter.html
Compatibility, Deprecation, and Migration Plan
Since we just add a new property and the default value for new property is empty string, existing use cases and behavior will be unaffectedThis is a new feature that add HTTP headers based on configuration property response.http.headers.config
. The default value for response.http.headers.config
is empty, so it is backward compatible to old version.
Rejected Alternatives
Another implementation would be writing a customized filter extension to intercept and set HTTP response headers and we have to define same configuration property described in this proposal. Ultimately the purpose of this KIP design will allow users to set HTTP response headers, using . Using this alternative approach make implementation much complex and doesn't gain any benefits.
Supporting multiple header filters add complexity to configuration properties and cause site administrator confusions on configuring Connect REST Server. Most important thing is we do not see applicable customer scenarios at this time. One header filter is sufficient for all HTTP response headers based on existing customer application scenarios and concern. We could expand existing implementation easily if customer need it in the future. For instance, we could add multiple header filters by adding name of header filter between prefix response.http.headers and config such as response.http.headers.{name}.config. We still keep response.http.headers.config as default header filter and internally set name of header filter to default if there is nothing between the response.http.headers and config.
Supporting other optional parameters, that are response.http.headers.included.paths
, response.http.headers.included.mime.types
, response.http.headers.included.methods
, response.http.headers.excluded.paths
, response.http.headers.excluded.mime.types
, response.http.headers.excluded.methods
, don't add value based on existing customer complain about missing header in HTTP response headers. Customer will be happy as long as the headers they want in HTTP response header. This will simply configuration properties lot. We could easily add these optional parameters if customer need it in the future.
Another option would be to try to determine what the right headers are in all cases, and always send them. This is appealing because it would not require any end-user customization. However new security headers are regularly added by the web development community, and it would be difficult for us to anticipate all of our users' needs. By providing a configurable option, users can implement the headers that make sense in their own security environments.