Status

Current state: Under Discussion

Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

The current Connect REST server only sets a few default HTTP response headers. It's missing many headers, including most headers related to security. The Connect REST server uses an embedded Jetty server as the Java HTTP server and Java Servlet container, so users have no way to configure HTTP response headers for Connect REST server. Many customers using Connect REST server are demanding some headers related to security in the HTTP response. Some examples of headers are X-XSS-Protection, Content-Security-Policy, Strict-Transport-Security and X-Content-Type-Options.  Some resources need more protection than others due to security required. So we will allow site administrators to control which resources the user agent is allowed or is not allowed to load for given page. For this case, the sensitive resources need use Content-Security-Policy in response header. Some resources can only be accessed by HTTPS instead of HTTP, then Strict-Transport-Security response header (often abbreviated as HSTS) needs be set to let a web site tell browsers that it should only be accessed using HTTPS. So that is why we provide feature to support configuring different headers for different resources.

Public Interfaces

There are no changes to existing public interfaces, but we will add new configuration options. We follow same pattern like configuring different Kafka listeners. We define a new property response.http.headers which defines the names of the header filters that will be configured. The default value for response.http.headers is empty string which means there are no header filters configured. We define a new prefix "response.http.headers.{name}.", then followed by a set of properties which define rules for header. The {name} will be one defined in response.http.headers. The following section has detailed description.

Proposed Changes

Adding Properties

We will add a set of new properties in the org.apache.kafka.connect.runtime.WorkerConfig class. It will allow the REST server administrator to configure headers based on their security policies. We borrow and take advantage of the Jetty HeaderFilter class and use the same format of headerConfig, includedPaths, excludedPaths, includedMimeTypes, excludedMimeTypes, includedHttpMethods, and excludedHttpMethods init parameters as HeaderFilter. Please see two references for Jetty HeaderFilter.

https://www.eclipse.org/jetty/documentation/current/header-filter.html
https://www.eclipse.org/jetty/javadoc/9.4.24.v20191120/org/eclipse/jetty/servlets/HeaderFilter.html

Description of Properties

/connectors/connector1/topics/*

Detailed Explanation for response.http.headers.{name}.header.config

The format will be "[[action] [header]:[header value],..." which is a list of [action] [header]:[value] separated by comma ",". So it is a CSV of actions to perform on headers with the following syntax:
[action] [header name]: [header value],
[action] can be one of "set, add, setDate, or addDate" which specify an action to perform on the header. 

• set action is the same as the setHeader function in HttpServletResponse, it will set a response header with the given name and value. If the header had already been set, the new value overwrites the previous one.
• add action is the same as the addHeader function in HttpServletResponse, it will add a new value to the header. Response headers can have multiple values.
• setDate action is the same as the setDateHeader function in HttpServletResponse. It will set a HTTP header with a date value. Such as "setDate Expires: 31540000000" which indicates the header will be expired approximately one year in the future.
• addDate action is the same as the addDateHeader function in HttpServletResponse.  It will add a response header with the given name and date-value. Such as "addDate Last-Modified: 0" which indicates the Last-Modified date is same as current system date.

[header name] specify name of header.
[header value] specify value for the header. We need to put double quotes around the value if the value contains commas because we use comma as separator for different headers. 

Path Spec Rules:

• If the spec starts with ^, the spec is assumed to be a regex based path spec and will match with normal Java regex rules.
• If the spec starts with /, the spec is assumed to be a Servlet url-pattern rules path spec for either an exact match or prefix based match.
• If the spec starts with *., the spec is assumed to be a Servlet url-pattern rules path spec for a suffix based match.
• All other syntaxes are unsupported.

Multiple Headers Configuration

We use Jetty HeaderFilter to implement HTTP response header configuration. We need support different rules applied to different response headers based on different resources. For most usage cases, one header should be sufficient. But we do provide flexible configuration for multiple headers usages.  

We define a prefix response.http.headers.{name}. to allow configure multiple configurations for different resources. The "{name}", for instance "connector1" in following example, will be used to uniquely identify a set of HTTP response headers applied to one resource under /connectors/connector1

Example:

response.http.headers.connetor1.header.config=set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0
response.http.headers.connetor1.header.included.paths=/connectors/connector1/*
response.http.headers.connetor1.excluded.paths=/connectors/connector1/status
response.http.headers.connetor1.included.mime.types=application/json
response.http.headers.connetor1.excluded.mime.types=application/xml
response.http.headers.connetor1.included.http.methods=POST,PUT
response.http.headers.connetor1.excluded.http.methods=GET

response.http.headers.connetor2.header.config=set X-Frame-Options: DENY, "add Cache-Control: no-cache, no-store, must-revalidate", setDate Expires: 31540000000, addDate Last-Modified: 0
response.http.headers.connetor2.header.included.paths=/connectors/connector2/*
response.http.headers.connetor2.excluded.paths=/connectors/connector2/status     
response.http.headers.connetor2.included.mime.types=application/json
response.http.headers.connetor2.excluded.mime.types=application/xml
response.http.headers.connetor2.included.http.methods=POST,PUT
response.http.headers.connetor2.excluded.http.methods=GET

Implementation

Implementation will use the Jetty HeaderFilter class. We need to update org.apache.kafka.connect.runtime.rest.RestServer class. During initialization process, the Connect REST server will check property response.http.headers. If the value of response.http.headers is not empty, then REST server will read all headers configurations from the properties with prefix response.http.headers.{name}, and create a list of FilterHolder with HeaderFilter class and add the list of filter holders to the Servlet context handler based on the name of the header. Implementation is similar to how we handle the header access.control.allow.origin in the Connect REST server.

Pseudocode

public void configureHttpResponsHeaderFilters(ServletContextHandler context) {
  Map<String, Map<String, String>> headersConfigs = extractHttpResponseHeaderConfig();
  FilterHolder headerFilterHolder = null;
  for (Map.Entry<String, Map<String, String>> entry : headersConfigs.entrySet()) {
    headerFilterHolder = new FilterHolder(HeaderFilter.class);
    headerFilterHolder.setName(entry.getKey());
    Map<String, String> oneHeaderConfig = entry.getValue();
    for (Map.Entry<String, String> oneHeader : oneHeaderConfig.entrySet()) {
      switch (oneHeader.getKey().toUpperCase()) {
        case "HEADER.CONFIG":
          headerFilterHolder.setInitParameter("headerConfig", oneHeader.getValue());
          break;
        case "INCLUDED.PATHS":
          headerFilterHolder.setInitParameter("includedPaths", oneHeader.getValue());
          break;
        case "EXCLUDED.PATHS":
          headerFilterHolder.setInitParameter("excludedPaths", oneHeader.getValue());
          break;
        case "INCLUDED.MIME.TYPES":
          headerFilterHolder.setInitParameter("includedMimeTypes", oneHeader.getValue());
          break;
        case "ENCLUDED.MIME.TYPES":
          headerFilterHolder.setInitParameter("excludedMimeTypes", oneHeader.getValue());
          break;
        case "INCLUDED.HTTP.METHODS":
          headerFilterHolder.setInitParameter("includedHttpMethods", oneHeader.getValue());
          break;
        case "ENCLUDED.HTTP.METHODS":
          headerFilterHolder.setInitParameter("excludedHttpMethods", oneHeader.getValue());
          break;
        default:
      }
    }
    context.addFilter(headerFilterHolder, "/*", EnumSet.of(DispatcherType.REQUEST));
  }
}

References

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://www.eclipse.org/jetty/documentation/current/header-filter.html
https://www.eclipse.org/jetty/javadoc/9.4.24.v20191120/org/eclipse/jetty/servlets/HeaderFilter.html

Compatibility, Deprecation, and Migration Plan

Since we just add a new property and the default value for new property is empty string, existing use cases and behavior will be unaffected.

Rejected Alternatives

Another implementation would be writing a customized filter extension to intercept and set HTTP response headers. Ultimately the purpose of this KIP will allow users to set HTTP response headers, using this alternative make implementation much complex and doesn't gain any benefits.