Status

Current state: [One of "Under Discussion", "Accepted", "Rejected"]

Discussion thread: 

JIRA or Github Issue: 

Released: <Doris Version>

Google Doc: <If the design in question is unclear or needs to be discussed and reviewed, a Google Doc can be used first to facilitate comments from others.>

Motivation

Currently, some http interfaces provided by doris are not authenticated (account password check and global/ database/ table permission check are not performed). These interfaces have security problems and need to be authenticated.

In addition, http data is transmitted in plaintext, which is subject to eavesdropping, tampering, and impersonation risks. https is a secure SSL encryption transmission protocol. The doris interface supports https, which greatly improves its security and provides enterprise-level security support.

Related Research

Detailed Design

1. FE Interface Authentication And Https Implementation

Organize Http Documents

Add missing documents and tidy them up.

Http Interface Authentication

For unauthenticated http interfaces, execute CheckPassword and CheckGlobalAuth or CheckDbAuth or CheckTblAuth;

For partially authenticated http interfaces, only CheckPassword is executed, but CheckGlobalAuth or CheckDbAuth or CheckTblAuth is not executed.

Execute CheckGlobalAuth or CheckDbAuth or CheckTblAuth based on the permissions required by the interface, as shown in follow table:

NOTE:

1. The enable_all_http_auth parameter is added to the fe configuration file. The default value is false, indicating that only the interfaces that have been authenticated before are authenticated. If it is set true, all interfaces (including the newly added interface authentication) are authenticated.

2. For BackendsAction, as required, the interface should require user have GlobalAuth-PrivPredicate.ADMIN permission. However, a user who uses spark-doris-connector/  flink-doris-connector does not have PrivPredicate.ADMIN permission, to ensure that the connector works properly, do not verify the permission of the interface.

3. For MetricsAction, account password is authenticated. You need to configure the account and password when you use Prometheus to collect monitoring items.

Https Implementation

We implement https based on spring boot, and implement https through configuration and http coding.

Currently doris uses many ports, so http and https are not supported at the same time. We could choose http or https by setting "ssl_enable" in fe.conf.

1. Generate an SSL Certificate
   1. Test environment certificate is generated using keytool or openssl, and production environment certificate is purchased from the CA.
2. Configure https in Spring Boot

   1. Add ssl certificate configuration items in the fe configuration file, including the port number, certificate path, password, format, alias, and enable or disable (ssl is disabled by default).

      http_port https_port = 8040 ssl_key_store_path ssl_key_store_password ssl_key_store_type ssl_key_store_alias ssl_enable

   2. When fe is started, related configuration parameters are read into the HttpServer class, and then ssl parameters are configured using SpringApplicationBuilder().properties(properties).

      server.port = https_port // https端口 server.http.port = http_port // http端口 server.ssl.key-store = ssl_key_store_path // 证书路径 server.ssl.key-store-password = ssl_key_store_password // 证书密码 server.ssl.key-store-type = ssl_key_store_type // 密钥库类型 server.ssl.keyalias: ssl_key_store_alias // 证书别名 server.ssl.enabled = ssl_enable // 是否开启ssl

3. Implement http connector and redirect http requests to https
   
   1. Construct the httpConnector and set the http port and https port.
   2. Through TomcatServletWebServerFactory::addAdditionalTomcatConnectors() to add multiple surveillance connection.

2.BE Interface Authentication And Https Implementation

Http Interface Authentication

1. When BE performs authentication, because it does not store metadata information, FE needs to provide the corresponding authentication interface, which is implemented through RPC calls.

TO DO

Https Implementation

1. Generate an SSL Certificate

   In developmet environment, SSL certificates can be generated by openssl as follow:

   openssl genrsa -out private.key 1024 openssl req -new -x509 -days 3650 -key private.key -out cert.crt -subj "/CN=my.host.name"

   In production environment, SSL certificates need to be purchased from the CA.

2. Configure path of SSL Certificate in be.conf

   enable_https = true;  ssl_certificate_path = DORIS_HOME_DIR + "/conf/ssl/cert.crt"; ssl_private_key_path = DORIS_HOME_DIR + "/conf/ssl/private.key";

3. Enable SSL in brpc server

   if (config::enable_https) { auto sslOptions = options.mutable_ssl_options();   sslOptions->default_cert.certificate = config::ssl_certificate_path;   sslOptions->default_cert.private_key = config::ssl_private_key_path; } _server->Start(port, &options);

4. Redirect http request to HTTPS

   In the brpc server, SSL-only mode can only be implemented using Controller::is_ssl() in service's callback.

   if (config::ssl_only && !cntl->is_ssl())  {     cntl->http_response().set_status_code(baidu::rpc::HTTP_STATUS_FOUND);     cntl->http_response().SetHeader("Location", "https://xxx"); }

   The global configure for SSL-only mode of brpc would be supported in the future.

Scheduling

I will implement http interface authentication and https implementation of be and fe in turn, maybe it will take about 2 months to finish this feature:

1. Collate all http interfaces, including interface names, function descriptions, request mapping and authentication.
2. Add authentication for an interface that is not authenticated.
3. Implement https interface and provide enterprise-level security support

Reference

https://clickhouse.com/docs/zh/interfaces/http

https://docs.altinity.com/operationsguide/security/clickhouse-hardening-guide/network-hardening/#enable-tls

https://github.com/OKaluzny/springboot-rest-api-angularjs-https