You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Status

Current state: [One of "Under Discussion", "Accepted", "Rejected"]

Discussion thread: 

JIRA or Github Issue: 

Released: <Doris Version>

Google Doc: <If the design in question is unclear or needs to be discussed and reviewed, a Google Doc can be used first to facilitate comments from others.>

Motivation

Currently, some http interfaces provided by doris are not authenticated (account password check and global/ database/ table permission check are not performed). These interfaces have security problems and need to be authenticated.

In addition, http data is transmitted in plaintext, which is subject to eavesdropping, tampering, and impersonation risks. https is a secure SSL encryption transmission protocol. The doris interface supports https, which greatly improves its security and provides enterprise-level security support.

Related Research

Detailed Design

1. FE Interface Authentication And Https Implementation

Http Interface Authentication

For unauthenticated http interfaces, execute CheckPassword and CheckGlobalAuth or CheckDbAuth or CheckTblAuth;

For partially authenticated http interfaces, only CheckPassword is executed, but CheckGlobalAuth or CheckDbAuth or CheckTblAuth is not executed.

Execute CheckGlobalAuth or CheckDbAuth or CheckTblAuth based on the permissions required by the interface, as shown in follow table:

接口名称

功能描述

RequestMapping

是否鉴权

MetaService

fe 元数据相关API，除`/dump` 以外，都为fe 节点之间内部通讯用，/dump 用于获取image 文件的存储路径

  1. GET /image (/info) (/version) (/put) (/journal_id) (/role) (/check)

  2. GET /dump

否（checkFromValidFe）1
否（executeCheckPassword ）2

BackendsAction

返回be 列表，包括IP、PORT 等信息

GET /api/backends

是（executeCheckPassword）
checkGlobalAuth-PrivPredicate.ADMIN

GetSmallFileAction

下载在文件管理器中的文件

GET /api/get_small_file?token=xxx&file_id=xxx

否（executeCheckPassword + checkGlobalAuth-PrivPredicate.ADMIN）

HealthAction

返回集群当前存活与宕机的be 节点数

GET /api/health

否（executeCheckPassword）

MetaInfoAction

获取集群内的元数据信息
数据库列表
表列表
表结构

  1. GET /api/meta/namespaces/<ns>/databases

  2. GET /api/meta/namespaces/<ns>/databases/<db>/tables

  3. GET /api/meta/namespaces/<ns>/databases/<db>/tables/<tbl_name>/schema

是（checkWithCookie）
checkGlobalAuth-PrivPredicate.ADMIN

ShowAction

获取集群/数据库的相关信息
元数据信息
PROC 信息
JVM 的Runtime 信息
集群/数据库的数据量

  1. GET /api/show_meta_info?action=xxx

  2. GET /api/show_proc

  3. GET /api/show_runtime_info

  4. GET /api/show_data/<db>

否（executeCheckPassword + checkGlobalAuth-PrivPredicate.ADMIN）1，3，4
是（executeCheckPassword + checkGlobalAuth-PrivPredicate.ADMIN）2

StmtExecutionAction

  1. 执行sql 语句并返回结果

  2. 获取sql 中相关表的建表语句

  1. POST /api/query/<ns>/<db>

  2. POST /api/query_schema/<ns>/<db>

是（checkWithCookie）
checkDbAuth-PrivPredicate.ADMIN 1

MetricsAction

获取metrics

GET /metrics

否（executeCheckPassword）

BootstrapFinishAction

判断fe 是否启动完成

GET /api/bootstrap

否（executeCheckPassword）

ExtraBasepathAction

返回接口API 的base path

GET /api/basepath

否（executeCheckPassword）

ImportAction

查看格式为CSV 或PARQUET 的文件内容

POST /api/import/file_review

否（executeCheckPassword）

StatisticAction

获取集群统计信息、库表数量等

GET /rest/v2/api/cluster_overview

否（executeCheckPassword）

Https Implementation

We implement https based on spring boot, and implement https through configuration and http coding.

  1. Generate an SSL Certificate
    1. Test environment certificate is generated using keytool or openssl, and production environment certificate is purchased from the CA.
  2. Configure https in Spring Boot
    1. Add ssl certificate configuration items in the fe configuration file, including the port number, certificate path, password, format, alias, and enable or disable (ssl is disabled by default).

      http_port
      https_port = 8040
      ssl_key_store_path
      ssl_key_store_password
      ssl_key_store_type
      ssl_key_store_alias
      ssl_enable

    2. When fe is started, related configuration parameters are read into the HttpServer class, and then ssl parameters are configured using SpringApplicationBuilder().properties(properties).

      server.port = https_port // https端口
      server.http.port = http_port // http端口
      server.ssl.key-store = ssl_key_store_path // 证书路径
      server.ssl.key-store-password = ssl_key_store_password // 证书密码
      server.ssl.key-store-type = ssl_key_store_type // 密钥库类型
      server.ssl.keyalias: ssl_key_store_alias // 证书别名
      server.ssl.enabled = ssl_enable // 是否开启ssl

  3. Implement http connector and redirect http requests to https
    1. Construct the httpConnector and set the http port and https port.
    2. Through TomcatServletWebServerFactory::addAdditionalTomcatConnectors() to add multiple surveillance connection.

2.BE Interface Authentication And Https Implementation

Http Interface Authentication


Https Implementation


Scheduling

I will implement http interface authentication and https implementation of be and fe in turn, maybe it will take about 2 months to finish this feature:

  1. Collate all http interfaces, including interface names, function descriptions, request mapping and authentication.
  2. Add authentication for an interface that is not authenticated.
  3. Implement https interface and provide enterprise-level security support

Reference

https://clickhouse.com/docs/zh/interfaces/http

https://docs.altinity.com/operationsguide/security/clickhouse-hardening-guide/network-hardening/#enable-tls

https://github.com/OKaluzny/springboot-rest-api-angularjs-https

  • No labels