Versions Compared

Old Version 4

changes.mady.by.user Tom Li

Saved on

compared with

New Version 5

changes.mady.by.user Tom Li

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

HS2 is a single process running as a composite service, which includes the Thrift based service (TCP or HTTP) and a Jetty web server for web UI. You can specify the transport mode of the Thrift service through the hive config: "hive.server2.transport.mode".

Thrift is a RPC framework for building cross-platform services. Its stack consists of 4 layers: Server, Transport, Protocol and Processor. You can find more details about the layers at: https://thrift.apache.org/docs/concepts.

The usage of those layers from HS2 implementation is described as followsbelow.

Server

HS2 uses TThreadPoolServer for TCP mode and Jetty server for HTTP transport and uses a TThreadPoolServer (from Thrift) for TCP mode, or a Jetty server for the HTTP mode. 

Regarding the TThreadPoolServer, it allocates one worker thread per TCP connection. Each thread is always associated with a connection even if the connection is idle. So there is a potential performance issue resulting from a large number of threads due to a large number of concurrent connections. In future we may think about switching to another server types for TCP mode, for example TThreadedSelectorServer. Here is a article about a performance comparison between different Thrift Java servers.  

Transport

HTTP mode is required when a proxy is needed between the client and server (for example, for load balancing or security reasons). That's why it's supported as well as TCP mode.

Protocol