Introduction

I introduced a few new components.

1) Scheduler

Scheduler is a new components it include many critical features.

...

Each queue will receive activation messages from the Kafka for a given action and send them to the ContainerProxy in respond to requests from it.

2) etcd

etcd is a distributed reliable key-value store. etcd is mostly used for transactional support and information sharing among components.

3) Akka-grpc

Akka-grpc is introduced to replace Kafka based execution path.

...

It is also being used to send queue creation request to the scheduler.

4) Akka-cluster

Akka-cluster is used for schedulers to communicate with each other.

...

Entire system is comprised of three flows, 1. queue creation flow, 2. container creation flow, 3. activation flow.

1. Queue Creation Flow

① Once a new request comes, there is no queue for the action, a PoolBalancer starts queue creation flow by randomly choosing a scheduler and calling QueueCreationAPI(Akka-grpc).

...

In the future, PoolBalancer would send activation requests to the scheduler directly.

2. Container Creation Flow

① When a new queue is created, it immediately sends a container creation request to the ContainerManager because there is no container yet.

...

⑥ Once the ContainerManager receives the ack message. It finishes the ContainerCreation in the normal case, and if it receives Rescheduling ack, it chooses another invoker and send ContainerCreation to it.

3. Activation Flow

① While the queue and the containers are being created, activation flows the system like above. PoolBalancer just sends the request to the action topic in Kafka.

...