1.Introduction      

ActivationService is a GRPC service implementation for the scheduler that communicates with the container proxy.
It's implemented with Akka gRPC framework which uses protobuf service descriptor, and requests and responses will be streamed over an HTTP/2 connection.

To communicate between the schedule and the container proxy, the GRPC client is located in ContainerProxy, and the server is located on the scheduler side.

In the upstream code (without scheduler), the activation message is pushed to the ContainerProxy, but in the scheduler, the containerProxy pulls the activation message actively via this ActivationService (akka-grpc service).

2.Architecture Diagram


  • ContainerProxy sends ActivationRequest via fetchActivation method (Akka-grpc)
  • ActivationService delegates the ActivationRequest to QueueManager.
  • QueueManager responds with ActivationResponse to ActivationService.
  • ActivationService responds with ActivationResponse to ContainerProxy

3.Design consideration

  In upstream logic, ContainerProxy uses push node, invoker component gets activationMessage from kafka and send it to ContainerProxy, containers will be created/deleted/paused frequently, 

  it is better for the performance to keep the created container rather than deleting or pausing it and reuse it, so if ContainerProxy uses pull mode can satisfy this.

Protobuf specification

1. rpc FetchActivation (FetchRequest) returns (FetchResponse) {}

FetchActivation rpc is for fetching activation in the memory queue.

  - FetchRequest includes 7 fields

       1. fqn: the serialized content of FullyQualifiedEntityName

       2. rev:  the serialized content of DocRevision, because one action can be updated, so need to provide corresponding DocRevision as well

  - FetchResponse includes one field only

       1. activationMessage, the serialize content of ActivationMessage

2. rpc RescheduleActivation (RescheduleRequest) returns (RescheduleResponse) {}

RescheduleActivation rpc is for rescheduling when the container proxy cannot process messages.


//#services
service ActivationService {

    //ContainerProxy call this grpc method to get ActivationMessage
    rpc FetchActivation (FetchRequest) returns (FetchResponse) {}

    rpc RescheduleActivation (RescheduleRequest) returns (RescheduleResponse) {}

}
//#services

//#messages
// The request message
message FetchRequest {

    //initiator
    string invocationNamespace = 1;

    //the serialize content of FullyQualifiedEntityName
    string fqn = 2;

    //the serialize content of DocRevision
    string rev = 3;

    //the container id
    string containerId = 4;

    //the requested container is warmed or not
    bool warmed = 5;

    //duration of last invoked activation
    google.protobuf.Int64Value lastDuration = 6;

    //the requested container is alive or not
    bool alive = 7;
}

// The response message
message FetchResponse {

    //the serialize content of ActivationMessage
    string activationMessage = 1;
}

message RescheduleRequest {
    string invocationNamespace = 1;
    string fqn = 2;
    string rev = 3;

    //the serialize content of ActivationMessage
    //message will be rescheduled by scheduler
    string activationMessage = 4;
}

message RescheduleResponse {
    // if reschedule request is failed, then it will be `false`
    bool isRescheduled = 1;
}

4.Exception / Failure cases

  • Response delay of QueueManager
    1.respond to the client with Failure
  • If there is no memory queue for the given action

    1.Respond to the client with Failure and ContainerProxy should renew the Akka-grpc client endpoint.

    2.If there is no endpoint for the leader MemoryQueue in ETCD, the ContainerProxy terminates.

  • No labels