Motivation

When implementing microservices, users are often face with the task of separating the business logic from the common "middleware" logic.

An example of a typical “middleware” task is auditing calls to business service methods (the system must understand which user called which methods and with what result).

Modern frameworks such as gRPC^[1] provide flexible API for implementing request interceptors, with which you can solve almost any middleware task.

Apache Ignite does not provide any mechanisms for solving such problems in general. The user needs to implement it himself, which often results in a lot of boilerplate code. 

Description

The Ignite Service Grid must support the following capabilities:

1. Ability to pass custom context from caller to service (similar to HTTP request headers).
2. Ability to define custom interceptors for service calls.

Phase 1 Service call context (done)

Public API

• ServiceCallContext -  immutable user parameter map that will be implicitly passed to the service (and interceptor) on every method call.
  

  ServiceCallContext.java Expand source

  public interface ServiceCallContext {
      public String attribute(String name);
  
      public byte[] binaryAttribute(String name);
  }

• ServiceContextBuilder -  builder for ServiceCallContext.

• New methods in IgniteServices to pass caller context to service proxy.

  ServiceCallContext.java Expand source

  public <T> T serviceProxy(String name, Class<? super T> svcItf, boolean sticky, ServiceCallContext callCtx)
  
  public <T> T serviceProxy(String name, Class<? super T> svcItf, boolean sticky, ServiceCallContext callCtx, long timeout)

• New method in ServiceContext for getting caller context inside the service method.

  ServiceCallContext.java Expand source

  /**
   * Gets context of the current service call.
   *
   * @return Context of the current service call, possibly {@code null}.
   * @see ServiceCallContext
   */
  @Nullable public ServiceCallContext currentCallContext();

• New property in ServiceResource annotation.

  ServiceCallContext.java Expand source

  /**
   * Flag indicating that the service call context should be passed to the injected service.
   *
   * @return {@code True} if the service call context should be passed to the injected service.
   */
  public boolean forwardCallerContext() default false;

Implementation details

The user can create context (map with custom parameters) and bind it to the service proxy. After that, each call to the proxy method will also implicitly pass context parameters to the service.

Service method can read current context parameters using ServiceContext#currentCallContext method. It is only accessible from the current thread during the execution of a service method.

If one service calls another, then by default the current call context will not be bound to the created proxy - the user must explicitly bind it. But Java service has a special ServiceResource annotation to inject another service proxy into the current service. If the user wants to redirect the current call context to this (injected) proxy, he can set the forwardCallerContext option of this annotation.

Phase 2 Interceptors (active)

Public API

ServiceCallContext should be available only on server side.

The user can bind a custom map/dictionary of parameters  to the service proxy and read them from the call context on the server side.

So, some current public APIs need to be changed (all discussed API are experimental):

• Remove ServiceCallContextBuilder

• Rework methods in IgniteServices to allow user to bind Map of attributes (instead of ServiceCallContext) to service proxy.

  ServiceCallContext.java Expand source

  public <T> T serviceProxy(String name, Class<? super T> svcItf, boolean sticky, Map<String, Object> callAttrs)
  
  public <T> T serviceProxy(String name, Class<? super T> svcItf, boolean sticky, Map<String, Object> callAttrs, long timeout)

• ServiceCallInterceptor  - intercepts service method calls.
  

  ServiceCallInterceptor.java Expand source

  public interface ServiceCallInterceptor extends Serializable {
      // Called BEFORE the service method is executed.
      public default void onInvoke(ServiceCallContext ctx) throws ServiceInterceptException {
          // No-op.
      }
  
      // Called AFTER the service method is executed.
      public default void onComplete(@Nullable Object res, ServiceCallContext ctx) throws ServiceInterceptException {
          // No-op.
      }
  
      // Called when onInvoke, onComplete or service method throws an exception.
      public default void onError(Throwable err, ServiceCallContext ctx) {
          // No-op.
      }
  }

• ServiceCallContext -  interceptor obtains method call parameters from it and can use it to update the user attributes).

  ServiceInterceptorContext.java Expand source

  public interface ServiceCallContext {
      public String method();
  
      public @Nullable Object[] arguments();
  
      public Map<String, Object> callAttributes();
  }

• ServiceInterceptException - unchecked exception that is used to highlight the exception that occurred during method interception (not execution).

• Change property name in ServiceResource annotation (forwardCallerContext => forwardCallAttributes).

  ServiceCallContext.java Expand source

  /**
   * Flag indicating that the service call attributes should be passed to the injected service.
   *
   * @return {@code True} if the service call attributes context should be passed to the injected service.
   */
  public boolean forwardCallAttributes() default false;

Usage example

        ServiceCallInterceptor security = new ServiceCallInterceptor() {
            @Override public void onInvoke(ServiceInterceptorContext ctx) throws ServiceInterceptException {
                // Check permission before execution of the method.
                if (!CustomSecurityProvider.get().access(ctx.method(), ctx.attribute("sessionId")))
                    throw new SecurityException("Method invocation is not permitted");
            }
        }

        ServiceCallInterceptor audit = new ServiceCallInterceptor() {
            @Override public void onInvoke(ServiceInterceptorContext ctx) {
                // Record an event before execution of the method.
                AuditProvider.get().recordStartEvent(ctx.method(), ctx.attribute("sessionId"));
            }

            @Override public void onComplete(@Nullable Object res, ServiceInterceptorContext ctx) {
                AuditProvider.get().recordFinishEvent(ctx.method(), ctx.attribute("sessionId"));
            }

            @Override public void onError(Throwable err, ServiceInterceptorContext ctx) {
                AuditProvider.get().recordError(ctx.method(), ctx.attribute("sessionId"), err.getMessage());
            }
        }

        // Set call attributes for service proxy.
        Map<String, String> attrs = new HashMap<>();
        attrs.put("sessionId", sessionId);

        ServiceConfiguration svcCfg = new ServiceConfiguration()
            .setName("service")
            .setService(new MyServiceImpl())
            .setMaxPerNodeCount(1)
            .setInterceptors(security, audit);

        // Deploy service.
        ignite.services().deploy(svcCfg);

        MyService proxy = ignite.services().serviceProxy("service", MyService.class, false, attrs, 0);

        // A business method call to be intercepted.
        proxy.placeOrder(order1);
        proxy.placeOrder(order2);  

Implementation details

Deployment

One service can have several interceptors. They are defined using the service configuration and deployed with the service.

To add/remove interceptor service should be redeployed.

Interceptor is located and executed where the service is implemented (for Java service - on Java side, for .NET-service on .NET side). Its execution should not cause additional serialization).

Resource injection and lifecycle

Interceptor must support ignite instance resource injection.

Interceptor should support LifeCycleAware

Interception scope

Interceptor only applies to user-defined service methods and does not apply to service lifecycle methods - init, execute and cancel,

Service call context

The user can create custom parameters map and bind it to the service proxy. After that, each call to the proxy method will also implicitly pass context parameters to the service.

Service method can read current context parameters using ServiceContext#currentCallContext#callAttributes method. It is only accessible from the current thread during the execution of a service method.

Interceptor can read and update context parameters using ServiceCallContext.

If an interceptor has been specified, but the user has not passed the caller context through the proxy, then for each call to the service method, an empty context will be dynamically created.

If one service calls another, then by default the current call context will not be bound to the created proxy - the user must explicitly bind it. But Java service has a special ServiceResource annotation to inject another service proxy into the current service. If the user wants to redirect the current call context to this (injected) proxy, he can set the forwardCallerContext option of this annotation.

Exception handling

Interceptor can only throw unchecked exceptions.

Any runtime exception thrown by the onInvoke/onComplete methods will be wrapped in a ServiceInterceptException. This exception will be passed to the initiator (user) and to onError method of the interceptor.

If onInvoke throws an exception, then the service method will not be called.

The exception thrown by the onError method will be added to the main exception as suppressed.

Risks and Assumptions

Discussion Links

https://lists.apache.org/thread.html/r4236c1f23e524dc969bc55057467a2bbe7f9a59a6db7c7fcdc1b7d37%40%3Cdev.ignite.apache.org%3E

Reference Links

[1] https://pkg.go.dev/github.com/grpc-ecosystem/go-grpc-middleware

Tickets