Page History

...

At runtime the Ignite user installs all required bundles (including ignite-core, any optional Ignite bundles as well as the application bundles) using either the standard mechanisms defined by the OSGi spec, or relying on the container's implementation-specific capabilities. For example, Apache Karaf (an OSGi implementation) offers a packaging/deloyment concept called "Feature" which roughly speaking is a list of bundles to automatically deploy when the OSGi Framework starts.

Marshalling

Before discussing the proposed marshalling enhancements, it's worth making Ignite's use cases more explicit:

1. Ignite is a client in an OSGi based application, the server nodes do not have the application classes available: all serialization/deserialization done on the client.
2. The server nodes do have the application classes (with or without the client side Ignite). For example, for running compute tasks against user-defined classes the server needs the cache key/value classes as well as the compute closure classes (that get sent by the client nodes to the server nodes) as well as any other classes that the closures require (the parameters).

Clearly use case 2 subsumes the case 1, hence the proposal is to target the use case 2.

The fundamental The main problem we need to solve in order to allow Ignite OSGi enablement is the marshalling. More specifically the issue is with deserialization of the classes that are provided by the bundles other than the JDK and the Ignite bundle itself.

When the Ignite transport layer receives a message it needs to figure out how to deserialize the bytes and for that it needs to know the bundle that provides the class to be deserailized. To make things more complex, the class may contain other classes that come from other bundles, and so on recursively. What In general, what is needed then is a way to map an FQN of a class to the its bundle (and hence to the class loader) which provides the class.

And this is where ClassLoaderCodec comes into to play.

On the high level the proposal is as follows:

1. Ignite core is to maintain the mapping of all external classes (the classes that are neither JDK nor Ignite class) to their corresponding Bundle instances. This can easily be achieved using the Bundle Tracker feature (standard OSGi). When a user bundle is started, ignite-core would scan the byte code of the jar looking for all Serializable/Externalizable classes (TODO: what about anonymous closures and lambdas?) and store the mapping of the class' FQN to the bundle instance.
2. A complication arises due to the fact that in OSGi the same class may be provided by different bundles. In such (rare) cases, the package version can be used to resolve the ambiguity. If the same package version is provided by multiple bundles, the fallback mechanism would be to explicitly include the bundle symbolic name and the bundle version during serialization of such classes.
3. during serialization, for each object of an external class is serialized along with its package version.
4. during deserialization, each external class's FQN is checked against the map created in step 1 to get the correct instance of the class loader (unless the bundle symbolic name and version are specified directly in which case the bundle instance can be obtained from the OSGi Framework itself).

ClassLoaderResolver

1. During serialization, Ignite marshaller calls ClassLoaderCodec.encodeClassLoader(cls) with the class to be serialized as its only parameter. The implementation of the method may return an arbitrary object that in some way (up to the implementation) represents the class loader of the class. The encoded representation of the class loader will be serialized along with the rest of the message data. The returned object may be a primitive, a serializable POJO, or a null.
2. During deserialization, the encoded class loader (as returned by the encodeClassLoader() call during serialization) as well as the FQN of the class being deserialized are passed into ClassLoaderCodec.decodeClassLoader(fqn, encodedClassLoader) method. The implementation of the method is expected to decode and return an instance of the class loader to use for loading the class with the given FQN.

It's responsibility of the implementation to ensure that the encoded representation is sufficient to unambiguously identify the correct bundle during deserialization.

ClassLoaderCodec

The ClassLoaderCodec should The ClassLoaderResolver should be called for every Object during serialization and deserialization and should be part of the IgniteConfiguraiton:

public interface ClassLoaderResolverClassLoaderCodec {
    @Nullable
    public Object encodeClassLoader(Class<?> cls) throws IgniteException;
    public ClassLoader decodeClassLoader(String fqn, @Nullable Object objencodedClsLdr) throws IgniteException;
}

ClassLoaderCodec Implementations

In general in OSGi, the same package may be exported by multiple bundles and therefore an FQN may not be sufficient to look up the correct class loader. In such cases, the codec implementation must employ a pessimistic approach and encode enough information (for example, the bundle symbolic name, plus the bundle version) for the deserializer to be able to resolve the FQN to the correct class loader. Such implementation will work for all use cases, but it introduces some overhead and increases the size of the serialized messages.

However, for the applications that can enforce one-to-one mapping of packages to bundles, a simplified (optimistic) approach can be used instead. With this approach, no encoding of the class loader is required (encodeClassLoader() returns null), and only the FQN is used for decoding of the class loader.

Here's how the pessimistic codec implementation might look like (in pseudo-code)We should also provide OSGI class loader resolver out of the box: 

public class OsgiClassLoaderResolver { PessimisticClassLoaderCodec implements ClassLoaderCodec {

    public PessimisticClassLoaderCodec() {}
 
    //@Nullable
 Manifest entry names public to look up bundles during deserialization.Object encodeClassLoader(Class<?> cls) throws IgniteException {
    private Collection<String> mfEntryNames;
     // TODO
    }

	public OsgiClassLoaderResolver() {}
ClassLoader decodeClassLoader(String fqn, @Nullable Object encodedClsLdr) throws IgniteException {
        // TODO
    }
}

Here's how the optimistic (opportunistic :)))) codec implementation might look like:

public OsgiClassLoaderResolver(Collection<String> mfEntryNames) class OptimisticClassLoaderCodec implements ClassLoaderCodec {

    public OptimisticClassLoaderCodec()   ...{}
 
    }@Nullable
 
    public Object encodeClassLoader(ClassLoaderClass<?> clsLdrcls) throws IgniteException {
        // TODO
    }
    
	public ClassLoader decodeClassLoader(String fqn, @Nullable Object objencodedClsLdr) throws IgniteException {
        // TODO
    }
}

Ignite Optional Dependencies

We need a clear path to support Ignite optional dependencies in OSGI environments.

TODO: use OSGI fragments?

Example

...