OSGI Compatibility

Ignite OSGi enablement involves at least two tasks:

"bundlezation" of the Ignite Core jar, as well as the optional jars.
OSGi-aware marshaller implementation.

Bundlezation

For the inter-package dependencies (package export/import) and other bundle metadata, OSGi relies on a number of special entries in the jar's META-INF/MANIFEST file. Those entries are usually generated semi-automatically by the Maven Bundle plugin during a Maven build. The OSGi Framework will refuse to load a jar without a valid OSGi manifest defined.

Since the Ignite's distribution includes a number of optional dependencies (such as ignite-indexing, ignite-log4j, etc) in addition to non-optional ignite-core, the proposal is to make the optional dependencies the bundle fragments hosted by the ignite-core bundle. The fact that fragments share the class loader of the host bundle should simplify inter operation between the components.

At runtime the Ignite user installs all required bundles (including ignite-core, any optional Ignite bundles as well as the application bundles) using either the standard mechanisms defined by the OSGi spec, or relying on the container's implementation-specific capabilities. For example, Apache Karaf (an OSGi implementation) offers a packaging/deloyment concept called "Feature" which roughly speaking is a list of bundles to automatically deploy when the OSGi Framework starts.

Marshalling

Before discussing the proposed marshalling enhancements, it's worth making Ignite's use cases more explicit:

Ignite is a client in an OSGi based application, the server nodes do not have the application classes available: all serialization/deserialization done on the client.
The server nodes do have the application classes (with or without the client side Ignite). For example, for running compute tasks against user-defined classes the server needs the cache key/value classes as well as the compute closure classes (that get sent by the client nodes to the server nodes) as well as any other classes that the closures require (the parameters).

Clearly use case 2 subsumes the case 1, hence the proposal is to target the use case 2.

The fundamental problem we need to solve in order to allow Ignite OSGi enablement is the marshalling. More specifically the issue is with deserialization of the classes that are provided by the bundles other than the Ignite bundle itself.

When the Ignite transport layer receives a message it needs to figure out how to deserialize the bytes and for that it needs to know the bundle that provides the class to be deserailized. To make things more complex, the class may contain other classes that come from other bundles, and so on recursively. What is needed then is a way to map an FQN of a class to the bundle (and hence to the class loader) which provides the class.

On the high level the proposal is as follows:

Ignite core is to maintain the mapping of all external classes (the classes that are neither JDK nor Ignite class) to their corresponding Bundle instances. This can easily be achieved using the Bundle Tracker feature (standard OSGi). When a user bundle is started, ignite-core would scan the byte code of the jar looking for all Serializable/Externalizable classes (TODO: what about anonymous closures and lambdas?) and store the mapping of the class' FQN to the bundle instance.
A complication arises due to the fact that in OSGi the same class may be provided by different bundles. In such (rare) cases, the package version can be used to resolve the ambiguity. If the same package version is provided by multiple bundles, the fallback mechanism would be to explicitly include the bundle symbolic name and the bundle version during serialization of such classes.
during serialization, for each object of an external class is serialized along with its package version.
during deserialization, each external class's FQN is checked against the map created in step 1 to get the correct instance of the class loader (unless the bundle symbolic name and version are specified directly in which case the bundle instance can be obtained from the OSGi Framework itself).

ClassLoaderResolver

The ClassLoaderResolver should be called for every Object during serialization and deserialization and should be part of the IgniteConfiguraiton:

public interface ClassLoaderResolver {
    public Object encodeClassLoader(ClassLoader clsLdr) throws IgniteException;
    public ClassLoader decodeClassLoader(Object obj) throws IgniteException;
}

We should also provide OSGI class loader resolver out of the box:

public class OsgiClassLoaderResolver {
    // Manifest entry names to look up bundles during deserialization.
    private Collection<String> mfEntryNames;
 
    public OsgiClassLoaderResolver() {}

    public OsgiClassLoaderResolver(Collection<String> mfEntryNames) {
        ...
    }
 
    public Object encodeClassLoader(ClassLoader clsLdr) throws IgniteException {
        // TODO
    }
    public ClassLoader decodeClassLoader(Object obj) throws IgniteException {
        // TODO
    }
}

Ignite Optional Dependencies

We need a clear path to support Ignite optional dependencies in OSGI environments.

TODO: use OSGI fragments?

Example

TODO

Page tree