Contexts are a way to structure the state and Configuration of an Evaluator. A Context exists on one and only one individual Evaluator. Each Evaluator has at least one Context, which we refer to as the root Context. This root context is special, as it is required on all Evaluators and because closing it is synonymous to releasing the Evaluator. In many simple REEF programs, this is the only Context used and it is therefore convenient to think of it as synonymous to "the Evaluator": The Driver can submit Tasks to it, is notified when they are done and can close the root context when it wants to dispose of the Evaluator. 

If you have more than one Context on an Evaluator, it is instructive to think of them as a stack:

Task
ActiveContext
...
RootContext

In this stack, the root context forms the bottom. The top is called the active context. The stack of contexts is formed by calls to submitContext() to the event types that allow this (AllocatedEvaluator and ActiveContext). Contexts are the main way for an Evaluator to be exposed and accessed. For instance, Tasks are submitted to an ActiveContext which represents the top Context on the Evaluator.

Beyond this, a Driver can submit a Context to the root, or in fact any, Context, as long as the resulting structure is that of a stack: The root Context forms the bottom of the stack, the top-most Context is called active, hence theActiveContext event. The two can be one and the same, and often are: The root Context is the subject of the first ActiveContext event on an Evaluator.

Nomenclature: When Context B is submitted to an already existing Context A, we say that Context A is the parent Context of Context B. Also, Context B is the child of Context A.

It is only the ActiveContext that allows the submission of Tasks or child Contexts. This ensures the stack property of contexts.

Objects and Configuration: What's in a Context?

It is convenient to think of a Context as a Configuration that gets merged with the Configuration supplied for Tasks and child Contexts. While not entirely true (see below), this view allows us to show just why Contexts are a convenient construct.

It is often the case that subsequent tasks that get executed on an Evaluator want access to the same Configuration variables and / or the same objects. Consider a simple LinkedList bound to a named parameter. If that linked list is part of the subsequent Task Configurations submited, each Task will get its very own LinkedList. If the named parameter is bound in the Context Configuration, all Tasks subsequently submitted to the Context will get the very same LinkedListinstance.

Contexts also have their own life cycle: They generate events like ContextStart and ContextStop.This allows us to group the life cycle of subsequent, related tasks. Consider the group communications library as an example: It establishes network connections between all the participating Evaluators. Having these managed in the Context (instead of a Task) allows us to amortize the cost of setting these up across Tasks.

Contexts are (Tang) Injectors

This mechanism is implemented by using Tang's Injectors. On the Evaluator, a Task is launched by first forking the Context's Injector with the TaskConfiguration and then requesting an instance of the Task interface from that forked Injector. By this mechanism and the fact that objects are singletons with respect to an Injector in Tang, object sharing can be implemented. All objects already instantiated on the Context Injector will also be referenced by the TaskInjector. Hence, the LinkedList in the example above would be shared amongst subsequent Task Injectors in the construction of the Task instance.

To a first degree, you can think of this structure as the following stack:

LevelInjector structure
Task
taskInjector = activeContextInjector.fork(taskConfiguration)
task = taskInjector.getInstance(Task.class)
ActiveContext
activeContextInjector = rootContextInjector.fork(contextConfiguration)
RootContext
rootContextInjector = Tang.newInjector(rootContextConfiguration)

This illustrates how objects and configuration from the lower contexts are shared with the upper contexts in the stack. We rely on Tang to make this happen. However, the above simplifies one core issue: When forking injectors in Tang, the same rules apply as when merging Configuration instances. Most crucially, one cannot override bindings already present in an Injector when forking it. This presents a problem with the above design: every context has its own (injectable) ID using the same named parameter. Hence, one of the fork calls above work. To circumvent this, we use two Configurations and Injectors: One that is local to this Context called ContextConfiguration and one that is shared with subsequent contexts called ServiceContext. In the stack above, this looks like this:

LevelCode on the DriverEvaluator side
Task
activeContext.submit(taskConfiguration)
taskInjector = activeContextInjector.fork(taskConfiguration)task = taskInjector.getInstance(Task.class)
ActiveContext
activeContext.submitContextAndService(contextConfiguration, serviceConfiguration)
activeServiceInjector = rootServiceInjector.fork(serviceConfiguration)
activeContextInjector = activeServiceInjector.fork(contextConfiguration)
RootContext
allocatedEvaluator.submitContextAndService(rootContextConfiguration, rootServiceConfiguration)
rootServiceInjector = Tang.newInjector(rootServiceConfiguration)
rootContextInjector = rootServiceInjector.fork(rootContextConfiguration)

Note that only the content of the ServiceInjector(s) is shared with downstream contexts.

 

  • No labels