You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Overview

Currently, the only way for a command developer to modify cluster configuration is through internal methods or to manually edit the configuration xml file while the cluster is offline.
It can be expected that a module developer who create commands will need to modify the cluster configuration as well, therefore, we should expose a public API to retrieve or manipulate the cluster configuration.

 

Background

The cache-1.0.xsd defines the permissible structure of the cluster config, currently represented in XML.
Developers can include their own cluster configuration elements in one of two places as defined in the cache-1.0.xsd: at the cache level or at the region level.
These accept any XML element, so long as the namespace does not match Geode's (i.e., {{http://geode.apache.org/schema/cache}}).

Goals

  • The API is intended to be used by sub-module developers to create their own commands that would need to access/modify the cluster configuration.
  • The API is intended to ONLY update the cluster or server-group's configuration saved by the locator(s). 
  • The API should not be tied with XML. We should be able to change how we save cluster configuration internally without changing the api.

Anti-Goals

  • This API is not to be confused with an API that app-developers can use from client or from servers to, say, create a region and update the cluster configuration at the same time. That API would entail a remote invocation of this lower-level api, but it's not the same.

Approach

  • Leverage JAXB to generate an initial set of configuration objects from cache-1.0.xsd. These will be used internally to marshall/unmarshll from/to xml. Developers only need to work with these configuration objects, not directly with xml.
  • Module developers will use their own xsd to create module specific configuration object. The api should allow developers to pass in these objects and be saved in the cluster configuration.
  • Expose a public ClusterConfigurationService via a public abstract command.

Proposal

As a minimum viable product, we would require only two methods: a getter and an updater.

public interface ClusterConfigurationService {
  /**
  * @param group The group to which the cluster configuration applies. default is "cluster"
  * @param additionalBindClass custom element classes created by other modules if present.
  * @return The cluster configuration for the specified group.
  */
  CacheConfig getCacheConfig(String group, Class<? extends CacheElement>... additionalBindClass);
 
  /**
  * @param group The group to which the cluster configuration applies. default is "cluster"
  * @param mutator Specification of how the cluster configuration should be altered.
  * @param additionalBindClass custom element classes created by other modules if present.
  */
  void updateCacheConfig(String group, UnaryOperator<CacheConfig> mutator, Class<? extends CacheElement>... additionalBindClass);
}


Intended Use:

Given an interface as the above, we would be able to implement, for instance, the CreateIndexCommand instead as

public class CreateIndexCommand implements GfshCommand {
  private static final CreateIndexFunction createIndexFunction = new CreateIndexFunction();

  @CliCommand(value = CliStrings.CREATE_INDEX, help = CliStrings.CREATE_INDEX__HELP)
  @CliMetaData(relatedTopic = {CliStrings.TOPIC_GEODE_REGION, CliStrings.TOPIC_GEODE_DATA})
  @ResourceOperation(resource = ResourcePermission.Resource.CLUSTER,
      operation = ResourcePermission.Operation.MANAGE, target = ResourcePermission.Target.QUERY)
  public Result createIndex(...){
 
  Result result;
  final Set<DistributedMember> targetMembers = findMembers(group, memberNameOrID);

  if (targetMembers.isEmpty()) {
    return ResultBuilder.createUserErrorResult(CliStrings.NO_MEMBERS_FOUND_MESSAGE);
  }

  // Objective: create this Index object.
  // This class was generated by JAXB
  RegionConfig.Index index = new RegionConfig.Index();
  index.setName(indexName);
  index.setExpression(indexedExpression);
  index.setType(indexType.name());
  index.setFromClause(regionPath);

  // invoke the function on each member, where the index will be created
  List<CliFunctionResult> functionResults =
      executeAndGetFunctionResult(createIndexFunction, index, targetMembers);
  result = ResultBuilder.buildResult(functionResults);

  ClusterConfigurationService service = getSharedConfiguration();
  if (result.getStatus().equals(Result.Status.OK) && service != null) {
    // If the function executes successfully and the ClusterConfigurationService exists then update the cluster config.
	service.updateCacheConfig("cluster", cc -> {
	  RegionConfig regionConfig = findRegionConfig(cc, regionPath);
      if(regionConfig == null) throw new EntityNotFoundException("region is not found");
      regionConfig.getIndex().add(index);
      return cc;
    });
  
  }
  return result;
}

 

Here, we declaratively build the RegionConfig.Index object we would like to add to the configuration.
On successfull creation, we search for the RegionConfig object to which the index configuration should belong and add this index to that array.
With the mutator defined, we execute (a concurrency-safe) update to the cluster configuration.

Custom Cache Element

The above example shows you how you can retrieve/manipuate those elements (e.g. regions, cache servers, gateway receivers, indecies etc) that are defined in CacheConfig (cache.xsd) itself. But cache.xsd allows module developers to put elements of different namespaces inside <cache> and <region> elements. Developers can access those elements from the CacheConfig object retrieved by the getCacheConfig call in the public interface:

List<CacheElement> customElements = cacheConfig.getCustomCacheElements();
List<CacheElement> customRegionElements = cacheConfig.getRegions().get(0).getCustomRegionElements();

For developers who want to easily add/remove these elements from cluster configuration, it is desirable to have these helper methods in the api. These are essentially achieved by calling the two interfaces methods we defined above (getCacheConfig and updateCacheConfig) internally. So they can be provided as default implemetations of the interface.

// helper methods to access custom elments under <cache>
<T extends CacheElement> List<T> getCustomCacheElements(String group, Class<T> classT);
<T extends CacheElement> T getCustomCacheElement(String group, String elementId, Class<T> classT);
void saveCustomCacheElement(String group, CacheElement element);
void deleteCustomCacheElement(String group, String elementId, Class<? extends CacheElement> classT);

// helper methods to access custom elements under <region>
<T extends CacheElement> List<T> getCustomRegionElements(String group, String regionPath, Class<T> classT);
<T extends CacheElement> T getCustomRegionElement(String group, String regionPath, String elementId, Class<T> classT)
void saveCustomRegionElement(String group, String regionPath, CacheElement element);
void deleteCustomRegionElement(String group, String regionPath, String elementId, Class<? extends CacheElement> classT);


Long-term Goals

  • Reduce duplication of effort by replacing "creation" classes (i.e., CacheCreation, RegionCreation, BindingCreation, et al) with JAXB-generated classes.
  • Remove requirement of JAXB / XML annotations on configuration element classes.
  • This api is meant to be used by commands that would modify the Cache configuration, e.g. create region, create gateway receiver etc. It's only available where these commands are executed. After this, we might expose another set of java API that would be available in other nodes like client or servers. That api would allow developer to create a region as well as modifying the cluster configuration at the same time. 

Additional Concerns

Any exposure or modification of cluster configuration should immediately raise concerns for both security and concurrency.

Regarding security:

The onus of responsible and authorized use of this interface rests on the developer using this interface.
Any command or function reporting or modifying cluster configuration should require appropriate permissions.
Within Geode itself, these permissions would be CLUSTER:READ and CLUSTER:WRITE respectively.
Outside Geode, these permissions would correspond to whatever permissions are expected by the application and implementation of SecurityManager.

Remember that permissions may be set for classes implementing CommandMarker (i.e., gfsh commands) via the @ResourceOperation annotation.
Permissions may be set for classes implementing org.apache.geode.cache.execute.Function interface via the interface's method getRequiredPermissions.

Regarding concurrency:

While we are motivated by thread safety to accept a UnaryOperator to mutate the cluster configuration rather than expose an explicit setter, race conditions can still easily exist.
The onus of responsible and safe concurrent use rests again on the developer using this interface.

Internally, we will guarantee safe concurrent writing to avoid data corruption.
However, potential inconsistency due to the order in which commands arrive is beyond our ability or responsibility to control.
Developers are strongly encouraged to use distributed locks to safeguard configuration information between acquisition and update, as well as any context which requires the most up-to-date cluster configuration information.

Resources

  • No labels