Top-Level Goal
The top-level goal is a single API for managing cluster configuration.
The beneficiaries of this work are those who want to change the configuration of the cluster (create/destroy regions, indices or gateway receivers/senders etc), and have these changes replicated on all the applicable servers and persisted in the cluster configuration service. In addition to developers building Geode-based applications, the target user group includes developers working on different parts of the Geode code such as Spring Data for Apache, queries for Lucene index, or storage for the JDBC connector.
Problem Statement
In the current implementation:
- Most cluster configuration tasks are possible, but only by coordinating XML file-based configuration files, properties files, and gfsh commands.
- Many of the desired outcomes are achievable through multiple paths.
- Establishing a consistent configuration and persisting it across the cluster is difficult, sometimes impossible.
Product Goals
The developer should be able to:
Create regions/indices on the fly.
Persist the configuration and apply it to the cluster (when a new node joins, it has the config; when the server restarts, it has the config)
Obtain a consistent view of the current configuration
Apply the same change to the cluster in the same way
Be able to change the configuration in one place
Obtain this configuration without being on the cluster
Proposed Solution
The proposed solution includes:
- Address the multiple path issue by presenting a single public API for configuring the cluster, including such tasks as creating a region destroying an index, or update an async event queue.
- Provide a means to persist the change in the cluster configuration.
- Save a configuration to the Cluster Management Service without having to restart the servers
- Obtain the cluster management service from a cache when calling from a client or a server
- Pass a config object to the cluster management service
- Use CRUD operations to manage config objects
This solution should meet the following requirements:
The user needs to be authenticated and authorized for each API call based on the resource he/she is trying to access.
User can call the API from either the client side or the server side.
The outcome (behavior) is the same on both client and server:
affects cluster wide
idempotent
What We Have Now
Our admin rest API "sort of" already serves this purpose, but it has these shortcomings:
- It's not a public API
- The API is restricted to the operations implemented as gfsh commands, as the argument to the API is a gfsh command string.
- Each command does similar things, yet commands may not be consistent with each other.
Below is a diagram of the current state of things:
From the current state of commands, It's not easy to extract a common interface for all the commands. And developers do not want to use gfsh command strings as a "makeshift" API to call into the command. We are in need of a unified interface and a unified workflow for all the commands.
Proposal
We propose a new Cluster Management Service (CMS) which has two responsibilities:
- Update runtime configuration of servers (if any running)
- Persist configuration (has to be enabled to use CMS)
Note that in order to use this API, Cluster Configuration needs to be enabled.
The CMS API is exposed as a new endpoint as part of "Admin REST APIs", accepting configuration objects (JSON) that need to be applied to the cluster. CMS adheres to the standard REST semantics, so users can use POST, PATCH, DELETE and GET to create, update, delete or read, respectively. The API returns a JSON body that contains a message describing the result along with standard HTTP status codes.
Management REST API
Root Create region End Point (implemented)
API | Status Code | Response Body |
---|
| GET POST Headers: Authorization | 200Permission Required: DATA:MANAGE Body: | Success Response | "number_of_locators | 3,
"number_of_servers | 8 | "region_url | /geode/v2/regions",
"gateway_receiver_url": "/geode/v2/gwr",
"gateway_sender_url": "/geode/v2/gws"
}
| 200 | 401 | | Error | message | "Missing | authentication | credential | header(s)"
}403 | Code Block |
---|
language | java |
---|
title | Error Response |
---|
|
{
"message": "User1 not authorized for CLUSTER:READ"
} |
Create End Point
"success" : true,
"message" : "success"
}
},
"statusCode" : "OK",
"statusMessage" : "successfully persisted config for cluster",
"successful" : true
}
|
|
409 |
API | Status Code | Response Body |
---|
Endpoint:http://locator:8080/geode-management/v2/regionsMethod: POST
Headers: Authorization
Body:
| | Request Body | regionConfig | | | name | FooENTITY_EXISTS",
"statusMessage" : "cache element |
| "type": | REPLICATE |
| } | 200 |
400 | | Success Error Response - missing required parameter |
| {
"memberStatuses" : { },
|
| | server-1 | {
| success | true,
"Name of the region has to |
| "message" : "success"
}
} | statusCode" : "OK",
"statusMessage" : "successfully persisted config for cluster",
" | true |
409Name conflictError Response - invalid parameter |
| {
"memberStatuses" : { },
"statusCode" : " |
ENTITYEXISTSARGUMENT",
"statusMessage" : " |
cacheelementFoo already exists.may not begin with a double-underscore: __Foo__",
"successful" : false
} |
400 |
401 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
|
| - missing required parameter | {
"memberStatuses" : { },
"statusCode" : " |
| ILLEGAL_ARGUMENTUNAUTHENTICATED",
"statusMessage" : " |
| Name | of | the | region | has to be specified.your credentials",
"successful" : false
} |
|
403 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| - invalid parameter | {
"memberStatuses" : { },
"statusCode" : " | ILLEGAL_ARGUMENTUNAUTHORIZED",
"statusMessage" : " | Regionnamesmaynot begin with a double-underscore: __Foo__",
"for DATA:MANAGE",
"successful" : false
} |
|
401500 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"memberStatuses" : { },
"statusCode" : " |
| UNAUTHENTICATEDERROR",
"statusMessage" : " |
| Authentication | error. | Please | check | your | credentials | " : false
}403 |
Note that the CREATE endpoint is not idempotent, you will receive a 409 when creating the a region with the same name the 2nd time.
401 and 403 responses are omitted for the rest of the end points.
List members end point (Implemented)
API | Status Code | Response Body |
---|
Endpoint:http://locator:8080/geode-management/v2/members Method: GET Headers: Authorization Permission Required: CLUSTER:READ | 200 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"memberStatuses": {},
"statusCode": "OK",
"statusMessage": null,
"result": [{
"class": "org.apache.geode.management.configuration.MemberConfig",
"id": "locator-0",
"host": "10.118.19.10",
"pid": "51876",
"ports": [49251],
"locator": true,
"coordinator": true
}, {
"class": "org.apache.geode.management.configuration.MemberConfig",
"id": "server-1",
"host": "10.118.19.10",
"pid": "51877",
"ports": [49267],
"locator": false,
"coordinator": false
}]
}
|
|
Endpoint:http://locator:8080/geode-management/v2/members?id=server-1 Method: GET Headers: Authorization Permission Required: CLUSTER:READ | 200 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"memberStatuses": {},
"statusCode": "OK",
"statusMessage": null,
"result": [{
"class": "org.apache.geode.management.configuration.MemberConfig",
"id": "server-1",
"host": "10.118.19.10",
"pid": "51877",
"ports": [49267],
"locator": false,
"coordinator": false
}]
}
|
|
Endpoint:http://locator:8080/geode-management/v2/members?id=Non-Existent Method: GET Headers: Authorization Permission Required: CLUSTER:READ | 200 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"memberStatuses": {},
"statusCode": "OK",
"statusMessage": null,
"result": []
}
|
|
Get members end point (Implemented)
API | Status Code | Response Body |
---|
Endpoint:http://locator:8080/geode-management/v2/members/server-1 Method: GET Headers: Authorization Permission Required: CLUSTER:READ | 200 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"memberStatuses": {},
"statusCode": "OK",
"statusMessage": null,
"result": [{
"class": "org.apache.geode.management.configuration.MemberConfig",
"id": "server-1",
"host": "10.118.19.10",
"pid": "51877",
"ports": [49267],
"locator": false,
"coordinator": false
}]
}
|
|
Endpoint:http://locator:8080/geode-management/v2/members/Non-Existent Method: GET Headers: Authorization Permission Required: CLUSTER:READ | 404 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"memberStatuses": {},
"statusCode": "ENTITY_NOT_FOUND",
"statusMessage": "Unable to find the member with id = Non-Existent",
"result": []
}
|
|
Root End Point (Not implemented)
...
...
"number_of_locators": 3,
"number_of_servers": 8,
"region_url": "/geode/v2/regions",
"gateway_receiver_url": "/geode/v2/gwr",
"gateway_sender_url": "/geode/v2/gws"
}
|
|
401 |
...
|
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
" |
|
...
...
: "Missing authentication credential header(s)"
} |
|
403 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "User1 not authorized for CLUSTER:READ"
} |
|
List End Point (not implemented)
{ },
"statusCode" : "ERROR",
"statusMessage" : "cluster persistence service is not running",
"successful" : false
}Note that the CREATE endpoint is not idempotent, you will receive a 409 when creating the a region with the same name the 2nd time.
List End Point
API | Status Code | Response Body |
---|
| 200 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"Total_results": 10,
"Regions" : [
{
"Name": "Foo",
"Url": "/geode/v2/regions/Foo"
},
...
]
} |
|
401 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Missing authentication credential header(s)"
} |
|
403 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "User1 not authorized for CLUSTER:READ"
} |
|
Describe End Point (Not Implemented)
API | Status Code | Response Body |
---|
| 200 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"Name": "Foo",
"Data_Policy": "partition",
"Hosting_Members": [
"s1",
"s2",
"s3"
],
"Size": 0,
"Indices": [
{
"Id": 111,
"Url": "/geode/v2/regions/Customer/index/111"
}
]
} |
|
401 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Missing authentication credential header(s)"
} |
|
403 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "User1 not authorized for CLUSTER:READ"
} |
|
404 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Region with name '/Foo' does not exist"
} |
|
Update End Point (not implemented)
API | Status Code | Response Body |
---|
Endpoint: http://locator:8080/geode-management/v2/regions/Foo Method: PATCH Headers: Authorization Body: Code Block |
---|
language | java |
---|
title | Request Body |
---|
| {
"regionConfig": {
"gateway_sender_id": ["1","2"]
}
} |
| 200 |
Code Block |
---|
language | java |
---|
title | Success Response |
---|
| {
"Metadata": {
"Url": "/geode/v2/regions/Foo"
}
} |
|
400 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Invalid parameter specified"
} |
|
401 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Missing authentication credential header(s)"
} |
|
403 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "User1 not authorized for DATA:MANAGE"
} |
|
404 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Region with name '/Foo' does not exist"
} |
|
500 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Failed to update region /Foo because of <reason>"
} |
|
Delete End Point (Not Implemented)
API | Status Code | Response Body |
---|
| 204 | |
304 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Region with name '/Foo' does not exist"
} |
|
401 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Missing authentication credential header(s)"
} |
|
403 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "User1 not authorized for DATA:MANAGE"
} |
|
500 |
Code Block |
---|
language | java |
---|
title | Error Response |
---|
| {
"message": "Failed to delete region /Foo because of <reason>"
} |
|
Note that the DELETE endpoint is idempotent – i.e. it should be a NOOP if the region does not exist.
Let's look at some code to see how users can use this service. The below example shows how to create a region using CMS.
Curl (any standard REST client)
Code Block |
---|
|
curl [-v] [-u user[:passord]] -H "Content-Type: application/json" http://<locator.host>:7070/geode-management/v2/regions -XPOST -d '
{
"name": "Foo",
"type": "PARTITION"
}'
Sample to copy/paste:
curl -H "Content-Type: application/json" http://localhost:7070/geode-management/v2/regions -XPOST -d '{"name": "Foo","type": "PARTITION"}'
|
Java Client
To ease the interaction with the rest end point, we provided a java client version of Cluster Management Service. Here is an example to get an instance of this service and use it in any java client code. You will need to have geode-management.jar in your classpath.
About the definition of Region type , can refer to the following link:
https://geode.apache.org/docs/guide/18/reference/topics/chapter_overview_regionshortcuts.html
Code Block |
---|
language | java |
---|
title | Java client |
---|
|
public static void main(String[] args) {
String regionName = args[0];
ClusterManagementService cms = ClusterManagementServiceProvider.getService("localhost", 7070);
RegionConfig config = new RegionConfig();
config.setName(regionName);
config.setType(RegionShortcut.PARTITION);
ClusterManagementResult result = cms.create(config, "cluster");
if (!result.isSuccessful()) {
throw new RuntimeException(
"Failure creating region: " + result.getStatusMessage());
}
}
|
The above example is for interacting with a locator's rest end point which has no ssl nor security turned on. To manage a cluster that's has security and ssl enabled, you will need to provide a SSLContext and credentials when getting the service:
Code Block |
---|
language | java |
---|
title | Java Client with security |
---|
|
public static void main(String[] args) {
String regionName = args[0];
SSLContext sslContext = SSLContext.getDefault();
HostnameVerifier hostnameVerifier = new NoopHostnameVerifier();
ClusterManagementService cms = ClusterManagementServiceProvider.getService("localhost", 7070, sslContext, hostnameVerifier, "username", "password");
.....
} |
...