Status
Principle
- Minimize user operations for managing and monitoring topologies
- REST Service centralized metadata management
- Consistent UI experience
Objective
- Make sure consistent semantics between the term application (user space) and topology (physical space only)
- Topology configuration can be edited on UI
- Use REST Service as only entry to manage application (START/STOP) and ApplicationManager to schedule the task execution and update the execution information
- Eagle UI should support to manage/monitor the application on site/application page
- Single site/application may have more than one topologies (for example hdfsAuditLogMonitoring and userProfileMonitoring should have topologies running separately)
Architecture
To have a better user experience with EAGLE ui, we could provide an interface to manage a certain topology via the ui page, such as submit a topology
Gliffy Diagram |
---|
| |
---|
name | ApplicationTopologyArch |
---|
|
Schema
Topology Description Service
- Service name: TopologyDescriptionService
- Entity: TopologyDescriptionEntity
- Table name: eagle_metadata
- Prefix: topologyDescription
| Attribute | Type | Description |
---|
tage | topology | String | topology name |
fields | exeClass | String |
...
topology entry class |
| type | String | topology type: DSL or CLASS |
| description | String | description on this topology |
| version | String | topology version |
Topology Execution Service
- Service name: AppManagementServiceTopologyExecutionService
- Entity: TopologyExecutionEntity
- Table name: application_definitioneagle_metadata
- Prefix: topologyExecution
- Description: define the relation between an application and an topology, and maintain the execution status as well
| Attribute | Type | Description |
---|
tags | site | String | topology site name |
| appNameapplication | String | |
| topology | String | application/topology name |
fields | mainClassfullName | String | topology execution name: eagle_${site}_${application}_${topology} |
| url | String | topology tracking urlapplication main class |
| jarFiledescription | String | jar file pathtopology running status description |
| configFilestatus | String | application configuration file pathrunning status {NEW, STARTING, STOPPING, STARTED, STOPPED} |
| deployEnvironmentmode | String | cluster nametopology running mode: cluster or local |
| statusenvironment | String | {SUCCESS, FAILURE} |
...
topology execution environment, e.g., storm |
| lastModifiedDate | long | last status update time |
Topology Operation Service
- Service name: ClusterDefinitionServiceTopologyOperationService
- Entity: TopologyOperationEntity
- Table name: cluster_definitioneagle_metadata
- Prefix: topologyOperation
| Attribute | Type | Description |
---|
tags | nameoperation | Stringcluster name | {START, STOP, STATUS} |
| typesite | String | e.g., storm, spark |
| configapplication | String | cluster config |
...
- NEW
- create a topology definition entity
- START
- click START button
- check the topology's current status on the nimbus host
- a new thread is created to execute the submitting topology task
- update status
- STOP
- click STOP button
- check the topology's current status on nimbus host
- a new thread is created to stop the topology
- update status
operationID | String | |
| topology | String | topology name: ${topology} |
fields | status | String | {INITIALIZED, PENDING, FAILED, SUCCESS} |
| message | String | exception message |
| lastModifiedDate | long | last status update time |
Customized Restful Apis
HTTP Method | URL | Payload | Description |
---|
POST | /app/operation | TopologyOperationEntity | create an topology operation |
DELETE | /app/topology/{topology} | | delete a topology description |
...