Status
Principle
- Minimize user operations for managing and monitoring topologies
- REST Service centralized metadata management
- Consistent UI experience
...
- Make sure consistent semantics between the term application (user space) and topology (physical space only)
- Topology should read configuration from SiteApplicationEntity#config instead of configuration fileconfiguration can be edited on UI
- Use REST Service as only entry to manage application (START/STOP/HEALTH CHECKING) and ApplicationManager to schedule the task execution and update the execution information
- Eagle UI should support to manage/monitor the application on site/application page
- Single site/application may have more than one topologies (for example hdfsAuditLogMonitoring and userProfileMonitoring should have topologies running separately)
...
To have a better user experience with EAGLE ui, we could provide an interface to manage a certain topology via the ui page, such as submit a topology
| Gliffy Diagram |
|---|
| |
|---|
| name | ApplicationTopologyArch |
|---|
|
| Gliffy Diagram |
|---|
| |
|---|
| name | topologyManagetopologyManagerArch |
|---|
|
Schema
...
Topology Description Service
- Service name: TopologyDescServiceTopologyDescriptionService
- Entity: TopologyDescriptionEntity
- Table name: eagle_metadata
- Prefix: topology_definitiontopologyDescription
| | Attribute | Type | Description |
|---|
| tage | nametopology | String | topology desc name |
| fields | mainClassexeClass | String | application main topology entry class |
| | type | String | topology type: build-in topology or main-class topology |
| | config | String | topology config |
| DSL or CLASS |
| | | description | String | description on this topology |
| | version | String | topology version |
...
Topology Execution Service
- Service name: TopologyExecutionService
- Entity: TopologyExecutionEntity
- Table name: eagle_metadata
- Prefix: topology_executiontopologyExecution
- Description: define the relation between an application and an topology, and maintain the execution status as well
| | Attribute | Type | Description |
|---|
| tags | site | String | topology site |
| | application | String | application |
| | nametopology | String | topology desc name |
| fields | idfullName | String | global unique topology idexecution name: eagle_${site}_${application}_${topology}_${random_hashcode} | fields |
| | url | String | application topology tracking url |
| | deploydescription | String | deploy environment topology running status description |
| | status | String | application running status {INITIALIZEDNEW, STARTING, STOPPING, STARTED, STOPPING, STOPPED} |
| | mode | String | topology running mode: cluster or local |
| | environment | String | topology execution environment, e.g., storm |
| | lastModifiedDatelastUpdateTime | long | last status update time |
Use cases
...
- create a topology definition entity
...
- click START button
- check the topology's current status on the nimbus host
- a new thread is created to execute the submitting topology task
- update status
Topology Operation Service
- Service name: TopologyOperationService
- Entity: TopologyOperationEntity
- Table name: eagle_metadata
- Prefix: topologyOperation
| | Attribute | Type | Description |
|---|
| tags | operation | String | {START, STOP, STATUS} |
| | site | String | |
| | application | String | |
| | operationID | String | |
| | topology | String | topology name: ${topology} |
| fields | status | String | {INITIALIZED, PENDING, FAILED, SUCCESS} |
| | message | String | exception message |
| | lastModifiedDate | long | last status update time |
Customized Restful Apis
| HTTP Method | URL | Payload | Description |
|---|
| POST | /app/operation | TopologyOperationEntity | create an topology operation |
| DELETE | /app/topology/{topology} | | delete a topology description |
...
