...
If necessary Airavata ADMIN users will also be able to maintain application deployments across gateways. Applications and workflows will always be deployed, created (workflows will be created using XBAYA), etc... prior to entering in the Application Catalog.
In Airavata there are two ways of executing experiments in supercomputers/ computational resources (Trestles, Stampede, Amazon EC2, Big Red II, etc…).
Execute applications - Single submission experiments
Execute workflows - Multiple node OR single node experiments
...
experiment
To execute experiment jobs of either above types; we rely on ‘Descriptors’ in Airavata to provide information required on application, application hosted supercomputers and input output parameters of the application.
There are three types of descriptors;
...
Host Descriptors
Specifies the application residing/hosting resource information. GRAM, GridFTP, Amazon EC2, etc….
Service Descriptors
Input and output parameter information for an application
Application Descriptors
Specifies executable location information, deployment location information, scratch locations of output files, meta data specific for the application, meta data specific for the deployment (location where the application is deployed in the resource), etc…
...
So what is more with Application Catalog? With application catalog user will be able to;
Store same set of information as with descriptors
Have number of new API methods to access and manage the applications as required
Create Application
Publish Application
Search/List Application
Update Application
Clone Application
Import Application
Delete Application
Turn On/Turn Off Application
Read from Application Catalog - Other components will read from the Catalog in order to get relevant information required for experiments.
In descriptors we have same set of parameters captured for clouds, supercomputers irrespective of the necessity. With the Catalog ‘the not required parameters’ will be ignored when creating, updating application information.
Airavata components; Orchestrator, Workflow Interpreter & GFac will connect to Application Catalog to obtain information required for executing jobs at resources.
Back-end services of Application Catalog. These will be automated services which would run periodically as specified by the Airavata ADMIN users.
Verifying application location in the resource(s) and actual existence
Verifying application version in the resource
Status | NOTE |
---|---|
CREATED | When created initially, when cloned or imported the new application or workflow instance will have the status CREATED. Created application, workflow can be updated and saved in CREATED status |
PUBLISHED | Applications and workflows which are CREATED will be PUBLISHED. Published application, workflow can be updated and saved in PUBLISHED status. Only applications & workflows with status PUBLISHED will be turned OFF. Applications and workflows which are turned OFF will only be turned ON. |
Create application includes many sub tasks in the Application Catalog. Creating an application instance extends to;
Entering application specific information such as;
Application version details
Deployment visibility to gateway users
Defining application specific parameters
Defining resource specific parameters required for the application execution
Define input and output data file sizes, file locations and scratch folder locations and also their folder sizes, etc for the application
Error file locations, log file locations ,etc…. for an application in residing resource
Application user group (Application is available only for a selected user group(s)
Maximum wall-time, number of processors, nodes per processor, etc….
Entering resource specific information such as;
Deployed resource/supercomputer information
Defining resource specific parameters such as application residing location
Queue information
Deployment information such as using MPI, GPU, Serial, etc….
Creating workflow consists;
Enter workflow information & resources where the workflow will be executed
Define workflow nodes and applications to be used at each nodes in the workflow
Defining subsequent input and output files
Application & workflow creation in the catalog will be a gateway ADMIN task.
Applications and workflows can be created by cloning & importing existing applications and workflows.
Application deployments and workflows which are CREATED will be published in order to be used by gateway users.
Publishing applications and workflows is a gateway ADMIN user level task.
At the time of publishing applications the system would validate;
Existence of at least 1 deployment of the application
Existence of at least 1 application and resource level parameters
…
At the time of publishing workflows system should validate;
Existence of at least one application in the workflow
Existence of at least one node in the workflow
...
User can search within created applications in order to publish by providing;
Application ID/Workflow ID (Generate ID within Application Catalog)
Application Name
Resource
Application User Group