Apache Airavata
MFT Agent deployment and lifecycle automation in distributed environment
Mentors: Suresh Marru, Dimuthu Wannipurage
Original Proposal Summary:
Apache Airavata MFT is a distributed high-performance, multi-protocol data transfer engine to orchestrate data movement and operations across most cloud and on-premises storages. Agent is the core of this distributed system which does the heavy lifting of transfer jobs. Right now, agents have to be manually deployed and configured to start the transfer. This is a problem for researchers and people who need to transfer data without having much technical skills or time. The proposed solution is to automate the process of deploying and lifecycle management of the agents when a transfer job is submitted from an edge machine. Upon requirements evaluation, along with my mentor I came up with an updated proposal including the cybershuttle agent to MFT agent.
Updated Proposal Summary:
Updated proposal includes creating a desktop application and cybershuttle server to communicate with each other. This will guide cybershuttle agent and MFT agent from cybershuttle server to perform execution of jobs and data transfer, this would also maintain the lifecycle of cybershuttle agent and mft agents.
Architecture of the Solution:
The architecture of this solution contains below major components with technologies used:
Workflow:
The total workflow of finished product is as follows:
Results:
The project is successfully tested with automation of all the workflow of agent management from launching to stopping which is always a click away. This could potentially provide time savings for the user in the magnitudes of hours to days depending on the complexity of the app/agent/code setup, prerequisites, ability to run etc.
Deliverables with respective Pull Requests:
Desktop application:
Cybershuttle app server:
Orchestration Agent:
Communication threads:
Most of the communication with mentors was over slack, video conference and personal chat. Though a few milestones in the process are documented over community threads as follows:
Future development opportunities:
Thank you for going through the project. My information is as below.
BIOGRAPHICAL INFORMATION:
Name: Praneeth Kumar Chityala
Email: praneethchityala44@gmail.com, pkchitya@iu.edu
Education :
August 2021 - May 2023 | Indiana University Bloomington |
August 2012 - May 2016 | National Institute of Technology, Warangal (NITW) B.Tech |
Programming Languages: Java, Python, Javascript, SQL, HTML5
Technologies/Frameworks: Spring, SpringBoot, GCP, AWS, Microservices, Docker, Nomad, Consul, Kubernetes, React, Vuejs, electonJS, gRPC, MongoDB, PostgreSQL, MySQL, Heroku, Jenkins, Maven, JUnit, Mockito, Git, YourKit, Log4j