Final Report
Title
Apache Custos Baremetal Deployment
Background
Although Apache Custos is currently deployed on a Kubernetes cluster — its size and complexity may not be ideal for a relatively small application like Custos. This project aims to develop an alternative deployment architecture for Custos that aims to simplify its deployment and backup process.
Design/description of work
I started off by shadowing Isuru’s dev work for merging all Custos Core microservices into one module and Custos Integration services into another module.
https://github.com/apache/airavata-custos/pull/292
- The next step was to design the new deployment architecture. Apache Airavata uses ansible extensively for a variety of tasks. We decided on using ansible scripts to set up and deploy Custos. We decided on 3 servers —
- Custos (the spring boot application runs here, MySQL DB)
I wrote the following roles for Custos environment:
env_setup: to install software dependencies and SSL certs on the server
nginx: configure and start Nginx reverse proxying on the server
database: set up and start MySQL DB used by Custos
zookeeper: set up and start zookeeper
kafka: set up and start zookeeper
custos: set up and start Custos core and integration services. - Keycloak (PostgreSQL DB)
Along with the env_setup and nginx roles that are common across the 3 servers, I created the Keycloak role that includes the PostgreSQL configuration along with the Keycloak application configuration. It also starts the PostgreSQL and Keycloak servers. - Vault (Hashicorp Consul DB)
Just like Custos and Keycloak servers — I used the env_setup and nginx roles for the Vault server too. Additionally, I created the consul role and the vault role to configure and start hashicorp consul and hashicorp vault.
https://github.com/apache/airavata-custos/pull/290
- In the next phase of development, I wrote migration scripts for the 3 DBs. The migrate_db role is run on each of the old servers and the new servers are run with the migrate_db tag.
https://github.com/apache/airavata-custos/pull/304
- This DB migration was made optional by setting up the migrate_db tag. Here’s the latest PR:
https://github.com/apache/airavata-custos/pull/305
The DB migration process for Vault needs additional work to load the data into the consul server accurately. Additionally, I am working on an academic paper comparing Apache Custos’ baremetal deployment architecture with that of the existing Kubernetes deployment cluster.
I loved the GSoC experience with the Apache Airavata group and I will cherish the bonds that I have formed with my mentors and other contributors.