Page History

...

• Create a Kubernetes cluster with desired size with different control parameters alongwith multi-master, HA support.

• Add and manage multiple Kuberneter Kubernetes versions with binaries ISO.

• Offline installation of Kubernetes and docker binaries on cluster nodes for selected Kubernetes version.

• Manage cluster, allowing it to be scaled up or down for the number of cluster nodes. Upgrade cluster to newer Kubernetes version.

• Access kube.config for cluster for accessing Kubernetes dashboard.

• Delete cluster while removing nodes and Kubernetes services.

Document History

...

References

Pull request:

https://github.com/apache/cloudstack/pull/3680

References

Pull request:

Use Cases

Plugin will provide Kubernetes integration with CloudStack enabling user to run containerized services using Kubernetes clusters.

...

CloudStack Kubernetes Service plugin will add Kubernetes integration in the CloudStack. Plugin The plugin will be disabled by default and admin can control its usage using the global setting. It will enable run containerized services using Kubernetes clusters.

CKS will use the CoreOS based template for node VMs for running Kubernetes services. For installation of Kubernetes binaries on cluster nodes, a binaries ISO will be used to achieve faster, offline installation. Root admin will be provided with API and UI to upload/register such ISOs for a particular Kubernetes version. A shell script will be added in the codebase to easily create such ISO for a Kubernetes version.

CoreOS had been chosen for cluster node VM operating system as it already provided prerequisite Docker and networking for deploying Kubernetes cluster. Future releases of CKS will try to incorporate the use of a user-provided guest OS.

For deployment and setup Kubernetes on cluster nodes, plugin will use Kubernetes tool, kubeadm. kubeadm is the command line tool for easily provisioning a secure Kubernetes cluster on top of physical or cloud servers or virtual machine. Under the hood, the master node(s) of the cluster will start a Kubernetes cluster using kubeadm init command with a custom token and worker nodes will be able to join this Kubernetes cluster using kubeadm join command with the same token. More about kubeadm, https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/

To access Kubernetes dashboard securely, the plugin will provide access to kube.config file data which would allow using Kubernetes tools such as kubectl to run proxy locally and thereby accessing the dashboard. More about kubectl, https://kubernetes.io/docs/reference/kubectl/overview/

...

name = (string) the name of the Kubernetes supported version. (required)

kubernetesversion semanticversion = (string) the semantic version of the Kubernetes. (required)url = (string) the URL to where the ISO is currently being hosted. (required)

zoneid = (uuid) the ID of the zone in which Kubernetes supported version will be available.

isoid url = (uuidstring) the ID URL of the binaries ISO for Kubernetes supported version. (required)

checksum = (booleanstring) the checksum value of the binaries ISO.url

mincpunumber = (stringinteger) the URL of the binaries ISO for Kubernetes supported version.minimum number of CPUs to be set with the Kubernetes version. (required)

minmemory = (integer) the minimum RAM size in MB to be set with the Kubernetes version. (required)

• deleteKubernetesSupportedVersionaddKubernetesSupportedVersion

Delete a supported Kuberneted version

...

id = (uuid) the ID of the Kubernetes supported version.deleteiso = (boolean) true if ISO associated with the

Kubernetes

...

Kubernetes supported version management related APIs can be accessed by all users

• listKubernetesSupportedVersions
  List Kubernetes supported versions
  
  Parameters
  ============
  id = (uuid) the ID of the Kubernetes supported version

  minimumsemanticversion = (string) the minimum semantic version for the Kubernetes supported version to be listed

  minimumkubernetesversionid = (uuid) the ID of the minimum Kubernetes supported version

...

• listKubernetesCluster
  List Kubernetes cluster(s)
  
  Parameters
  ============
  id = (uuid) the ID of the container cluster
  state = (string) state of the container cluster
  name = (string) name of the container cluster
  
  
  
• createKubernetesCluster
  Creates a cluster of VM's for launching Kubernetes containers.
  
  Parameters
  ============

  name = (string) name for the container cluster (required)

  description = (string) description for the container cluster

  zoneid = (uuid) availability zone in which container cluster to be launched (required)

  kubernetesversionid = (uuid) the ID of the Kubernetes version with which cluster to be created (required)

  serviceofferingid = (string) the ID of the service offering for the virtual machines in the cluster (required)

  accountid = (uuid) an optional account for the virtual machine.

  projectid = (uuid) deploy cluster for the project

  networkid = (uuid) network in which container cluster is to be launched

  sshkeypairname = (string) name of the ssh key pair used to login to the virtual machines sshkeypair

  masternodes = (integer) number of Kubernetes cluster master nodes when setting up multi-master (default: 1)

  externalloadbalanceripaddress = (string) IP address of an exter external load balancer when using multi-master on a shared network

  size = (integer) number of container cluster nodes (required)

  noderootdisksize = (long) root disk size of root disk for each node

...

• upgradeKubernetesCluster
  Upgrade Kubernetes cluster to a given newer version
  
  Parameters
  ============
  id = (uuid) the ID of the container cluster

  kubernetesversionid = (uuid) the ID of the Kubernetes version

...

1. Table for CKS cluster
   kubernetes_cluster:
   id (bigint, auto_increment),
   uuid (varchar, unique),
   name (varchar, unique),
   description (varchar),
   zone_id (bigint),
   service_offering_id (bigint),
   template_id (bigint),
   network_id (bigint),
   node_count (bigint),
   account_id (bigint),
   domain_id (bigint),
   state (enum/string),
   key_pair (varchar),
   cores (bigint),
   memory (bigint),
   endpoint (varchar),
   created (datetime)
   removed  (datetime)
   
   
   

2. Table for CKS cluster details
   kubernetes_cluster_details:
   id (bigint, auto_increment),
   cluster_id (bigint),
   name (varchar),
   value (varchar)
   
   
   

3. Table for CKS cluster VM reference
   kubernetes_cluster_vm_ref:
   id (bigint, auto_increment),
   cluster_id (bigint),
   vm_id (bigint)
   
   
   

4. Table for CKS Kubernetes version
   kubernetes_supported_version:

   id (bigint, auto_increment),

   uuid (varchar, unique),

   name (varchar),

   kubernetessemantic_verion (varchar),name

   iso_id (varcharbigint),iso

   zone_id (bigint),zone_id (bigint

   state (char),

   min_cpu (int),

   min_ram_size (int),

   created (date),

   removed (date)

UI Changes

List of currently created clusters will be show shown in UI, in ui/plugins/cks.js.

UI will allow the user to add, manage supported Kubernetes versions.

UI will allow the user to perform lifecycle tasks on the cluster. It will show a tab-based interface to provide cluster details, running VM instances, network rules and steps to access Kubernetes dashboard using kubectl tool.

Lifecycle

Testing

Marvin tests cases

    CRUD tests:
        Add cluster, list clusters to confirm addition
        Delete cluster, list clusters to confirm deletion
        Scale cluster, list clusters to confirm scaling
    Cluster management tests:
    Start cluster
    Stop cluster
    Get cluster config

...