Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Apache Bigtop is managing its own setup of CI system. We have 1 master + 4 slaves total. There's 3 * s one 1T storages storage assigned to master, and another 2 * 1T 800G go to two of the slaves (06 and 07), respectively. Other two nodes (02 and 03) got 40G got 200G for each node to operate therefore they are reserved to run non storage sensitive jobs such as provisioner.

Table of Contents

Setup a Jenkins master

...

If you already have a managed Jenkins/Hudson, or you prefer your all installation, skip and go to Setup Bigtop packages build matrix

...

And the needed Plugin(s):

  • git plugin

Setup Jenkins slaves

Assuming that you are going to add or replace an Amazon EC2 instance which have already been created with Amazon Linux 2 AMI, execute the following commands in that node:

No Format
// Install Docker and Docker Compose (execute as ec2-user)
$ sudo yum update -y
$ sudo yum install -y docker git java ruby
$ sudo amazon-linux-extras install -y docker  # to make sure the latest version of docker is installed
$ sudo service docker start
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose

// Create jenkins user and setup SSH auth so that Jenkins master can connect to this node
$ sudo adduser jenkins
$ sudo usermod -aG docker jenkins
$ sudo -i
# su - jenkins
$ mkdir .ssh
$ cat << EOF > .ssh/authorized_keys
> (MASTER_PUBLIC_KEY_HERE)
> EOF
$ chmod 700 .ssh
$ chmod 400 .ssh/authorized_keys

// Make sure Docker and Docker Compose work for jenkins user, and login to
// Docker Hub so that Docker-related Jenkins jobs can push generated images
$ docker run hello-world
$ docker-compose version
$ docker login --username bigtop

Then add the node or update its configuration (e.g., FQDN) via Jenkins. In the latter case, you may have to disconnect and relaunch agent so that Jenkins recognizes the change.

Setup Bigtop packages build matrix

...