jenkins spinup example, use at your own risk
Use this repo to configure a stateless jenkins instance from scratch on a remote linux box for building and testing android apps.
These instructions (and the scripts in this repo) assume you are using a mac and have a github account for oauth access to jenkins. A basic familiarity with linux commands would be pretty helpful to understand what’s going on, maybe briefly research and try out the following commands: cd, ls, echo, touch, cat, rm, chmod
First clone this repo. All the files we create need to be placed inside the top level direcory of the cloned repo (i.e. inside the jenkins-spinup folder)
To do this on the command line:
git clone git@github.com:erdo/jenkins-spinup.gitcd jenkins-spinup
There are a few things we need to specify inside the jenkins.env file before running the script. For example the github user that we want to be the initial admin of the jenkins instance. We also need to specify the github oauth app details (if we don’t have a github oauth app, we need to create one using the github web ui first, go to: Settings > Developer settings > OAuth Apps).
Create a file and name it jenkins.env then edit the contents of the file so it looks something like this:
GITHUB_ADMIN_USER=mygithubusername
GITHUB_OAUTHAPP_CLIENT_SECRET=05ag56f38myoauthappclientsecret73fa7765e0
GITHUB_OAUTHAPP_CLIENT_ID=103myoauthappclientid55be
To do this on the command line:
echo "GITHUB_ADMIN_USER=mygithubusername
GITHUB_OAUTHAPP_CLIENT_SECRET=05ag56f38myoauthappclientsecret73fa7765e0
GITHUB_OAUTHAPP_CLIENT_ID=103myoauthappclientid55be" > jenkins.env
You can check this worked by typing:lscat jenkins.env
We next need to get hold of a linux box, we want something like Ubuntu Server 18.04, get it from Amazon EC2 or DigitalOcean or elsewhere.
Make sure to download the private ssh key of the remote linux box so that we can ssh into it later, we’ll refer to that private key file as linuxbox_ssh.key (ssh doesn’t care what you call this file, it’s just a text file and it’s the contents that matters, not the extension).
I have noticed that around 2G RAM seems to be the bare minimum spec that docker / jenkins will run with. Docker seems very space intensive and probably a 10G hard disk would be a minimum (potentially much more if you want to add more android SDKs to the docker image).
You’ll most likely need to setup security rules so that the box is inbound accessible for Custom-TCP(8080) for administrating jenkins / github hooks, and for SSH(22) for accessing the box over ssh.
some help setting this up on Amazon:
https://d1.awsstatic.com/Projects/P5505030/aws-project_Jenkins-build-server.pdf
some help setting this up on DigitalOcean:
https://www.youtube.com/watch?v=-exFANqwlPM
For running private tests and development, a local ubuntu instance is very handy, and multipass can give you multiple free local ubuntu instances.
download and install it: https://multipass.run/
create your own key pair: echo | ssh-keygen -t rsa -b 4096 -C "ssh key pair, mac to linux box" -f "linuxbox_ssh.key" -P '';
create an ubuntu instance: multipass launch --name furry-packet (you can omit the name and multipass will choose one for you)
view ip address: multipass list
push public key to ubuntu instance (note the backtick: `, not \’ ):multipass exec furry-packet — bash -c “echo `cat ../config/linuxbox_ssh.key.pub` >> ~/.ssh/authorized_keys”
test ssh access: ssh -v -i ../config/linuxbox_ssh.key ubuntu@xx.xx.xx.xx
The setup script(s) do a number of things, including transfering the jenkins.env file to our remote linux box, applying security updates, setting up various ssh key pairs, installing and configuring Docker and Jenkins.
We can kick the whole process off by running 01_mac.sh from this repository (it will call the rest of the scripts itself, 02_ubuntu.sh and 03_docker.sh), you may need to make the first script file executable manually first:
chmod u+x 01_mac.sh
During the running of this script, you will occasionally be asked to copy and paste public key texts into the github web ui, just go to: Settings > SSH and GPG keys > New SSH key it doesn’t matter what you call them
Once the script knows the url of your jenkins instance, it will ask you to set the Authorization callback URL in the github oauth app which you specified in the jenkins.env file in Step 1. Go to: Settings > Developer settings > OAuth Apps > {your app} and update it there
Apart from that, occasionally there will be a software update that needs to be applied that the script was not expecting, in that case you will be asked to confirm the installation (usually by typing yes) before the script will continue
You will get a minute or so to update the public SSH keys in github before the script moves on, so it’s probably best to log into github first (if you miss the chance, don’t worry, either scroll up or just run the script again - it will use the same public key files unless you manually go in and delete them). To run the script:
./01_mac.sh ubuntu@ec2-blah-blah-blah.eu-west-3.compute.amazonaws.com
The script sets up secure communication between different parts of the system without using passwords (which would have to be backed up and/or stored somewhere):
For each case the script needs to create an ssh key pair (a private key file, and a public key file).
The script handles everything for you except the copying and pasting bit, which it will prompt you to do.
All sorts of things can go wrong, but the script tries to anticipate problems where it can, it will tell you if there are files missing or if you are not calling the script correctly.
However the main feature for ensuring the script runs to completion is that you can run it multiple times without causing any problems, it either skips over steps that it already completed, or re-performs steps where it causes no harm.
In particular, if an ssh key pair has already been created, it won’t recreate it again - there is no need to replace the previous public ssh key you pasted in github, it won’t have changed (unless you manualy ssh into the linux box and delete the generated key files yourself)
To diagnose an issue:
Here’s how to ssh manually into the linux box:
ssh -v -i linuxbox_ssh.key ubuntu@blah-blah-blah.eu-west-3.compute.amazonaws.com
Unfortunately the scripts are not unit tested, so in theory it would be fairly easy to break them structurally. They expect various files and environment variables to be spelled in a specific way across different config files and scripts, so if you change the name of something in one script, make sure you check all the other files in the repo for other places you might also need to change.
The scripts are heavily commented to help people who aren’t familiar with bash scripting to see what is going on.
Don’t forget that the spinup process pulls itself from github on to the linux box, so if you make any changes on your mac, you need to commit and push those changes to github before the linux box will see any of the changes.
This causes another complication (mentioned briefly in the Diagnosing problems section) by default the second script clones the repo and continues from the master branch, if you are developing in a feature branch it’s probably easiest to uncomment the line in 02_ubuntu.sh below where it says “# switch to feature branch if developing script changes”. This way you can test everything works before re-commenting that line and merging into the master branch.
Aside from the bash scripts, here are some important files you will likely want to change for your own purposes:
jenkins-pluggins.txt - this is just a list of all the pluggins you want pre-installed on your jenkins instance, add whatever you want here
casc.yml - this is the main config file for jenkins configuration as code, you will certainly want to change the seed job and add further seed jobs in the job-dsl plugin section, the seed jobs are designed to find further instructions inside a Jenkinsfile that should be present in the repository being cloned for the job
Dockerfile - is where you will find the image definition that jenkins runs in, we start with a plain alpine linux image with jenkins installed on it and add various things to it, including the android command line tools and android build tools (which you will need to change periodically when newer versions become available) the more you add here of course, the larger the final image will be.
Generally I have chosen to always update the latest versions of everything where possible because this makes keeping up with security updates the default - but that of course increases the chance that one day the script will stop working and need to be fixed because an update broke something.