Deploy docker-machine

We have our fork of docker-machine available at http://gitlab.com/gitlab-org/ci-cd/docker-machine which we use for GitLab.com Linux shared runners. A list of releases can be found in https://gitlab.com/gitlab-org/ci-cd/docker-machine/-/releases.

Deploying a new version of docker-machine requires no downtime, and only requires a chef role change.

Rollout

To deploy a new version of docker-machine open a Criticality 3 Change Management issue, which should specify the rollout strategy below.

Prerequisites

1. Determine which version to deploy by looking at https://gitlab.com/gitlab-org/ci-cd/docker-machine/-/releases.
2. Determine the checksum for docker-machine-Linux-x86_64 binary by download the release.sha256 from the index page of the release.

Update prmX

1. Create a merge request to add override_attributes inside of roles/gitlab-runner-prm.json to specify a different version of docker-machine.

   roles/gitlab-runner-prm.json

   "override_attributes": {
     "cookbook-gitlab-runner": {
       "docker-machine": {
         "version": "0.16.2-gitlab.9",
         "source": "https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.9/docker-machine-Linux-x86_64",
         "checksum": "75522b4a816c81b130e7fb6f07121c1d5ea4165c4df5fbf05663eac88b797f02"
       }
     }
   },

2. Merge the merge request.

3. Run the apply_to_prod job on the merge commit.

4. Run chef-client on the nodes: knife ssh -C2 -afqdn 'roles:gitlab-runner-prm' -- 'sudo -i chef-client'.

5. Monitor the following:

   1. runner_system_failure metrics
   2. ci-runner apdex

Update rest of the runner fleet

1. Create a merge request

   1. Remove the override_attributes.cookbook-gitlab-runner.docker-machine from roles/gitlab-runner-prm.json that was added previous steps.

   2. Update default_attributes.cookbook-gitlab-runner.docker-machine inside of roles/gitlab-runner-base.json to the version that you want to update the rest of the runner fleet.

      roles/gitlab-runner-base.json

      "default_attributes": {
        ...
        "cookbook-gitlab-runner": {
          "docker-machine": {
            "version": "0.16.2-gitlab.9",
            "source": "https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.9/docker-machine-Linux-x86_64",
            "checksum": "75522b4a816c81b130e7fb6f07121c1d5ea4165c4df5fbf05663eac88b797f02"
          },
          ...
        }
      },

   3. Update default_attributes.cookbook-gitlab-runner.docker-machine inside of roles/org-ci-base-runner.json to the version that you want to update the rest of the runner fleet.

      roles/org-ci-base-runner.json

      "default_attributes": {
        ...
        "cookbook-gitlab-runner": {
          "docker-machine": {
            "version": "0.16.2-gitlab.9",
            "source": "https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.9/docker-machine-Linux-x86_64",
            "checksum": "75522b4a816c81b130e7fb6f07121c1d5ea4165c4df5fbf05663eac88b797f02"
          },
          ...
        }
      },

2. Merge the merge request.
3. Run the apply_to_prod job on the merge commit.
4. Run chef-client on all the nodes: knife ssh -C2 -afqdn 'roles:gitlab-runner-base OR roles:org-ci-base-runner' -- 'sudo -i chef-client'.
5. Monitor the following:
   1. runner_system_failure metrics
   2. ci-runner apdex