Kubernetes on Mesos on Google Cloud – Version Alignment

A few pointers on how I got Kubernetes to work with Mesos on Google Cloud

Nicki Watt

4 minute read

I was recently trying to follow the instructions in the tutorial by Mesosphere detailing how to run Kubernetes on Mesos on the Google Cloud Platform. The tutorial (At time of writing) did not quite work for me out the box and I had to make a few adjustments to get it working. This simply details a few of the extra steps I needed to take to get this setup working for me. This should hopefully mean that those who find themselves in a similar situation, can avoid running into my same issue upfront, and thus get to a working version faster! The Mesosphere team do update the tutorials periodically, usually when new releases occur, so the official tutorial may well be updated in due course in which case this entry will probably be redundant. But until then, for those who want to try this out now, I hope this can help!

Underlying issue

The underlying cause of all my problems was centred around the fact that I was mixing Mesos versions in various places. In the pre-requisite list in the tutorial, you need to have a Mesosphere cluster on Google Compute Engine. (BTW Google provides you with $300 worth of free trial Google compute resources to play with). You however, are not in control of what version of Mesos is deployed for you by Mesosphere, and this is where I came unstuck. I was blindly following the instructions (centred around 0.21.0 at the time of writing) whilst Mesosphere had provisioned me a 0.21.1 cluster. Apparently the kubernetes-mesos framework is currently using the native Mesos binding compiled against a specific version of Mesos. So if the versions are out of sync, as mine were then issues may arise. My main issue was that my pods were simply not able to run on my minions. They appeared to stay in the “Pending” state forever.

Steps to resolve

The basic resolution strategy is to: Adjust everything in the tutorial to align with whatever Mesos version Mesosphere ultimately provisions for you in the Google cloud.

Practically this is what you need to do:

  • Follow the instructions for setting up your Mesosphere cluster on Google Compute Engine as per normal.

  • Build the Kubernetes-Mesos executables- When you reach this step then:

    • After executing the first (dpkg) command, take note of the Mesos version which has been provisioned for you as you will use this in subsequent steps. For example running the step below I can see the main Mesos version is 0.21.1

      $ dpkg -l | grep -e mesos
ii mesos 0.21.1-1.2.debian77 amd64 Cluster resource manager ...

    • In addition to simply cloning the git repo in the next command, you need to ensure you are using a version of the kubernetes-mesos code compatible with the Mesos version you are actually running against. There is actually a note about this tutorial in the readme in the kubernetes-mesos Github repo stating that this tutorial is not necessarily always kept in sync with the repo, and it is suggested that tutorial users stick with the 0.3.x releases to ensure everything still works. In my case v0.3.2 was the latest kubernetes-mesos release compatible with Mesos 0.21.1

      git clone https://github.com/mesosphere/kubernetes-mesos k8sm
cd k8sm
git checkout v0.3.2

    • The final step under this section is to run the kubernetes-mesos docker image. Once again, modify the image name to use a docker image compatible with the version of Mesos you are running. Note the use of build-mesos-0.21.1-compat docker image, instead of build-mesos-0.21.0-compat

      mkdir -p bin && sudo docker run --rm \ 
-v $(pwd)/bin:/target -v $(pwd)/k8sm:/snapshot \ 
jdef/kubernetes-mesos:build-mesos-0.21.1-compat

  • Run the Example Guestbook App- When you get here:

    • Provided you have checked out the correct git tag in the previous step, the example code should be compatible with your Mesos version. Note however that on the tutorial itself, the link to the example instructions is to an older v0.2.2 version, so if you follow the link, you may be following incorrect instructions.

Thats it! Everything else should work just fine!

comments powered by Disqus