Creating Azure Custom Linux VM Image

Image for post
Image for post
The Packer configuration file (left) and the cloud-init file (right)

Introduction

I previously did an exercise in which I deployed my Clojure Simple Server to Azure managed Kubernetes Service — AKS, you can read about that story in my previous blog posts: “Creating Azure Kubernetes Service (AKS) the Right Way” and “Deploying Kubernetes Configuration to Azure AKS”.

Now that I had some experience how to use Azure AKS as a computing platform I thought that it would be an interesting exercise to create an Azure custom Linux virtual machine image with the Simple Server baked into the image (golden image) and then create an Azure Scale set using Terraform and use that custom VM image to provision servers into the Scale set. In this blog post I describe some of the experiences creating the custom VM image and testing it. In the next blog post I’ll describe how I created the cloud infra for the Azure Scale set using Terraform and testing the application in that environment.

The project is in my Github account in the packer directory.

Packer

I’m using Packer which is quite widely used tool to create custom VM images. I have used Packer previously in the AWS side (see my previous article “How to Create EC2 Images in AWS?”) and now it was interesting to see what it is like to use Packer also in the Azure side. Packer worked smoothly also in the Azure side — I’ll describe how I used Packer in the following chapters in more detail.

Create a Service Principal

You need a service principal that Packer is going to use. You can easily create the service principal e.g. using Azure command line interface:

You get the service principal credentials. It is a good idea to create some environment file in your ~/.azure directory and export those variables there so you don’t need to write them in plain text in your scripts.

Create the Packer Configuration File

I mainly followed Microsoft’s own documentation how to use Packer — “How to use Packer to create Linux virtual machine images in Azure”. I created a Packer configuration file “ss-azure-vm-template.json” (see also the screenshot in the beginning of this article) in which I first define that my base image will be UbuntuServer 18 (which supports cloud-init that I’m going to use later). Then I install a couple of software packages that the server needs (OpenJDK) and I may need during debugging purposes (Emacs, my favorite editor).

My script creates the application distributable as a tar.gz file which I upload to the image using Packer File provisioner.

Finally the template removes the original image user.

How Packer does all this? Packer first creates a temporary Azure resource group and then creates a virtual machine in that resource group using the base image that you define in the Packer configuration file. Then Packer provisions everything according to your Packer configuration file and finally creates a new image from the snapshot of that temporary virtual server state (with all installed packages).

Ok, everything sounds easy like that but of course in real life it’s not like dancing on a bed of roses. You can read in the project’s README.md file that I spent quite a few hours to get all this stuff right. When comparing Docker container image building process to the virtual machine image building process — VM image building takes a really long time. So, do not test e.g. various provisioning in the image building process but in a running virtual server — and once you have verified that all provisioning works, only then configure them as part of the image building process.

Create Cloud-Init Configuration File

Ok. Now we have the new Azure custom Linux VM image. We have baked into the image the application untarred in a directory and also OpenJDK that the application uses as a runtime. Next we should provision the user for running the application (do not run anything as root), and some mechanism to start the application automatically on boot, and also inject in the parameters (e.g. which mode the application is running: single-node test mode or the real thing azure-table-storage mode + the storage account connection string to the Azure table storage which hosts the Tables that the application uses — in a real production system you should store all credentials to Azure Key Vault, of course).

Typically you cannot bake the parameters into the image since some of the information is known only at the time when the infra takes the VM image and starts to create the actual virtual machines in to the runtime environment (which in my case is going to be Azure Scale set). I considered various options and chose an easy way to do this: using cloud-init. Most modern Linux systems support cloud-init. Using cloud-init you can easily create various configurations to be run on the first boot. I created a very simple bash file which I inject to the infra which uses the VM image and the cloud-init script to make the first-time-boot configurations. The cloud-init script “cloud-init-set-env-mode-single-node.sh” (see also the screenshot in the beginning of this file) is pretty simple: It first creates the application start script (“my-start-server.sh”), then it creates the application user (ssuser), changes the ownership of the application directory to the application user and finally creates a rc.local file which calls the start server script on boot. Finally the cloud-init script boots the server so that rc.local will start the server on the next boot.

Once you are ready with all these configuration files you can try to build a test virtual machine using azure command line interface (in the next blog post I describe how I create an Azure Scale set infrastructure using Terraform in which I use this VM image):

The script starts creating the virtual machine and finally tells that the VM is ready and tells some information regarding the VM like the public IP — use the public IP and the ssh key you used previously to logon to the server and check if the application is running:

All right! The server started automatically with the new virtual machine. Now open port 3045 in the virtual server network security group for your local workstation (I already configured port 3045 to be open in the relevant subnet using Terraform, more about that in the next blog post) and test the server once again with my poor man’s Robot framework simulator:

Conclusions

Creating a custom VM image using Packer is pretty much the same in both AWS and Azure. Comparing building Docker container image and VM image the main thing is that building a VM image takes a really long time — you have to test all your provisioning and startup scripts in a live VM before you use them in the image building script.

Written by

I’m a Software architect and developer. Currently implementing systems on AWS / GCP / Azure / Docker / Kubernetes using Java, Python, Go and Clojure.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store