Building a modern system testing application

So, part of the project that led to Seymour (my drive geometry script) picked up yesterday — we are building a simplified system test suite that runs over a PXE image.

Right now, we have a Ruby script that uses the Sinatra gem to generate a web page that we can use to monitor the progress. It’s fairly simple so far – it displays the status of a memory testing tool, and we’re implementing hard drive testing soon, but it’s a good aesthetic, and it’s useful – plus it is way, way cheaper than our current solution.

I learned today how to do forking in Ruby. Beyond being really, really easy, it is really useful for something like this – we don’t want the application to be held up because we’re waiting for the memory test to complete.

To do the forking, I wrote the following:
memstatus = "Testing in Progress"
fork do
memstatus = system("sudo memtester #{memTestAmt.to_s} 1").passfail
exit
end

I’m using tempfiles to store the results, which is fortunately a nice clean package in Ruby.

So, we fork for the memory test and again for the hard drive test, then use the result to determine the content of the page. Seems pretty simple, I guess? It’s been interesting so far, even for a simple application that does so many hacky things.

Drive Geometry Specs in Clonezilla

Yesterday, I posted about automating drive geometry in Clonezilla.  I realized that the actual information is somewhat lacking, so here’s an explanation about how this all fits together.

sda-pt.sf is the only file that I’ve found to be strictly necessary to change drive geometry between images. sda-pt.sf contains the partition table for device sda, appropriately enough.

Here’s an example of sda-pt

label: dos
label-id: 0xf0a1cc41
device: /dev/sda
unit: sectors

/dev/sda1 : start= 2048, size= 204800, type=7, bootable
/dev/sda2 : start= 206848, size= 142272512, type=7
/dev/sda3 : start= 142479360, size= 13821952, type=7

The first four lines aren’t particularly noteworthy. It’s a DOS partition table, it came from /dev/sda (the first disk in a system, essentially), and the numbers used are specified in sectors of the disk (usually 512 bytes per sector).

What’s really interesting here is the last three lines. These are the actual geometry specs, and the .sf format specifies start and a size.

Before we begin digging into the process to resize, I want to point out one small factor that left me a little confused at first. Note how, for example, sda1 (first partition) has a size of 204800 (exactly 100MB, coincidentally), and starts at 2048, while sda2 starts as 206848. While this may seem intuitive to some, it had me a little confused at first, because I forgot that the start sector is included in the size.

Moving on, let’s get into the real fun – calculating the new partition tables from this. In this example, we want the second partition to grow, since this image is for a Windows 7 machine. The first partition is the system partition, third partition is recovery, so we don’t need to worry about those growing.
486 178 048
For this example, we’ll make the geometry spec for a 256GB drive, which we figure will have 500,000,000 sectors.

We start from the end and work our way back on this. sda3 is 13,821,952 sectors, so the start point should be (500,000,000 - 13,821,952) = 486,178,048

We can change the line for sda3 to the following: /dev/sda3 : start= 486178048, size= 13821952, type=7

Since sda2 is our primary partition, we now need to calculate its new size. This, fortunately, is a simple matter of subtracting the start of sda3 from the start of sda2. In this instance, 486178048 - 206848 = 485,971,200.

This means we can change the line for sda2 to the following: /dev/sda2 : start= 206848, size= 485971200, type=7

The first partition doesn’t change at all, so we now have a new geometry specification:

/dev/sda1 : start= 2048, size= 204800, type=7, bootable
/dev/sda2 : start= 206848, size= 485971200, type=7
/dev/sda3 : start= 486178048, size= 13821952, type=7

Please be sure to keep types the same, though – that will break things.

That’s it for today, though! Hope you got something useful out of this, and feel free to ask questions!

Creating multiple images from a single base

At work, we send out multiple PCs per day, imaged across a range of models.  Within each model, we have a range of HDD sizes.

I was asked to make a networked imaging server, and that’s been relatively easy (I’ll write on that later), using DRBL and Clonezilla.

We have a few issues within our use case, though – Clonezilla can technically handle making an image larger, but *only* proportionally.  This is an issue when you’re going from an 80GB image to a 1TB drive – “Why is the recovery partition 80GB?”

Of course, we could just make hundreds of images for every possible drive size, but that’s a lot of room, especially when the difference between drives is purely logical.

Instead, a bit of digging into how Clonezilla stores images and metadata led me to a great discovery.  In each image folder, there is a set of metadata files, specifically the geometry specification located in sda-pt.sf.

By modifying this file, we are able to change the drive geometry to our specifications.  This was the first step.

After that, it was a simple matter of creating hard links to each non-changing file for that model (the partclone images, specifically).  We now have a Ruby script that generates the new image folders, links the files to the base image, and copies over the files that will change between images.  After that, it’s a simple matter of generating the geometry (which the script handles in the most common case), and we can have any number of drive size images for the cost of the base image and a few KB per extra image.