Image Color Analysis with RMagick

Early last week I made a little app called Ice Cream. It's a simple web app that takes an uploaded photo and generates a color palette from that photo, based on which colors appear most often.

It shows you a percentage distribution for the 12 most popular colors, a palette of the top 256 colors, and gives you tools to pick and choose form those to create color schemes, for which it provides HEX color codes, RGB values, and a downloadable Adobe Swatch Exchange format swatch file.

A lot of what the app does is fairly straightforward and it relies on a 3rd party gem to generate the ASE files, but I wanted to take some time today to share the magic sauce behind the color analysis.

The General Principle

The basic idea behind color analysis in Ice Cream is to analyze the image histogram, pull out the most commonly used colors, and rank them by frequency of appearance.

RMagick provides us with a nice Ruby interface to ImageMagick on the filesystem, which contains all of the tools we need to do this. That said, we can't just dive right in and look at every color without first doing some prep work.

Color Quantization

We'll be analysing photos, which means we're looking at a palette containing 16 million colors, many of which are entirely indistinguishable from one another or so similar as to be indistinguishable on your average digital display.

Therefore, analyzing 16 million colors wouldn't provide much usable data. This is where color quantization comes in.

In a nutshell, color quantization referrs to performing some fancy algorithmic math on an image in order to reduce its number of total colors, while preserving as much image quality as possible. Exactly which algorithm is used depends on the type of image. It's the same thing that happens when you try and view a full-color photo on a 256 color display, for instance.

Fortunately, ImageMagick can do this, too. So let's look at some code.

# self.photo.path:
#   the paperclip file path for the uploaded photo
#
# num: 
#   the number of colors in our final palette, e.g. 12 or 256

image = Magick::ImageList.new(self.photo.path)
q = image.quantize(num, Magick::RGBColorspace)

In the code above, we first begin by opening the image file for analysis. Next, we run the image ImageMagick color quantization command on the open image, passing in the number of colors we want to reduce the palette down to and the name of the colorspace we're using, in this case, RGB.

Histograms and Pixel Counts

Now that we have a reduced color palette to work with, our next step in creating our ordered list of swatches is to arrange the colors in the newly created palette by order of frequency within the image.

To do this, we take the histogram of our reduced color photo and simply sort it from most used color to least used color.

palette = q.color_histogram.sort {|a, b| b[1] <=> a[1]}

Next, let's go ahead and calculate the total number of pixels in the image. Since the histogram data will tell us how many pixels exist of any given color, we can do some basic math using both of these values to calculate the percentage distribution of any given color.

# width in px times height in px
total_depth = image.columns * image.rows

Creating HEX and RGB Color Values

With our photo analyzed and our new histogram created, now we can iterate through it and calculate RGB and HEX values for all colors.

First, we'll set up an empty array to plop our results into.

results = []

Then we can iterate through our quantized color palette's histograph times to analyze colors.

palette.count.times do |i|
  p = palette[i]

Look at the red, green, and blue histogram values and divide by 256 - the number value associated with each color when specified as RGB.

  r1 = p[0].red / 256
  g1 = p[0].green / 256
  b1 = p[0].blue / 256

Now take those newly calculated colors and convert them to hexadecimal strings using ruby's built in to_s method, to determine our HEX color values for each color channel.

  r2 = r1.to_s(16)
  g2 = g1.to_s(16)
  b2 = b1.to_s(16)

Sometiems our resulting HEX is only a single character, rather than two, but our fully returned HEX value needs to be 6 characters in length. To solve this, we concatanate the string with itself to pad it unless it's already long enough.

Note: This is the equvilent of taking a shortened #FFF in your CSS and expanding it to its complete form as #FFFFFF.

  r2 += r2 unless r2.length == 2 
  g2 += g2 unless g2.length == 2
  b2 += b2 unless b2.length == 2

Now we'll simply generate strings using the variables we set above, for easy storage in our database.

  rgb = "#{r1},#{g1},#{b1}"
  hex = "#{r2}#{g2}#{b2}"
  depth = p[1] # number of pixels of this color in the image

Finally, let's put these into our results array and calculate the percenage of color occurance, using the depth determined above as compared to the total depth set previously.

  results << {
    rgb: rgb,
    hex: hex,
    percent: ((depth.to_f / total_depth.to_f) * 100).round(2)
  }
end

Putting It All Together

And finally, here's a look at the complete method, which lives in the Palette model inside of the Ice Cream app.

def colors_from_photo(num)
  image = Magick::ImageList.new(self.photo.path)
  q = image.quantize(num, Magick::RGBColorspace)
  palette = q.color_histogram.sort {|a, b| b[1] <=> a[1]}
  total_depth = image.columns * image.rows
  results = []

  palette.count.times do |i|
    p = palette[i]

    r1 = p[0].red / 256
    g1 = p[0].green / 256
    b1 = p[0].blue / 256

    r2 = r1.to_s(16)
    g2 = g1.to_s(16)
    b2 = b1.to_s(16)

    r2 += r2 unless r2.length == 2 
    g2 += g2 unless g2.length == 2
    b2 += b2 unless b2.length == 2

    rgb = "#{r1},#{g1},#{b1}"
    hex = "#{r2}#{g2}#{b2}"
    depth = p[1]

    results << {
      rgb: rgb,
      hex: hex,
      percent: ((depth.to_f / total_depth.to_f) * 100).round(2)
    }
  end

  results
end

Conclusion

So there you have it! Image color analysis can be pretty mathematically intense. Fortunately, ImageMagick does the heavy lifting for us, and the RMagick gem gives us a simple to use interface that doesn't stray wildly far from ImageMagick's native command line syntax.

Once you have your image histogram data readily available, analying its contents is fairly simple, and by reducing the histogram to only the colors you need before analysis, you'll get much more meaningful data.

Update Thanks to Matthew Miller for finding/fixing a small bug when looping through the quantized palette results for images with small color palettes. I've edited the code to reflect the changes.