How does Spotify work

Spotify uses a combination of techniques to allow a user to produce pleasing results:

  • identify areas in an image that are to be ignored
  • handle artifacts, whether created by anti-aliasing, JPEG artefacts or 'impurities' in the image
  • find the specified number of individual colors that best represent the whole image
  • assign the best matching color to each pixel (or leave it transparent by masking it out)

Areas to ignore

Areas in an image may have to be ignored for a number of reasons:

  • in most cases, white areas are to be ignored; this may include areas that are 'almost white'
  • almost always, fully transparent areas are to be ignored; sometimes this also applies 'almost fully transparent' areas
  • in rare cases, gray areas are also to be ignored (this is only available in Spotify mode on the command line)
  • in rare cases, black areas are also to be ignored (this is only available in Spotify mode on the command line)

Before going through the actual processing, and depending on how parameters are set, pixels in certain areas will not be taken into account (and will later on be masked out).

Artefacts

Even for digitally created – and seemingly very 'clean' – images there will be artefacts, most notably going back to antialiasing and – when JPEG compression is applied – to JPEG compression block artefacts.

The biggest issue with both is that in these areas colors will be present that are not representartive of the image appearance. Just envision an image with blue square next to a yellow square. If antialising kicks in, the pixels at the border between the two will use some shade(s) of green. Nonetheless the representative colors for both areas still are blue and yellow, and not green.

Now envision a yellow background with some very thin text (e.g. using Helvetice Neue Light) on it using blue: there will be many yellow pixels (from the background), some but not many blue pixels (for the text) – and a sometimes surprisingly large number of green (or greenish) pixels – for the border between the blue text and the yellow background. In some real world files, the number of green pixels my be larger than the number of plain blue pixels.

Spotify has options – based on edge detection algorithms – that make it possible to ignore such border pixels when looking for the best matching colors, but to still to take these pixels into acount when assigning the best matching colors (and in this example assigning either blue or yellow to the green or hreenish pixels).

Determine best matching colors

Spotify uses k-means clustering algorithms to identify those (usually few) colors that best represent an image. Research has shown that first looking for more colors than needed, and then condensing the list of found colors into a smaller list of just those few desired colors almost always gives better results than looking directly for the desired number of colors.

As k-means clustering by its very nature has random aspects to it, running the algorithm twice (or more often) on the same data set will yield slightly (or even not so slightly) different results each time. As a consequence, Spotify will go through several iterations and the pick the iteration with lowest aggregated difference between original image and image represented by the result colors.

Assigning spot colors to pixels

Regardless whether Spotify is used on images or PDFs containing not just images but also text and vector, the result will always be an image on a PDF page, where that image extends over the whole page area. That image

  • has one channel for each spot color
  • each channel is bitonal, that is it is either 100% of the given spot color, or 0%
  • for each pixel, only one of the channels will be set to 100%
  • for every pixel, where all channels are set to 0%, a mask is applied to make the image transparent at th given pixel.

As the only way to encode such images in PDF is to use a DeviceN color space, the created image will be a DeviceN colored multi-channel bitonal image.

Names for the spot colors are either taken from a spot color library provided by the user, or generated from the RGB values that represent the appearance of the spot color.

The alternate space for the spot colors is by default encoded using sRGB. For the Spotify mode on the command line it is also possible to use DeviceRGB or Lab as the alternate space. Using Lab has proven to cause rendering problems on at least some viewers.

This video below goes directly to 'how Spotify works':