I decided to do some image editing, and wanted to see if I could do some region detection (and storage in some data structure) and analysis/manipulation based on these regions.

My approach was the following:

• Compare Tiles with Neighbors Iteratively
• If the euclidean color distance is below a threshold (changeable) they belong to the same region
• If the neighbor already has a region, take his region. Otherwise, make a new region.
• Change their colors to a weighted average between the new tile added to a region (new tile weight 1/N, old tile weight 1-1/N, where N is the size of the region).
• Update the color of all other tiles in the region

My implementation of this can be found on github (link below). Example euclideanColor function:

int euclideanColor(const int a[3], const int b[3]){
//a, b is a 3 color component variable
	return sqrt(pow(b[0]-a[0],2)+pow(b[1]-a[1],2)+pow(b[2]-a[2],2));
}

I thought that for low thresholds, it would be possible to combine certain regions with very similar colors to larger regions, and then take all regions whose areas are above a certain size and simply grow them uniformly until the entire image is filled. I hoped to remove small details and have a bunch flat-colored surfaces representing a simplified form of my image.

I wanted to pass through all pixels of the array a single time (because images are large and expensive to process), thinking that the updating of the colors of the entire region might have few artifacts, but I was wrong.

The computation for this early test version was very expensive, so I switched to smaller images for testing. It also showed visual artifacts, such as “streaking” in the direction through which it performs this single pass. Regions also tend to grow from the left. This was very strongly dependent on the threshold value applied.

I explain these artifacts as following:

When running through the image, I run from top to bottom and then left to right. I also update the image data directly when averaging with a neighbor. This is what leads to the downward streaking effect.

Images don’t usually have very distinct color regions, but rather gradients which they follow throughout the image.

When following a color gradient in a direction, the color of the region behind it follows but lags behind the new tile we’re adding, due to the weighted average and growing region size. This leads to regions tending to grow more readily along the direction of search.

For higher thresholds, this leads to major vertical streaking, especially in images without stark color contrasts or gradients (e.g. the featured image above, as it is all slightly red-shifted). The image also grows from the left because of the parsing and overwriting direction.

I soon noticed that high threshold values would lead to complete mixing of large regions of color, and that the degree of “melting” appeared to be rather continuous as you raised the threshold value. So I decided to make some animations.

I wrote some C++ code that lets you progressively “melt” an image, in the form of a command-line tool. Here are some more examples:

I added a wrapper that lets you do it with a specific threshold step-size and n-frames. You can then very easily convert it to an animated GIF through the command line. How to do this is explained on my github page.

Github Link: https://github.com/weigert/drip

I think if you have an image with a large amount of vertical component, it would be very easy to scale it down and use a low threshold value to average small color regions. This removes the gradient you usually get when scaling down a large detail image, and gives you “matte regions”, while the vertical artifacts almost seem like an artistic addition. If you wanted a pixel-city backdrop it might be simple to generate in this way.

It also has very impressive effects when used on faces. Try it yourself! It helps to colorize it first with a program like GIMP.

I am quite happy with the visual effect and the way it is packaged as a command line tool. It makes generating such a face-melting GIF quite simple.

In the future I hope to maybe program a command line tool where you can specify a kernel you would like to pass over an image, and it spits it out for you. Then you could use this to pipe together image filters in a very basic manner.