Convolutions & Gaussian Blur

Nov 29, 2010 at 12:30 PM

Hi, I needed gaussian blur for a project where I was allready using WriteableBitmapEx. Since WriteableBitmapEx doesn't currently support Convolutions I created the code below.It could easily be extended with new kernels to do things like unsharp masking.

Feel free to include it in the project if you'd like.

public static class WriteableBitmapConvolutionExtensions
    {
        private static byte[] _tinyGaussianBlurKernel =
            {16,26,16,
             26,41,26,
             16,26,16};

        private static byte[] _gaussianBlurKernel =
            { 1, 4, 7, 4,1,
              4,16,26,16,4,
              7,26,41,26,7,
              4,16,26,16,4,
              1, 4, 7, 4,1};
       
        public static void TinyGaussianBlur(this WriteableBitmap bmp)
        {
            bmp.Convolute(_tinyGaussianBlurKernel, 3, 3);
        }

        public static void GaussianBlur(this WriteableBitmap bmp)
        {
            bmp.Convolute(_gaussianBlurKernel, 5, 5);
        }

        public static void Convolute(this WriteableBitmap bmp, byte[] kernel, int kernelWidth, int kernelHeight)
        {
            if ((kernelWidth & 1) == 0)
            {
                throw new InvalidOperationException("Kernel width must be odd!");
            }
            if ((kernelHeight & 1) == 0)
            {
                throw new InvalidOperationException("Kernel height must be odd!");
            }
            if (kernel.Length != kernelWidth * kernelHeight)
            {
                throw new InvalidOperationException("Kernel size doesn't match width*height!");
            }

            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int index = 0;
            int halfKernelWidth = kernelWidth / 2;
            int halfKernelHeight = kernelHeight / 2;

            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int kernelSum = 0;
                    int r = 0;
                    int g = 0;
                    int b = 0;

                    for (int kx = -halfKernelWidth; kx <= halfKernelWidth; kx++)
                    {
                        int px = kx + x;
                        if (px < 0 || px >= w)
                        {
                            continue;
                        }

                        for (int ky = -halfKernelHeight; ky <= halfKernelHeight; ky++)
                        {
                            int py = ky + y;
                            if (py < 0 || py >= h)
                            {
                                continue;
                            }

                            int kernelIndex = (ky + halfKernelWidth) * kernelWidth + (kx + halfKernelHeight);
                            byte kernelWeight = kernel[kernelIndex];
                            kernelSum += kernelWeight;

                            int innerIndex = py * w + px;
                            int col = pixels[innerIndex];
                            r += ((byte)(col >> 16)) * kernelWeight;
                            g += ((byte)(col >> 8)) * kernelWeight;
                            b += ((byte)col) * kernelWeight;
                        }
                    }

                    byte br = (byte)(r / kernelSum);
                    byte bg = (byte)(g / kernelSum);
                    byte bb = (byte)(b / kernelSum);

                    int color =
                        (255 << 24)
                        | (br << 16)
                        | (bg << 8)
                        | (bb);

                    pixels[index++] = color;
                }
            }
        }
    }
Coordinator
Nov 29, 2010 at 1:03 PM

Thanks for this contribution. I created a work item for this here:

http://writeablebitmapex.codeplex.com/workitem/14796

 

- Rene Schulte

Nov 29, 2010 at 1:29 PM
Edited Nov 29, 2010 at 1:30 PM
Super, I've updated the code a bit since I posted - I realized that you can use 1d gaussian blur twice (once horizontally and once vertically), which is faster.
 
I'm also looking at box-blur and other similar methods - so I might publish more updates in the thread.
 
cheers,
mattias
 
 

using System;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;
using System.Windows.Ink;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;
using System.Windows.Media.Imaging;

namespace SilverlightApplication1
{
    public static class WriteableBitmapConvolutionExtensions
    {
        private static byte[] _tinyGaussianBlurKernel =
            {16,26,16,
             26,41,26,
             16,26,16};

        private static byte[] _gaussianBlurKernel =
            { 1, 4, 7, 4,1,
              4,16,26,16,4,
              7,26,41,26,7,
              4,16,26,16,4,
              1, 4, 7, 4,1};

        public static void GaussianBlurTiny(this WriteableBitmap bmp)
        {
            bmp.Convolute(_tinyGaussianBlurKernel, 3, 3);
        }

        public static void GaussianBlur(this WriteableBitmap bmp)
        {
            bmp.Convolute(_gaussianBlurKernel, 5, 5);
        }

        private static byte[] _1dgaussianBlurKernel = { 1, 4, 7, 4,1,};
        private static byte[] _tiny1dGaussianBlurKernel = {  4, 7, 4 };

        public static void GaussianBlurFast(this WriteableBitmap bmp)
        {
            bmp.ConvoluteX(_tiny1dGaussianBlurKernel);
            bmp.ConvoluteY(_tiny1dGaussianBlurKernel);
        }

        public static void Convolute(this WriteableBitmap bmp, byte[] kernel, int kernelWidth, int kernelHeight)
        {
            if ((kernelWidth & 1) == 0)
            {
                throw new InvalidOperationException("Kernel width must be odd!");
            }
            if ((kernelHeight & 1) == 0)
            {
                throw new InvalidOperationException("Kernel height must be odd!");
            }
            if (kernel.Length != kernelWidth * kernelHeight)
            {
                throw new InvalidOperationException("Kernel size doesn't match width*height!");
            }

            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int index = 0;
            int halfKernelWidth = kernelWidth / 2;
            int halfKernelHeight = kernelHeight / 2;

            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int kernelSum = 0;
                    int r = 0;
                    int g = 0;
                    int b = 0;

                    for (int kx = -halfKernelWidth; kx <= halfKernelWidth; kx++)
                    {
                        int px = kx + x;
                        if (px < 0 || px >= w)
                        {
                            continue;
                        }

                        for (int ky = -halfKernelHeight; ky <= halfKernelHeight; ky++)
                        {
                            int py = ky + y;
                            if (py < 0 || py >= h)
                            {
                                continue;
                            }

                            int kernelIndex = (ky + halfKernelHeight) * kernelWidth + (kx + halfKernelWidth);
                            byte kernelWeight = kernel[kernelIndex];
                            kernelSum += kernelWeight;

                            int innerIndex = py * w + px;
                            int col = pixels[innerIndex];
                            r += ((byte)(col >> 16)) * kernelWeight;
                            g += ((byte)(col >> 8)) * kernelWeight;
                            b += ((byte)col) * kernelWeight;
                        }
                    }

                    byte br = (byte)(r / kernelSum);
                    byte bg = (byte)(g / kernelSum);
                    byte bb = (byte)(b / kernelSum);

                    int color =
                        (255 << 24)
                        | (br << 16)
                        | (bg << 8)
                        | (bb);

                    pixels[index++] = color;
                }
            }
        }

        public static void ConvoluteX(this WriteableBitmap bmp, byte[] kernel)
        {
            int kernelWidth = kernel.Length;
            if ((kernelWidth & 1) == 0)
            {
                throw new InvalidOperationException("Kernel width must be odd!");
            }

            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int index = 0;
            int halfKernelWidth = kernelWidth / 2;

            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int kernelSum = 0;
                    int r = 0;
                    int g = 0;
                    int b = 0;

                    for (int kx = -halfKernelWidth; kx <= halfKernelWidth; kx++)
                    {
                        int px = kx + x;
                        if (px < 0 || px >= w)
                        {
                            continue;
                        }

                        int kernelIndex = kx + halfKernelWidth;
                        byte kernelWeight = kernel[kernelIndex];
                        kernelSum += kernelWeight;

                        int innerIndex = y * w + px;
                        int col = pixels[innerIndex];
                        r += ((byte)(col >> 16)) * kernelWeight;
                        g += ((byte)(col >> 8)) * kernelWeight;
                        b += ((byte)col) * kernelWeight;

                    }

                    byte br = (byte)(r / kernelSum);
                    byte bg = (byte)(g / kernelSum);
                    byte bb = (byte)(b / kernelSum);

                    int color =
                        (255 << 24)
                        | (br << 16)
                        | (bg << 8)
                        | (bb);

                    pixels[index++] = color;
                }
            }
        }

        public static void ConvoluteY(this WriteableBitmap bmp, byte[] kernel)
        {
            int kernelHeight = kernel.Length;
            if ((kernelHeight & 1) == 0)
            {
                throw new InvalidOperationException("Kernel height must be odd!");
            }
  
            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int index = 0;
            int halfKernelHeight = kernelHeight / 2;

            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int kernelSum = 0;
                    int r = 0;
                    int g = 0;
                    int b = 0;

                    for (int ky = -halfKernelHeight; ky <= halfKernelHeight; ky++)
                    {
                        int py = ky + y;
                        if (py < 0 || py >= h)
                        {
                            continue;
                        }

                        int kernelIndex = (ky + halfKernelHeight);
                        byte kernelWeight = kernel[kernelIndex];
                        kernelSum += kernelWeight;

                        int innerIndex = py * w + x;
                        int col = pixels[innerIndex];
                        r += ((byte)(col >> 16)) * kernelWeight;
                        g += ((byte)(col >> 8)) * kernelWeight;
                        b += ((byte)col) * kernelWeight;
                    }                   

                    byte br = (byte)(r / kernelSum);
                    byte bg = (byte)(g / kernelSum);
                    byte bb = (byte)(b / kernelSum);

                    int color =
                        (255 << 24)
                        | (br << 16)
                        | (bg << 8)
                        | (bb);

                    pixels[index++] = color;
                }
            }
        }
    }
}

Coordinator
Nov 29, 2010 at 1:49 PM

Nice, so you already made one optimization I had in mind. BTW, this technique is called Convolution by Separability. Here's a good explanation. http://www.dspguide.com/ch24/3.htm

Box Blur is described here: http://www.vcskicks.com/box-blur.php

Thanks. Contributions are always welcome and are honored on the start page of the project (see Credits).

Nov 30, 2010 at 8:05 AM

Here's the latest and hopefully last version, BoxBlur is the fastest as it uses an accumulator and doesn't look up the values over and over again. It's complex enough though...

using System;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;
using System.Windows.Ink;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;
using System.Windows.Media.Imaging;

namespace System.Windows.Media.Imaging
{
    public static class WriteableBitmapConvolutionExtensions
    {
        private static byte[] _tinyGaussianBlurKernel = 
            {16,26,16,
             26,41,26,
             16,26,16};

        private static byte[] _gaussianBlurKernel = 
            { 1, 4, 7, 4,1,
              4,16,26,16,4,
              7,26,41,26,7,
              4,16,26,16,4,
              1, 4, 7, 4,1};

        public static void GaussianBlurTiny(this WriteableBitmap bmp)
        {
            bmp.Convolute(_tinyGaussianBlurKernel, 3, 3);
        }

        public static void GaussianBlur(this WriteableBitmap bmp)
        {
            bmp.Convolute(_gaussianBlurKernel, 5, 5);
        }

        private static byte[] _1dgaussianBlurKernel = { 1, 4, 7, 4, 1, };
        private static byte[] _tiny1dGaussianBlurKernel = { 4, 7, 4 };

        public static void GaussianBlurFast(this WriteableBitmap bmp)
        {
            bmp.ConvoluteX(_tiny1dGaussianBlurKernel);
            bmp.ConvoluteY(_tiny1dGaussianBlurKernel);
        }

        public static void Convolute(this WriteableBitmap bmp, byte[] kernel, int kernelWidth, int kernelHeight)
        {
            if ((kernelWidth & 1) == 0)
            {
                throw new InvalidOperationException("Kernel width must be odd!");
            }
            if ((kernelHeight & 1) == 0)
            {
                throw new InvalidOperationException("Kernel height must be odd!");
            }
            if (kernel.Length != kernelWidth * kernelHeight)
            {
                throw new InvalidOperationException("Kernel size doesn't match width*height!");
            }

            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int index = 0;
            int halfKernelWidth = kernelWidth / 2;
            int halfKernelHeight = kernelHeight / 2;

            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int kernelSum = 0;
                    int r = 0;
                    int g = 0;
                    int b = 0;

                    for (int kx = -halfKernelWidth; kx <= halfKernelWidth; kx++)
                    {
                        int px = kx + x;
                        if (px < 0 || px >= w)
                        {
                            continue;
                        }

                        for (int ky = -halfKernelHeight; ky <= halfKernelHeight; ky++)
                        {
                            int py = ky + y;
                            if (py < 0 || py >= h)
                            {
                                continue;
                            }

                            int kernelIndex = (ky + halfKernelHeight) * kernelWidth + (kx + halfKernelWidth);
                            byte kernelWeight = kernel[kernelIndex];
                            kernelSum += kernelWeight;

                            int innerIndex = py * w + px;
                            int col = pixels[innerIndex];
                            r += ((byte)(col >> 16)) * kernelWeight;
                            g += ((byte)(col >> 8)) * kernelWeight;
                            b += ((byte)col) * kernelWeight;
                        }
                    }

                    byte br = (byte)(r / kernelSum);
                    byte bg = (byte)(g / kernelSum);
                    byte bb = (byte)(b / kernelSum);

                    int color =
                        (255 << 24)
                        | (br << 16)
                        | (bg << 8)
                        | (bb);

                    pixels[index++] = color;
                }
            }
        }

        public static void ConvoluteX(this WriteableBitmap bmp, byte[] kernel)
        {
            int kernelWidth = kernel.Length;
            if ((kernelWidth & 1) == 0)
            {
                throw new InvalidOperationException("Kernel width must be odd!");
            }

            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int index = 0;
            int halfKernelWidth = kernelWidth / 2;

            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int kernelSum = 0;
                    int r = 0;
                    int g = 0;
                    int b = 0;

                    for (int kx = -halfKernelWidth; kx <= halfKernelWidth; kx++)
                    {
                        int px = kx + x;
                        if (px < 0 || px >= w)
                        {
                            continue;
                        }

                        int kernelIndex = kx + halfKernelWidth;
                        byte kernelWeight = kernel[kernelIndex];
                        kernelSum += kernelWeight;

                        int innerIndex = y * w + px;
                        int col = pixels[innerIndex];
                        r += ((byte)(col >> 16)) * kernelWeight;
                        g += ((byte)(col >> 8)) * kernelWeight;
                        b += ((byte)col) * kernelWeight;

                    }

                    byte br = (byte)(r / kernelSum);
                    byte bg = (byte)(g / kernelSum);
                    byte bb = (byte)(b / kernelSum);

                    int color =
                        (255 << 24)
                        | (br << 16)
                        | (bg << 8)
                        | (bb);

                    pixels[index++] = color;
                }
            }
        }

        public static void ConvoluteY(this WriteableBitmap bmp, byte[] kernel)
        {
            int kernelHeight = kernel.Length;
            if ((kernelHeight & 1) == 0)
            {
                throw new InvalidOperationException("Kernel height must be odd!");
            }

            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int index = 0;
            int halfKernelHeight = kernelHeight / 2;

            for (int y = 0; y < h; y++)
            {
                for (int x = 0; x < w; x++)
                {
                    int kernelSum = 0;
                    int r = 0;
                    int g = 0;
                    int b = 0;

                    for (int ky = -halfKernelHeight; ky <= halfKernelHeight; ky++)
                    {
                        int py = ky + y;
                        if (py < 0 || py >= h)
                        {
                            continue;
                        }

                        int kernelIndex = (ky + halfKernelHeight);
                        byte kernelWeight = kernel[kernelIndex];
                        kernelSum += kernelWeight;

                        int innerIndex = py * w + x;
                        int col = pixels[innerIndex];
                        r += ((byte)(col >> 16)) * kernelWeight;
                        g += ((byte)(col >> 8)) * kernelWeight;
                        b += ((byte)col) * kernelWeight;
                    }

                    byte br = (byte)(r / kernelSum);
                    byte bg = (byte)(g / kernelSum);
                    byte bb = (byte)(b / kernelSum);

                    int color =
                        (255 << 24)
                        | (br << 16)
                        | (bg << 8)
                        | (bb);

                    pixels[index++] = color;
                }
            }
        }

        public static void BoxBlur(this WriteableBitmap bmp, int range)
        {
            if ((range & 1) == 0)
            {
                throw new InvalidOperationException("Range must be odd!");
            }

            bmp.BoxBlurHorizontal(range);
            bmp.BoxBlurVertical(range);
        }

        public static void BoxBlurHorizontal(this WriteableBitmap bmp, int range)
        {
            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int halfRange = range / 2;
            int index = 0;
            int[] newColors = new int[w];

            for (int y = 0; y < h; y++)
            {
                int hits = 0;
                int r = 0;
                int g = 0;
                int b = 0;
                for (int x = -halfRange; x < w; x++)
                {
                    int oldPixel = x - halfRange - 1;
                    if (oldPixel >= 0)
                    {
                        int col = pixels[index + oldPixel];
                        if (col != 0)
                        {
                            r -= ((byte)(col >> 16));
                            g -= ((byte)(col >> 8));
                            b -= ((byte)col);
                        }
                        hits--;
                    }

                    int newPixel = x + halfRange;
                    if (newPixel < w)
                    {
                        int col = pixels[index + newPixel];
                        if (col != 0)
                        {
                            r += ((byte)(col >> 16));
                            g += ((byte)(col >> 8));
                            b += ((byte)col);
                        }
                        hits++;
                    }

                    if (x >= 0)
                    {
                        int color =
                            (255 << 24)
                            | ((byte)(r / hits) << 16)
                            | ((byte)(g / hits) << 8)
                            | ((byte)(b / hits));

                        newColors[x] = color;
                    }
                }

                for (int x = 0; x < w; x++)
                {
                    pixels[index + x] = newColors[x];
                }

                index += w;
            }
        }

        public static void BoxBlurVertical(this WriteableBitmap bmp, int range)
        {
            int[] pixels = bmp.Pixels;
            int w = bmp.PixelWidth;
            int h = bmp.PixelHeight;
            int halfRange = range / 2;

            int[] newColors = new int[h];
            int oldPixelOffset = -(halfRange + 1) * w;
            int newPixelOffset = (halfRange) * w;

            for (int x = 0; x < w; x++)
            {
                int hits = 0;
                int r = 0;
                int g = 0;
                int b = 0;
                int index = -halfRange * w + x;
                for (int y = -halfRange; y < h; y++)
                {                    
                    int oldPixel = y - halfRange - 1;
                    if (oldPixel >= 0)
                    {
                        int col = pixels[index + oldPixelOffset];
                        if (col != 0)
                        {
                            r -= ((byte)(col >> 16));
                            g -= ((byte)(col >> 8));
                            b -= ((byte)col);
                        }
                        hits--;
                    }

                    int newPixel = y + halfRange;
                    if (newPixel < h)
                    {
                        int col = pixels[index + newPixelOffset];
                        if (col != 0)
                        {
                            r += ((byte)(col >> 16));
                            g += ((byte)(col >> 8));
                            b += ((byte)col);
                        }
                        hits++;
                    }

                    if (y >= 0)
                    {
                        int color =
                            (255 << 24)
                            | ((byte)(r / hits) << 16)
                            | ((byte)(g / hits) << 8)
                            | ((byte)(b / hits));

                        newColors[y] = color;
                    }

                    index += w;
                }

                for (int y = 0; y < h; y++)
                {
                    pixels[y * w + x] = newColors[y];
                }
            }
        }
    }
}

Coordinator
Nov 30, 2010 at 8:25 AM

Thanks! I will have a look at it and integrate it.

Coordinator
Feb 10, 2011 at 7:34 PM
Edited Feb 10, 2011 at 7:37 PM

I added a similar, generic convolution method in r68353, which repeats the pixels at the border, supports alternative summation factor / offset and some optimizations.

Thanks for the suggestion! I added your name to the Credits list on the start page: http://writeablebitmapex.codeplex.com/

 

- Rene

Dec 22, 2012 at 4:48 AM

Hi, I am new to C# and Windows Phone development. What is the correct usage of APIs to get image blurred?

                    WriteableBitmap wb = new WriteableBitmap(albumArtImage);
                    WriteableBitmapExtensions.Convolute(wb, WriteableBitmapExtensions.KernelGaussianBlur5x5);
                    AlbumBackground.ImageSource = wb;

I am using WriteableBitmapEx_v1.0.5.0. Thanks a lot!

Feb 12, 2013 at 3:31 AM
Calling the WriteableBitmap.Pixels to get the pixels yields a problem in my project.

What I am trying to use this is for a project using the Kinect.
On every frame received from the Kinect, I would like to use Box Blur to blur the Image.

The .Pixels call results with a problem due to a SecurityException: The WriteableBitmap is created from protected content. The Pixels array is inaccessible in this case.

Instead I try using WriteableBitmapExtensions.KernelGaussianBlur5x5 from the 1.0.6.0 version.
Using WriteableBitmapEx_v1.0.6.0 this is what I have in my project:
__
colorFrame.CopyPixelDataTo(this.colorPixels);
                // Write the pixel data into our bitmap
                // convert back to original format
                for (int i = 3; i < this.colorPixels.Length; i += 4) this.colorPixels[i] = 255;

                this.colorBitmap.WritePixels(
                    new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
                    this.colorPixels,
                    this.colorBitmap.PixelWidth * sizeof(int),
                    0);


                this.colorBitmap.Convolute(WriteableBitmapExtensions.KernelGaussianBlur5x5);__
This however does not cause any GaussianBlur on the image - it blocks the application completely.
Is this because this operation is too expensive and it is called every time a new frame is received (there are 30 fps).
Feb 12, 2013 at 4:36 AM
How can we circumvent the problem with <code>.Pixels</code>?
Coordinator
Feb 12, 2013 at 6:31 AM
Yes, it's a quite expensive call.

For your own implementaion of BoxBlur make sure you only create a WB on the UI thread.

Also see my notes here about performance and the right usage of the BitmapContext: http://writeablebitmapex.codeplex.com/discussions/432111
  • Rene
Mar 9, 2014 at 10:24 AM
Trying to use convolute on one of the images in my project but the blur doesn't apply to the image at runtime.
                StorageFile file = await StorageFile.GetFileFromPathAsync(Model.CurrentUser.BackgroundImagePath);
                IRandomAccessStream fileStream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read);
                WriteableBitmap wb = await new WriteableBitmap(1, 1).FromStream(fileStream);
                var wb1 = WriteableBitmapExtensions.Convolute(wb, WriteableBitmapExtensions.KernelGaussianBlur3x3);
                MainTile.Source = wb1; 
Any ideas?