Byte overflow in convolution function

Mar 11, 2011 at 9:30 AM
Edited Mar 11, 2011 at 10:44 AM

Hi, I using WriteableBitmapEx and  I needed Sharpen effect for my project. Convolution function doesn't work well. It seems that the cause of that is overflow in byte. I resolved that problem with  following code (file: WriteableBitmapFilterExtensions , line:196 ):

               int ta = ((a / kernelFactorSum) + kernelOffsetSum);
               int tr = ((r/kernelFactorSum) + kernelOffsetSum);
               int tg = ((g / kernelFactorSum) + kernelOffsetSum);
               int tb = ((b/kernelFactorSum) + kernelOffsetSum);


               var ba = (byte)((ta > 255) ? 255 : ((ta < 0) ? 0 : ta));
               var br = (byte)((tr > 255) ? 255 : ((tr < 0) ? 0 : tr));
               var bg = (byte)((tg > 255) ? 255 : ((tg < 0) ? 0 : tg));
               var bb = (byte)((tb > 255) ? 255 : ((tb < 0) ? 0 : tb));

I hope it helps. 

Coordinator
Mar 11, 2011 at 10:48 AM

Thanks for bringing this to the attention.

Seems like I forgot this byte boundary clamping here, but it also seems you found the usual clamping code and applied it.

It's fixed in r70166 

- René Schulte