Simple image blitting/compositing with a background image and a foreground image.

Jan 31, 2013 at 8:33 PM
Hello

I'm having trouble using the blit function with windows RT. I have one foreground image that is generated from an InkManager. I can save this and it shows the strokes correctly, with transparency everywhere else. I have one background image that is a static resource in the project.

I want to overlay the foreground onto the background image and then save it. Instead, this code produces just the background image with no foreground contents. The blit operation seems to do nothing to the image.

I've tried a variety of options and changes over the last couple hours and this one has me stumped. Unrelated to WriteableBitmapEx, I'm aware that new byte[50000]); in the code below is bad. If anyone has any suggestions on how to avoid that, that would also be nice. Thanks for any help.
Windows.Storage.Pickers.FileSavePicker save = new Windows.Storage.Pickers.FileSavePicker();
save.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.Desktop;
save.DefaultFileExtension = ".png";
save.FileTypeChoices.Add("PNG", new string[] { ".png" });
StorageFile filesave = await save.PickSaveFileAsync();

//Image background
WriteableBitmap backgroundBmp = BitmapFactory.New(480,454);
backgroundBmp.GetBitmapContext();
var file = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Images/mybgimage.png"));
IBuffer myBuffer  = await Windows.Storage.FileIO.ReadBufferAsync(file);
byte[] myArray = myBuffer.ToArray();
MemoryRandomAccessStream bgstream = new MemoryRandomAccessStream(myArray);
await backgroundBmp.SetSourceAsync(bgstream);

//Image foreground
WriteableBitmap foregroundBmp = new WriteableBitmap(300,300);
foregroundBmp.GetBitmapContext();
MemoryRandomAccessStream a = new MemoryRandomAccessStream(new byte[50000]);
await _mInkManager.SaveAsync(a);
await foregroundBmp.SetSourceAsync(a);

// Combined
backgroundBmp.Blit(new Rect(0, 0, backgroundBmp.PixelWidth, backgroundBmp.PixelHeight), foregroundBmp, new Rect(0, 0, foregroundBmp.PixelWidth, foregroundBmp.PixelHeight), WriteableBitmapExtensions.BlendMode.Alpha);
Guid encoderId = Windows.Graphics.Imaging.BitmapEncoder.PngEncoderId;
await WinRTXamlToolkit.Imaging.WriteableBitmapSaveExtensions.SaveToFile(foregroundBmp, filesave, encoderId);
-Karl
Coordinator
Feb 1, 2013 at 8:13 AM
Edited Feb 1, 2013 at 8:14 AM
There are a couple of issues with your code:

a) SetSourceAsync does not guarantee that all the pixel data was loaded into the WB after it's finished. It just sets the source. You should instead use the WBX FromStream method to load the data into the WB like so:
WriteableBitmap foregroundBmp = await BitmapFactory.New(1, 1).FromStream(a);
b) Also note the bitmap context copies the data over, so use that only around bitmap operations, not when loading data and also call Dispose of it. If you don't Dispose the data is never copied back. That copying is needed since WinRT uses a different pixel layout than the WB on other platforms.
I usually use a using for disposing:
using(bmp.GetBitmapContext())
{
   // Perform draw stuff here
}
In your case you can completely avoid it since the Blit operation (like all other WBX methods) does that automatically. The manual GetBitmapContext is only recommended when performing multiple operations at once. The BitmpContext uses ref counting.


c) Can't you just avoid passing the byte buffer by using an InMemoryRandomAccessStream? Also it's a good practice to Dispose all disposable objects. A using statement is pretty handy most of the time.

d) For the loading of the background from the project's resource you should also use the FromContent method, avoid the context and not use SetSource.
Embedding images as Build Action = Content is always recommended. Check you project if the image is added as Content.

e) You also save the foreground bmp although you blit all to the backgroundBmp so the backgroundBmp is the final composed result you want to actually save.

The complete code snippet should look like this:
//Image background
var backgroundBmp = await BitmapFactory.New(1, 1).FromContent(new Uri("///Images/mybgimage.png"));
          
//Image foreground
WriteableBitmap foregroundBmp;
using (InMemoryRandomAccessStream a = new InMemoryRandomAccessStream())
{
    await _mInkManager.SaveAsync(a);
    a.Seek(0);
    foregroundBmp = await new WriteableBitmap(1, 1).FromStream(a);
}

// Combined
backgroundBmp.Blit(new Rect(0, 0, backgroundBmp.PixelWidth, backgroundBmp.PixelHeight), foregroundBmp, new Rect(0, 0, foregroundBmp.PixelWidth, foregroundBmp.PixelHeight), WriteableBitmapExtensions.BlendMode.Alpha);

// Save
Windows.Storage.Pickers.FileSavePicker save = new Windows.Storage.Pickers.FileSavePicker();
save.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.Desktop;
save.DefaultFileExtension = ".png";
save.FileTypeChoices.Add("PNG", new string[] { ".png" });
StorageFile filesave = await save.PickSaveFileAsync();
Guid encoderId = Windows.Graphics.Imaging.BitmapEncoder.PngEncoderId;
await WinRTXamlToolkit.Imaging.WriteableBitmapSaveExtensions.SaveToFile(backgroundBmp, filesave, encoderId);
  • Rene
Feb 1, 2013 at 2:11 PM
Thanks for such a detailed response. The solution you provided works great.

I'm new to the windows platform so that's probably why some of my code choices were not the best.


-Karl
Feb 4, 2013 at 5:51 PM
Hello,

I noticed that any white (255,255,255,255) pixels in the foreground will not show up in the blitted result. I tried both Multiply and Alpha blitting. Is there another step I have to do for this to work.

Thanks for any assistance.

-Karl
Coordinator
Feb 4, 2013 at 9:11 PM
Weird. Could be a bug from the pre-multiplied alpha blitting. Can you try a Light Gray and see if it works?
  • Rene
Feb 5, 2013 at 5:43 PM
Edited Feb 5, 2013 at 5:52 PM
It appears to be in the step between the ink and the bitmap. See image below.

Image

Figure A shows what the ink canvas looks like. (I took a screen snip)
Figure B shows the bitmap that the saveAsync produces.
Figure C shows the blitted image.

So I don't think it is a bug in the blitting. I think that something is wrong in SaveAsync or the conversion to the WriteableBitmap. Maybe I should contact Microsoft about this one.
Also the R and B channels appear to be swapped. An issue of storing ARGB vs ABGR perhaps.
WriteableBitmap foregroundBmp;
using (InMemoryRandomAccessStream a = new InMemoryRandomAccessStream())
{
    await _mInkManager.SaveAsync(a);
    a.Seek(0);
    foregroundBmp = await new WriteableBitmap(1, 1).FromStream(a);
}
The white squares are produced with the WriteableBitmap.FillQuad on the foregroundBmp
byte val = 255;
for (int i = 0; i < 10; i++)
{
     foregroundBmp.FillQuad(10, 10 + i * 10,
                                          10, 20 + i * 10, 
                                          20, 20 + i * 10,
                                          20, 10 + i * 10, Color.FromArgb(255, val, val, val));
     val -= 20;
}
Let me know if you have any ideas.
On a side note: if I wanted to render text to the image, what technique do you think would be the best.
Coordinator
Feb 5, 2013 at 7:19 PM
Are you using the latest binaries? There was a bug in one of the previous versions with swapped channels but it should be okay now.

For text rendering you would need to use bitmap fonts since WinRT can't render a TextBlock to a WB. :( I think someone wrote a blog post about bitmap fonts or check the WinRtToolkit.
  • Rene
Feb 5, 2013 at 7:43 PM
I'm using WriteableBitmapEx version 1.0.6.0

-Karl
Feb 15, 2013 at 3:00 PM
Edited Feb 15, 2013 at 3:03 PM
Same for me in WinRT.
Blue and Red color is completely swapped. I use version 1.0.6.0 from NuGet.

My code(below) is pretty similar to what discussed above.

                using (var stream = new InMemoryRandomAccessStream())
                {
                    await _inkManager.SaveAsync(stream.GetOutputStreamAt(0));

                    var width = (int)_inkManager.BoundingRect.Width;
                    var height = (int)_inkManager.BoundingRect.Height;
                    var resultBmp = BitmapFactory.New(width, height);
                    PreviewImage.Source = resultBmp; // Preview XAML Image component.
                    resultBmp.Clear(Colors.White);

                    // Call stroke gif stream.
                    WriteableBitmap strokeBmp = await BitmapFactory.New(1,1).FromStream(stream);

                    // Combine two bitmap
                    var rect = new Rect(0, 0, width, height);
                    resultBmp.Blit(rect, strokeBmp, rect, WriteableBitmapExtensions.BlendMode.Alpha);

                    // File handling
                    var file = await resultBmp.SaveToFile(ApplicationData.Current.LocalFolder);
                }
Coordinator
Feb 15, 2013 at 4:21 PM
Would be great if you can upload a full repro somewhere so I can quickly look into it. Thanks. :)
  • Rene
Feb 15, 2013 at 7:52 PM
I should have time tonight to get a minimal project uploaded, unless Youngjae beats me to it.

-Karl
Feb 16, 2013 at 1:56 AM
Edited Feb 16, 2013 at 1:57 AM
Thanks for your kindness.

Please click this SkyDrive http://sdrv.ms/Z3UESv link. You will get the 7zip compressed file.
I added WBX test code based on the Microsoft official "Input Simplified ink" sample.

For me, if you write with blue color and click the "WBX Test!" button, the preview result shows red color on the top left panel.

Thanks and hope the problem to be solved :)

YJ
Feb 16, 2013 at 2:13 AM
Edited Feb 18, 2013 at 8:54 PM
My example is here. http://sdrv.ms/XMjF5y

You should be able to just start the app and it asks you where to save a file. The file should contain 3 lines. Red at top, then green, then blue at bottom. But the red and blue are swapped.

-Karl
Coordinator
Feb 16, 2013 at 4:58 PM
The swapped colors are related to the InkManager and the FromStream method. The InkManager uses Bgra instead of Rgba which the FromStream method expected.
I added a new optional parameter to the FromStream method where you can provide the BitmapPixelFormat.

In your case you have to change the FromStrean call to:
foregroundBmp = await new WriteableBitmap(1, 1).FromStream(a, BitmapPixelFormat.Bgra8);
The code changes are checked in but I did not built a new NuGet package. Please grab the latest source code from source control for now and built your own version.
  • Rene
Feb 17, 2013 at 1:48 AM
I really appreciate your lightning support.
Thank you very much :)
Feb 17, 2013 at 7:33 PM
Edited Feb 18, 2013 at 8:55 PM
I haven't had time to try it out, but I will on Monday. Thanks for the help.

-Karl


Edit: It works. Thanks.

I'm still having problems with white color.
Light gray seems to work.

In this test project http://sdrv.ms/XoxNB7

The white one is missing between the red and the light gray as shown below.
(this sample project uses the nuget package so the RB swap is still there)
Image
Apr 18, 2013 at 6:50 AM
Edited Apr 18, 2013 at 6:51 AM
Hi Guys,

I am trying to add source image to destination image using blit function.

I used above code but it is not working for me.

Can any one help me please.

Here is my code..


Dim objRoomItem As SurveyDataSource.RoomItem = DirectCast(itemListView.SelectedValue, SurveyDataSource.RoomItem)
        Dim strImageData As String = GetByteArray(objInkManager)
        'cvink.Children.Clear()


        Dim background = BitmapFactory.[New](imgItem.ActualWidth, imgItem.ActualHeight).FromByteArray(Convert.FromBase64String(objRoomItem.ImageFile))

        Dim forground As WriteableBitmap = BitmapFactory.[New](objInkManager.BoundingRect.Width, objInkManager.BoundingRect.Height).FromByteArray(Convert.FromBase64String(strImageData))

        forground.Blit(New Rect(0, 0, background.PixelWidth, background.PixelHeight), background, New Rect(0, 0, background.PixelWidth, background.PixelHeight), WriteableBitmapExtensions.BlendMode.Alpha)

        'background.Blit(New Rect(0, 0, background.PixelWidth, background.PixelHeight), forground, New Rect(0, 0, forground.PixelWidth, forground.PixelHeight), WriteableBitmapExtensions.BlendMode.Alpha)
        'background.Clear(Windows.UI.Colors.White)

        Dim tempData As Byte()
        ReDim tempData(forground.PixelBuffer.Length)

        tempData = background.PixelBuffer.ToArray()

        strImageData = Convert.ToBase64String(tempData)
Apr 20, 2013 at 9:30 AM
Hey guys,

I am waiting for your reply.

Please its urgent.!!!!:(