Transparency gets lost with rawsave_buffer #4585
-
I'm not sure if this is a bug or expected behavior, so I'm posting here to get your opinion. We're using the .NET bindings and in case it has something to do with that, I'm happy to move it there if more appropriate. But it's more likely a case of "you're using it wrong". I have a complete tiny reproduction case https://github.com/martindisch/AlphaRepro but will explain in more detail here. We're building an image pipeline to create 8 smaller thumbnails of various sizes from 130px to 2880px. Let's assume our original image is very large so it takes a while to process, for example 10'000px. We started out by first creating an initial thumbnail that is as large as our largest thumbnail, then using that as the base for all subsequent thumbnail operations. The thinking being that maybe the first thumbnail operation is cached and can be reused for the others, meaning it doesn't have to start from the full 10k image for each, making it faster (not actually true, but we'll get to that). Also note that when saving the image as JPEG (which doesn't have an alpha channel), we're setting the background to 255 to get white instead of black. This will be important later. So far, so simple and misguided 😉 var inputBytes = await File.ReadAllBytesAsync("Rectangles.png");
// This is the original 10000x10000 image
using var image = Image.NewFromBuffer(inputBytes);Add commentMore actions
// Use a more manageable size as a base for the other thumbnails
using var preThumb = image.ThumbnailImage(2880, size: Enums.Size.Down);
foreach (var width in new[] { 2880, 1440, 1000, 720, 480, 320, 260, 130 })
{
var stopwatch = Stopwatch.StartNew();
using var thumb = preThumb.ThumbnailImage(width, size: Enums.Size.Down);
var bytes = thumb.JpegsaveBuffer(background: [255]);
await File.WriteAllBytesAsync($"out/Rectangles_{width}.jpeg", bytes);
Console.WriteLine($"Created thumbnail with width {width} pixels in {stopwatch.Elapsed}");
} As it turns out, our thinking regarding the optimization of the pipeline behavior is wrong—it looks like libvips will start from the big 10k image for each thumbnail, which takes quite a while (well, two seconds on my machine): $ dotnet run
Created thumbnail with width 2880 pixels in 00:00:03.0623880
Created thumbnail with width 1440 pixels in 00:00:02.2564205
Created thumbnail with width 1000 pixels in 00:00:02.1876179
Created thumbnail with width 720 pixels in 00:00:02.3148522
Created thumbnail with width 480 pixels in 00:00:02.3396053
Created thumbnail with width 320 pixels in 00:00:02.2443918
Created thumbnail with width 260 pixels in 00:00:02.3450774
Created thumbnail with width 130 pixels in 00:00:02.1492182 That's an interesting observation for later, but not the issue here. Besides that, everything works and we get our JPEGs with white background as expected. Because our original attempt to have libvips start with a smaller thumbnail for each operation didn't work out, we next tried to save the base thumbnail to memory in order to force libvips to evaluate up to that point and guarantee starting from this smaller image each time. --- a/Program.cs
+++ b/Program.cs
@@ -7,13 +7,22 @@
using var image = Image.NewFromBuffer(inputBytes);
// Use a more manageable size as a base for the other thumbnails
+// and force libvips to fully build it in memory
using var preThumb = image.ThumbnailImage(2880, size: Enums.Size.Down);
+var preThumbBytes = preThumb.RawsaveBuffer();
+using var preThumbImage = Image.NewFromMemory(
+ preThumbBytes,
+ preThumb.Width,
+ preThumb.Height,
+ preThumb.Bands,
+ preThumb.Format
+);
foreach (var width in new[] { 2880, 1440, 1000, 720, 480, 320, 260, 130 })
{
var stopwatch = Stopwatch.StartNew();
- using var thumb = preThumb.ThumbnailImage(width, size: Enums.Size.Down);
+ using var thumb = preThumbImage.ThumbnailImage(width, size: Enums.Size.Down);
var bytes = thumb.JpegsaveBuffer(background: [255]);
await File.WriteAllBytesAsync($"out/Rectangles_{width}.jpeg", bytes); This does work and has an effect on the timings: $ dotnet run
Created thumbnail with width 2880 pixels in 00:00:00.0325585
Created thumbnail with width 1440 pixels in 00:00:00.0302317
Created thumbnail with width 1000 pixels in 00:00:00.0289217
Created thumbnail with width 720 pixels in 00:00:00.0313277
Created thumbnail with width 480 pixels in 00:00:00.0216524
Created thumbnail with width 320 pixels in 00:00:00.0193554
Created thumbnail with width 260 pixels in 00:00:00.0193175
Created thumbnail with width 130 pixels in 00:00:00.0168024 However, here comes the issue: the background of the JPEGs is now black as if the transparency information would get lost somewhere along the way. Some additional observations:
|
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 3 replies
-
Hello @martindisch, I'd use vips format ( In python, for example: image = pyvips.Image.thumbnail("some-source-image.png", 2880)
image.write_to_file("tmp-2880.v")
image = pyvips.Image.new_from_file("tmp-2880.v")
image.write_to_file("thumb-2880.jpg", background=255)
image = pyvips.Image.thumbnail("tmp-2880.v", 1440)
image.write_to_file("tmp-1440.v")
image = pyvips.Image.new_from_file("tmp-1440.v")
image.write_to_file("thumb-1440.jpg", background=255)
If you have plenty of RAM, you can also use memory for intermediates with Perhaps: image = pyvips.Image.thumbnail("some-source-image.png", 2880).copy_memory()
image.write_to_file("thumb-2880.jpg", background=255)
image = image.thumbnail_image(1440).copy_memory()
image.write_to_file("thumb-1440.jpg", background=255)
etc. Finally, you can flatten out the alpha at the beginning of the process, rather than removing it during each JPG save. Perhaps: image = pyvips.Image.thumbnail("some-source-image.png", 2880).flatten(background=255).copy_memory()
image.write_to_file("thumb-2880.jpg")
image = image.thumbnail_image(1440).copy_memory()
image.write_to_file("thumb-1440.jpg")
etc. It should be a bit quicker. |
Beta Was this translation helpful? Give feedback.
-
You have: var inputBytes = await File.ReadAllBytesAsync("Rectangles.png");
// This is the original 10000x10000 image
using var image = Image.NewFromBuffer(inputBytes);Add commentMore actions
// Use a more manageable size as a base for the other thumbnails
using var preThumb = image.ThumbnailImage(2880, size: Enums.Size.Down); I suppose you are simulating reading from a pipe? You can do this in one step with // make a source object that reads from a pipe
var source = NewFromDescriptor(file-descriptor);
var image = ThumbnailSource(source, 2880, size: Enums.Size.Down); (sorry, I've not written C# for a few years, I probably messed up the syntax) That will thumbnail directly from the pipe with no need to buffer the input image in memory. You can make a custom source that can read from anything, in case this isn't coming from a file descriptor. |
Beta Was this translation helpful? Give feedback.
-
I messed about a bit and I think this would probably be the best approach: #!/usr/bin/env python3
import sys
import pyvips
# thumbnail to the largest size in memory, flattening any alpha
image = pyvips.Image.thumbnail(sys.argv[1], 2880)
if image.hasalpha():
image = image.flatten(background=255)
image = image.copy_memory()
image.write_to_file(f"thumb-2880.jpg")
# for subsequent sizes, thumbnail_image to memory
for size in [1440, 1000, 720, 480, 320, 260, 130]:
image = image.thumbnail_image(size).copy_memory()
image.write_to_file(f"thumb-{size}.jpg") I made a large test PNG with alpha:
Running the prog ^^ I see:
So 2 seconds, 220mb of RAM. There's not a lot of concurrency in this test (at least 50% of time is spent in PNG decode!) and this PC has 32 threads, so you save quite a bit of memory by dropping the threadpool size down. |
Beta Was this translation helpful? Give feedback.
-
Hey all, thank you so much for the quick, helpful and detailed (even in C#!) responses. You guys rock! The reason I'm doing all the inefficient loading and saving of bytes myself instead of using the optimized libvips facilities is that we're part of a processing pipeline that already has everything in memory. So the Thanks and all the best, libvips/NetVips has really been amazing for us so far 👍 |
Beta Was this translation helpful? Give feedback.
I messed about a bit and I think this would probably be the best approach:
I made a large test PNG with alpha: