Bug report

Describe the bug
I'm not sure if the bug is in libvips, libheif or libaom, but you got to start somewhere...

When converting a PNG (specifically PNG, WEBP does not exhibit this for example) image to an AVIF image of the same dimensions, libvips takes 10x longer than when resizing the image as well.

To Reproduce
Steps to reproduce the behavior:

1. Use Image source
2. vipsthumbnail.exe .\source.png --size 3840x -o destination.heif[compression=av1,lossless=false,effort=2]
3. libvips takes roughly 11 seconds on my machine (effort=2 to keep it lower).

Expected behavior
Running the above command slightly different results in it only taking ~1 second to be done.
vipsthumbnail.exe .\source.png --size 4000x -o destination.heif[compression=av1,lossless=false,effort=2]

You can also downsize and observe the same behaviour.

vipsthumbnail.exe .\source.png --size 3400x -o destination.heif[compression=av1,lossless=false,effort=2]

Or simply only change one dimension.

vipsthumbnail.exe .\source.png --size 3840x1080 -o destination.heif[compression=av1,lossless=false,effort=2]

Actual behavior
Converting to AVIF without resizing the image takes ~10x as long.

Screenshots

vipsthumbnail.exe --vips-info .\source.png --size 4000x -o destination.heif[compression=av1,effort=2,lossless=false,Q=80,strip=true]
VIPS-INFO: 10:10:54.474: thumbnailing .\artwork_landscape.png
VIPS-INFO: 10:10:54.491: selected loader is VipsForeignLoadPngFile
VIPS-INFO: 10:10:54.497: input size is 3840 x 2160
VIPS-INFO: 10:10:54.504: loading with factor 1 pre-shrink
VIPS-INFO: 10:10:54.519: pre-shrunk size is 3840 x 2160
VIPS-INFO: 10:10:54.524: converting to processing space srgb
VIPS-INFO: 10:10:54.529: residual scale 1,04167 x 1,04167
VIPS-INFO: 10:10:54.535: converting to output space srgb
VIPS-INFO: 10:10:54.540: thumbnailing .\source.png as .\destination.heif[compression=av1,effort=2,lossless=false,Q=80,strip=true]

vipsthumbnail.exe --vips-info .\source.png --size 3840x -o destination.heif[compression=av1,effort=2,lossless=false,Q=80,strip=true]
VIPS-INFO: 10:11:09.687: thumbnailing .\artwork_landscape.png
VIPS-INFO: 10:11:09.704: selected loader is VipsForeignLoadPngFile
VIPS-INFO: 10:11:09.712: input size is 3840 x 2160
VIPS-INFO: 10:11:09.734: loading with factor 1 pre-shrink
VIPS-INFO: 10:11:09.742: pre-shrunk size is 3840 x 2160
VIPS-INFO: 10:11:09.750: converting to processing space srgb
VIPS-INFO: 10:11:09.759: converting to output space srgb
VIPS-INFO: 10:11:09.763: thumbnailing .\source.png as .\destination.heif[compression=av1,effort=2,lossless=false,Q=80,strip=true]

Environment
(please complete the following information)

• OS: Debian Bullseye, Windows 10
• Vips: 8.13.0, 8.12.2

Additional context
We managed to reproduce this locally with the prebuilt Windows binaries, but also with Linux binaries built from source with both libaom 3.4.0 and libaom 3.3.0, and both vips 8.13.0 and 8.12.2.

Upscaling and downscaling an image takes roughly ~2 seconds, compared to not-scaling the image at all taking ~11 seconds.

Interestingly the "slow" version has a maximum memory consumption of 17MB, whereas upscaling it takes 38MB.

vips-profile-slow.txt
vips-profile-fast.txt