Automatic Image Format Selection Based on Browser#356

SUMMARY
It would be nice to have a format=auto option in Kirby’s thumb generator, that would automatically pick a file format based on the current browser’s capabilities - e.g., send an AVIF file for the latest Firefox, and WebP for Safari. The ‘auto’ option could be the default, but still allowing us to specify a specific format, if needed.

DETAILED USE-CASE
With version 3.6, Kirby will be able to generate images for us in WebP, AVIF, as well as JPG and PNG, which is awesome. This helps developers automatically create the vast amount of images that are needed in order to support all browsers and sizes optimally - which nowadays is a ridiculous amount. For example, to display a single image on the page, we often have to code something like this:

<picture>
    <source type="image/avif" 
            srcset="<?= $img->thumb(['width'=>300, 'format' => 'avif'])->url() ?> 300px,
                    <?= $img->thumb(['width'=>600, 'format' => 'avif'])->url() ?> 600px,
                    <?= $img->thumb(['width'=>900, 'format' => 'avif'])->url() ?> 900px"
             sizes="(min-width: 640px) 300px, 100%" >
    <source type="image/webp" 
            srcset="<?= $img->thumb(['width'=>300, 'format' => 'webp'])->url() ?> 300px,
                    <?= $img->thumb(['width'=>600, 'format' => 'webp'])->url() ?> 600px,
                    <?= $img->thumb(['width'=>900, 'format' => 'webp'])->url() ?> 900px"
             sizes="(min-width: 640px) 300px, 100%" >
    <img src="<?= $img->thumb(['width'=>300, 'format' => 'jpg'])->url() ?>"
         srcset="<?= $img->thumb(['width'=>300, 'format' => 'jpg'])->url() ?> 300px,
                 <?= $img->thumb(['width'=>600, 'format' => 'jpg'])->url() ?> 600px,
                 <?= $img->thumb(['width'=>900, 'format' => 'jpg'])->url() ?> 900px" 
         sizes="(min-width: 640px) 300px, 100%" 
         alt="tropical beach sunset">
</picture>

That is because we want to control which file format gets displayed by the browser, based on their capability. In this example, if the browser supports AVIF (the most compresed format), it will load it. If not, it will try WebP, and finally, if it is an older browser, it might fallback on JPG.

But the fact is, that we often already know which browsers support which formats - eg., based on https://caniuse.com. So, in theory, we could actually make that decision ourselves, based on the current user-agent in the request. If the file format was automatically decided based on the current browser, we wouldn’t need to use the <picture> element in the example above, and could just use the standard <img> element instead:

<img src="<?= $img->thumb(['width'=>300])->url() ?>"
         srcset="<?= $img->thumb(['width'=>300])->url() ?> 300px,
                 <?= $img->thumb(['width'=>600])->url() ?> 600px,
                 <?= $img->thumb(['width'=>900])->url() ?> 900px" 
         sizes="(min-width: 640px) 300px, 100%" 
         alt="tropical beach sunset">

There will be situations where we might still need to specifically generate a PNG, or JPG, or another format, so having the format option would still be needed.

After thinking some more about this, I can see some potentially serious drawbacks to this, too - namely, that it would means that pages that use this would have to be dynamically generated with every request, and would not be able to be cached.

If the developer is not using page caching at all, or if a particular page doesn’t need it, this could still be useful - but perhaps it would be more prudent not to have it as a ‘default’ setting for thumb()…

Why would it have to be dynamically generated? It seems to me that the implementation is pretty much the same the only difference is that thumb() has to encode multiple images not one.

The main difference is on the webserver level. AFAIK the biggest trouble people mentioned is that you have to be careful about cashing of the images because you suddenly serve multiple assets under one url.

There is nice article here https://corydowdy.com/blog/webp-jxr-nginx-content-negotiation-test where you can test if it works in your browser.

The article mentioned is over 6 years old, and a lot has changed in Nginx, browsers and PHP in that time. The idea of serving multiple, varying STATIC assets (like images) from the same URL is unintuitive, and likely to cause issues with most caching mechanisms. The Nginx configuration that is suggested has potential security issues, might impact server performance, and it makes it a lot more difficult to integrate Kirby plugins like Cachebuster, which also require tweaks to routing at server-level.

It may be safer and easier to use a combination of browser-side JS and a Kirby plugin to do this. The JS library would work similarly to a lazyloading one: it could detect the browser’s support for different image formats, and then dynamically inject/replace the images on the page, by requesting an image in the appropriate format from the server. On the Kirby side, we could create routes that intercept these image requests, then create and/or return the right image based on the file extension. That would keep everything compatible with all caching mechanisms - including Kirby’s - and not require any special server configuration at all.

FYI: All browsers send the supported formats (mime types) in the HTTP request (FF in this case):
Accept: image/avif,image/webp,/

Since $app is aware of this infomation it could set a config value thumb() may check prior generating the actual URL for the template. This way you’d have a default format for the current browser session.
There’s no need for a JS lib and client side sniffing. It’s always been in the request header for ages.

And I think its actually way more supported than those 6 years ago. And many companies are using this. Considering its in googles webp docs as one of the ways how to implement new formats correctly.

I’ve tested my browsers and they all send the headers. (my safari is pretty old so thats why no webp).

I don’t think there are that many changes to make this work

1. Nginx/Apache config for public folder setup. (already around from other projects)
2. Change to the thumb() that checks the Accept headers header and returns/renders the appropriate file.

You can take a look at this page method suggestion: https://forum.getkirby.com/t/thumb-array-rule-doesnt-work/23056/40

Regarding caching: If the response contains Vary: Accept, then any caches will know that a different response may be returned when the client sends a different Accept header.

So I think the behavior should be:

• There could be a global config.php option that controls the default thumb formats. E.g. the default would be to use the same format as the input (as before), but you could also set an array like ['avif', 'webp'] with additional formats that are created for each thumb, with the order defining the priority if the browser supports multiple formats. We could also hardcode this behavior in Kirby (e.g. we would always create these formats if no particular format is requested, but this would lead to reduced thumb creation performance and increased thumb storage).
• On each call to $file->thumb() without an explicit format, Kirby would create all configured or hardcoded thumb formats. The resulting thumb URL will have the extension .auto.
• If that .auto URL is requested, the Kirby router will determine the best format based on the browser support and priority configuration. It will also set a Vary: Accept header for caching.
• Advanced users could create a rule for their web server (nginx, Apache, Caddy etc.) that does this matching directly in the web server for better performance.

Using image paths with an ‘.auto’ extension, that would be trapped and dynamically mapped by Kirby, could work - and shouldn’t cause any issues with page caching mechanisms.

On the server side, we could setup an ‘intelligent proxy’, with a little logic in the config files that would check the headers sent by the browser, and then try to return an image file (with the appropriate extension) based on the formats accepted. So, if the browser requests “/page/image.auto”, and the server sees that it accepts ‘avif’, then the server would try to serve “/page/image.avif” instead. But we’d still need the Kirby router to do some magic here (eg., to server the image from the ‘media’ folder), and for Kirby to auto-generate the image, if it doesn’t already exist.

I’m not sure whether putting some of the routing logic on the server - rather than handling it all inside Kirby - would be any faster than just letting the Kirby router handle it all on its own. Handling it all inside Kirby also has the added benefit that we then don’t need different config files for different web servers (Apache, Nginx, LightSpeed, etc.).

Implementing this directly in the server configuration would definitely be a variant for advanced users. We cannot provide an example configuration for every web server, but we can implement the feature in a way that is easy to augment with server configuration (i.e. it would all be based on media URLs and the URLs would follow a clear structure that can be used by the server configuration).

The media auto-generation is already handled by Kirby at the moment, so we can use it for this enhancement as well.

+1 for being able to configure everything inside Kirby - I don’t know how to configure the server, and don’t even know if I can on my shared hosting account.

Well Kirby already provides .htacces - config thats required for Kirby to work with Apache webserver which is what majority of people use. So you are configuring webserver probably without knowing it. There rules could be added there.

It’s issue for other webservers but LiteSpeed is only one used by shared hosts and AFAIK also uses .htacess. Nginx and Caddy would need configuration from developer but users of those webservers mostly already have to do that.

In any case i don’t think having the logic in PHP wouldn’t be that complicated i just think we shouldn’t encourage it - it’s wasteful. Images should be served by webserver without firing up PHP.