Super resolution ... ?

[Guppy]

Hi

Adobe is giving the new Camera Raw 13.2 some new options.
One of them is "Raw Details" and another one is "Super Resolution".

The comparison images show a detail from the same shot at a magnification of 2:1.
Camera Nikon D810, lens Rodenstock Apo-Rodagon-D 2x f=75mm 1:4.5.
Recorded with the camera as RAW with maximum resolution of 7360x4912 at 14bit color depth, ISO 64.

First the RAW image optimally sharpened during development:


Developed the following image with the new "Raw Details" option:


And following developed with the "Super Resolution" option.
The image is then twice as large with the edge length, so 14720x9824.
Here I then reduced the resolution back to the original size (7360x4912).


The image with the super resolution is about 10% -20% higher resolution, but shows sharpening artifacts.
(You should not use the default setting for sharpening in Camera Raw. Better is, Sharpen = 0!)

With normal development, maximum sharpening and contrast enhancement, such a high resolution can never be achieved.
The most successful way to increase resolution is to apply Super Resolution to RAW images.
All this without PixelShift!

It will have to be seen how well this simple and quick to accomplish option of Super Resolution, proves itself in practice.

Kurt

[seta666]

I do not like those big halos, I would never use it.
Also I normally export files as 16bit tiff..that upscaling would fill my hdd pretty fast.

The RAW details looks better but seems to affect colour..at least on my phone it does..

[Guppy]

Hola seta666
Gracias por su comentario.

Now we are talking about the same shots in RAW 14Bit format with the Nikon D810
and the Rodenstock Apo-Rodagon-D 2x 75mm 1:4.5 at an image scale of 3:1,
maximum resolution of 7360x4912 at 14bit color depth, ISO 64 recorded.

Different is the development of the RAW images.
Normal development with Camera Raw
and for comparison with Camera Raw 13.2 with the Super Resolution option.

First the overview:


Following is a section of the unprocessed image from Zerene Stacker without Super Resolution.


Now the "optimal" processing of the image without Super Resolution.


The following is the unprocessed section from Zerene Stacker,
when the frames were developed with the Super Resolution option.
(The frames in DNG format with the size of 14720x9824 pixels,
were reduced to the original size (7360x4912 pixels) and converted to TIF format,
the resolution of the object is not reduced in the image.
These images (TIF 7360x4912 pixels) were stacked with Zerene.)
The following is the unprocessed section from Zerene Stacker,
when the frames were developed with the Super resolution.
When developing with Camera Raw, the image should not be sharpened!


Kurt

[seta666]

Well; that looks better now..May work with some subjects but I do not think it can recover details lost by diffraction..
I will check it up, thanks!!

[rjlittlefield]

See https://blog.adobe.com/en/publish/2021/03/10/from-the-acr-team-super-resolution.html . To my eye, it gives a good description of Adobe Camera Raw's "Super Resolution" feature.

My version of a short explanation is that this feature is in the business of recognizing low-resolution versions of things that it knows about, and turning them into cleaner high resolution versions of the same things.

I expect it will work really well, or not so well, depending on how your subject matter matches the patterns that their machine learning system was trained on.

As a matter of principle, no, it cannot recover details that were lost to diffraction.

On the other hand, it may be perfectly willing to provide results that look like details that were lost to diffraction.

Whether that actually happens is a matter for experiment.

I expect this product is typical of most machine learning results, where even the creators don't know what the thing is actually using for clues while it's producing its result. You might think, for example, that an automatous vehicle vision system trained to recognize STOP signs would be paying attention to the shape, color, and letters. But that is demonstrably not the case, as discussed at https://bair.berkeley.edu/blog/2017/12/30/yolo-attack/ and the many articles that it links to.

Ironically, it looks from Guppy's results that one thing ACR did not train very heavily on was charts of resolution bars!

--Rik

[MarkSturtevant]

This does look interesting, though. Maybe an advance on this feature would be for it to prepare different "solutions" to its algorithm, and to then grant the user the ability to inspect those solutions. The human will know better than the machine what will be the better solution.

[Guppy]

Hi Rik
Thank you very much for the information.

Hi Mark
Also the human intelligence shows us that it is not reliable and therefore not always suitable and good.
In the meantime, I have also received results that are not better and also those that are worse. ☹

Kurt

[seta666]

For focus stacking I would probably just use the Enhance details feature; I do not really feel confortable with software adding textures that it may think do fit
Eg; Feathers texture on a butterfly´s wing scales, AI may think they are same thing

[Guppy]

Hi

The following comparisons are with the same RAW file,
only the development is different!

The Artificial Intelligence (AI) knows only one animal, the zebra.
That's why the stripe pattern is also well recognized and cleanly displayed by the resolution test target.
If you look at the stripe lines (resolution) in the following images,
in the first image, with normal development using Adobe Camera Raw (ACR),
a resolution of about 190LP/mm can be seen.
In the second image with ACR and Raw-Details, the stripes have a resolution of 220LP/mm
and the third image with Super Resolution also shows a maximum resolution of 220LP/mm but much higher contrast.
With the option Raw-Details and Super Resolution zebras are resolved about 16% higher.

1) ACR Standard Development


2) ACR developed with the Raw-Details option


3) ACR developed with the Super Resolution option


Now looking at the caption, the number 400 is detected,
at the next smaller number, one does not know if it is 410 or 450?
The number is correctly 450, the next smaller number is 500.
This is about the same for all images, minim better is Raw-Detail, but not 16% higher resolution.
Numbers are not recognized better in the images with the Raw-Detail or Super Resolution option, no real smaller details are displayed.
Artificial intelligence recognizes zebras, but numbers and unknown structures do not.
So no higher resolution!

Images 4 to 6, cutout strongly enlarged font for comparison

4) ACR standard development


5) ACR developed with the Raw-Details option


6) ACR developed with the Super Resolution option



SUMMARY

My apologies.
Under the term Super Resolution I assumed that the photographed object is developed with higher resolution from the RAW file,
but this is not the case.
By super resolution is meant an optimized "empty (blank?) enlargement", with quadrupling of the number of pixels in the image.

If you don't want to print a square meter sized image, the whole thing is not particularly interesting.

Kurt

[seta666]

That 450 could be guessed from just sharpening the RAW file; one thing I do not like about those two methods is what it does to colour, blacks are turned into reds.
Maybe that is just an extreme example but one could think anything not captured by the camera the software just makes up.
Thanks for posting this, I did not even know these features existed

[physicsmajor]

I'm pretty sure they are un-doing some aspect of the Bayer interpolation to get these results, potentially only in areas of high contrast at the Bayer limit. The sharpness uncovered is real but it is only 'seen' by colored subpixels and there's no physical way to expose it without color artifacts. Thus you get to choose: subpixel level sharpness, almost as if there was no Bayer filter but with color artifacts - or accurate colors sacrificing a small amount at the extreme of potential lp/mm.

I personally would not use this feature.