how to retouch stereo images

Images taken in a controlled environment or with a posed subject. All subject types.

Moderators: Chris S., Pau, Beatsy, rjlittlefield, ChrisR

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

how to retouch stereo images

Post by WojTek »

Hello everybody,
I have been playing with cross-eyed stereo for a few days.
So, I have just created the following photo with Zerene.
Image
https://live.staticflickr.com/65535/530 ... 601e_o.jpg
As you can see, there is a lot of retouching to do.
But what's the best way to do it?
Best, ADi
Last edited by WojTek on Sat Jul 08, 2023 3:41 am, edited 1 time in total.

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

Re: how to retouch stereo images

Post by WojTek »

Hello everyone,
Next stereo image stacked with Zerene:
Image
https://live.staticflickr.com/65535/529 ... 37b4_o.jpg
Here you can also clearly see that the retouching would be necessary.
Best , ADi

rjlittlefield
Site Admin
Posts: 24434
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: how to retouch stereo images

Post by rjlittlefield »

I have just now rewritten the Retouching section of https://zerenesystems.com/cms/stacker/docs/syntheticstereo to read as follows:
Retouching

In most cases you cannot retouch shifted outputs from unshifted inputs because image features will not line up correctly.

So, the key trick for retouching shifted images is to retouch shifted outputs from other outputs that were generated with the same shift.

You can use a targeted Stack Selected, or systematic batch slabbing, to generate the shifted outputs to use as source for retouching.

As a general rule, we recommend that you avoid retouching of stereo images, except to fix transparent foreground artifact. The reason for this recommendation is that any inconsistency between the two images will be glaringly obvious when viewed as a stereo pair. At the least, mismatching areas will call the viewer's attention by appearing to “flash”. At worst, if the retouching process has resulted in horizontal shifts of any patch, then in the stereo view that patch will be seen as pulled forward or pushed backward, so the apparent geometry will have been changed.

It is extremely difficult to avoid such inconsistencies when retouching each image independently, except for the special case of fixing transparent foreground artifact by retouching from Stack Selected of the foreground regions.

As a result, it often takes much more time and trouble to retouch a stereo pair than it does to retouch a single image to the same level of residual artifact.

So consider yourself warned: avoid retouching stereo images except to fix transparent foreground.

As an alternative, consider presenting your results in two forms: a retouched single image that is free of artifact but whose geometry may be unclear, combined with a stereo pair that clearly shows the geometry but may suffer from various artifacts.

Now, if you do want to proceed with retouching, then start by considering the situation.

If the artifacts to be removed are isolated to just one or a few areas, then the simplest and most efficient approach is to do targeted Stack Selected that covers just the troublesome areas. That will produce shifted outputs that do not show the transparent foreground problem, and you can retouch from the shifted Stack Selected outputs into the correspondingly shifted full stack outputs.

If the artifacts to be removed are widespread through the stack depth, then it can be more efficient to generate a large set of shifted Stack Selected outputs by using batch slabbing, as described below. However, again we note that even at its best this process takes a lot of time and trouble to do well, so we do not recommend retouching stereo pairs or multi-frame animations that have widespread artifacts.
I hope this helps, but I expect there will still be issues that are unclear or confusing. Please ask about those.

--Rik

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

Re: how to retouch stereo images

Post by WojTek »

Hello Rik,

Thanks for the info!

OK, I would first try to retouch the source files.
To do that, I would create the small number of substacks.
Then I would retouch the substacks with LR,
then synchronize, and finally edit and retouch individually if needed.

But what would be the smallest number of substacks,
so that the stereo image looks good.

Best, ADi

Pau
Site Admin
Posts: 6262
Joined: Wed Jan 20, 2010 8:57 am
Location: Valencia, Spain

Re: how to retouch stereo images

Post by Pau »

ADi, I see both pairs as parallel stereos.
Being unable to cross my eyes I use StereoPoto Maker and mirror glasses. I see the right stereo as posted and when I switch sides they look weird.
Pau

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

Re: how to retouch stereo images

Post by WojTek »

Pau wrote:
Mon Jun 19, 2023 1:09 am
ADi, I see both pairs as parallel stereos.
Being unable to cross my eyes I use StereoPoto Maker and mirror glasses. I see the right stereo as posted and when I switch sides they look weird.
Hello Pau,
I know this :-) and use the following glasses:
http://www.stereoskopie.com/Handbetrach ... oskop.html
Best, ADi

rjlittlefield
Site Admin
Posts: 24434
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: how to retouch stereo images

Post by rjlittlefield »

WojTek wrote:
Mon Jun 19, 2023 12:44 am
But what would be the smallest number of substacks,
so that the stereo image looks good.
It varies depending on the subject and how critical the viewer is. I have seen stereo images made from as few as 15 frames that looked OK on casual inspection. And I have seen stacks made from 50 frames that broke up into looking like obviously separated layers painted on cardboard when modestly zoomed in.

The fellow who showed me that latter image was quite chuffed with it, until I had him zoom in to look at the head of his wasp. I hated to burst his bubble, but he was glad that I had. It turned out that he had routinely been using a step size that was significantly larger than recommended by the 1/4-lambda criterion as used by the Zerene DOF calculator. So his stacked images were also full of focus banding, which he also had not noticed. To be honest, I probably would not have noticed the focus banding either, because the geometry was quite complex. But when the stereo rendering combined with the large step size to produce separated layers, it became clear that something was wrong.

In general I tend to be quite picky about stereo geometry, because for me the whole point of stereo is to make the geometry clear.

You will have to experiment and decide for yourself what is needed to meet your own goals.

--Rik

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

Re: how to retouch stereo images

Post by WojTek »

Hi Rik,
I was afraid of that too :-(
Well, I'm getting on with the testing :-)
Best, ADi

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

Re: how to retouch stereo images

Post by WojTek »

Hello everyone,

Now the first attempt to retouch a stereo image (very rough).
Retouched in paralell view then switched for crossed eyes.
Does this work at all?

Image
https://live.staticflickr.com/65535/529 ... 671e_o.jpg

M6M2, EF100L, Mitu5x, FB1
Best, ADi
Last edited by WojTek on Sat Jun 24, 2023 11:46 am, edited 1 time in total.

rjlittlefield
Site Admin
Posts: 24434
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: how to retouch stereo images

Post by rjlittlefield »

Yes, this is proper format. As you say, the retouching is rough, but I assume that will get better with practice.

--Rik

physicsmajor
Posts: 110
Joined: Sun May 10, 2020 12:56 pm

Re: how to retouch stereo images

Post by physicsmajor »

rjlittlefield wrote:
Sun Jun 18, 2023 10:34 am
I have just now rewritten the Retouching section of https://zerenesystems.com/cms/stacker/docs/syntheticstereo to read as follows:
Retouching

In most cases you cannot retouch shifted outputs from unshifted inputs because image features will not line up correctly.

So, the key trick for retouching shifted images is to retouch shifted outputs from other outputs that were generated with the same shift.

You can use a targeted Stack Selected, or systematic batch slabbing, to generate the shifted outputs to use as source for retouching.

As a general rule, we recommend that you avoid retouching of stereo images, except to fix transparent foreground artifact. The reason for this recommendation is that any inconsistency between the two images will be glaringly obvious when viewed as a stereo pair. At the least, mismatching areas will call the viewer's attention by appearing to “flash”. At worst, if the retouching process has resulted in horizontal shifts of any patch, then in the stereo view that patch will be seen as pulled forward or pushed backward, so the apparent geometry will have been changed.

It is extremely difficult to avoid such inconsistencies when retouching each image independently, except for the special case of fixing transparent foreground artifact by retouching from Stack Selected of the foreground regions.

As a result, it often takes much more time and trouble to retouch a stereo pair than it does to retouch a single image to the same level of residual artifact.

So consider yourself warned: avoid retouching stereo images except to fix transparent foreground.

As an alternative, consider presenting your results in two forms: a retouched single image that is free of artifact but whose geometry may be unclear, combined with a stereo pair that clearly shows the geometry but may suffer from various artifacts.

Now, if you do want to proceed with retouching, then start by considering the situation.

If the artifacts to be removed are isolated to just one or a few areas, then the simplest and most efficient approach is to do targeted Stack Selected that covers just the troublesome areas. That will produce shifted outputs that do not show the transparent foreground problem, and you can retouch from the shifted Stack Selected outputs into the correspondingly shifted full stack outputs.

If the artifacts to be removed are widespread through the stack depth, then it can be more efficient to generate a large set of shifted Stack Selected outputs by using batch slabbing, as described below. However, again we note that even at its best this process takes a lot of time and trouble to do well, so we do not recommend retouching stereo pairs or multi-frame animations that have widespread artifacts.
I hope this helps, but I expect there will still be issues that are unclear or confusing. Please ask about those.

--Rik
Thanks for this update Rik, it tracks with my experience. Systematic slabbing would be the 'right' way but requires perfection and consistency in how both images are retouched; short of two identical Zerene retouching instances on two side-by-side monitors doing each image of a stereo pair painstakingly matching, it often isn't satisfactory.

The one thing I would add is that I have had decent success generating a DMap pair and a PMax pair, choosing the best overall (I usually prefer DMap), and retouching with the other. Retouching across algorithms is often able to mitigate much of the obvious artifacts with flare and edge artifacts. It isn't perfect but at least efficient, and when you focus only on the worst artifacts it's easier to be symmetric.

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

Re: how to retouch stereo images

Post by WojTek »

Hello,
short of two identical Zerene retouching instances on two side-by-side monitors doing each image of a stereo pair painstakingly matching, it often isn't satisfactory.

You mean the retouching of the two output files.
But they are not identical, they are rotated by an angle.
The rotation axis probably runs vertically in the middle of the object and in the middle of the depth of the object.
So a synchronous retouching is not really possible.

But I have the impression that the background does not carry any 3D information as long as there are no shadows.
Here I tried to remove the background and the 3D effect still remained.
Image
https://live.staticflickr.com/65535/530 ... a70d_o.jpg


I would be interested to know how I can add a shadow afterwards.
What is the relationship between the left image and the right one?
At what angle should the shadow finally be placed (left and right)?

Best, ADi

rjlittlefield
Site Admin
Posts: 24434
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: how to retouch stereo images

Post by rjlittlefield »

WojTek wrote:
Thu Jun 29, 2023 6:19 am
short of two identical Zerene retouching instances on two side-by-side monitors doing each image of a stereo pair painstakingly matching, it often isn't satisfactory.
they are not identical, they are rotated by an angle.
...
So a synchronous retouching is not really possible.
True, the retouching cannot be exactly synchronous. However, it definitely can be painstakingly matched.

For example one can load the left and right side views into Photoshop, arrange them side-by-side, and fuse them into a stereo pair that can be observed in real time while each image is retouched separately. I do this occasionally when I think it is worth the trouble. But it really is a lot of trouble. Photoshop provides no way to automatically avoid vertical disparity, so I have to do that manually by monitoring the numeric coordinates. And Photoshop provides no assistance in getting the horizontal position correct to avoid depth errors, so I have to do that entirely by visually matching depth while I'm doing the retouching. This is all so much trouble that I really should say I do it "rarely", rather than "occasionally".

I have the impression that the background does not carry any 3D information as long as there are no shadows.
This is correct. Stereo 3D depends entirely on determining disparity by matching local textures. No texture, no disparity, no depth information.

I would be interested to know how I can add a shadow afterwards.
What is the relationship between the left image and the right one?
The center depth plane of the stack is in the same place in both images. All other depth planes are shifted left or right in proportion to their position in the stack, multiplied by the shift limits specified at Options > Preferences > Stereo/Rocking.

At what angle should the shadow finally be placed (left and right)?
Imagine that the shadow is cast onto a plane that sits behind the subject, perpendicular to the optical axis. Then following the rule given above, that shadow should simply be shifted left or right in proportion to its depth behind the center of the stack. This situation is simple because the shape of the shadow is identical in both views.

But if you imagine that the shadow is cast onto a horizontal plane that sits under the subject and is viewed obliquely by the camera, then life gets much more difficult. In that case the shadow must have different shapes in the left and right images, because portions of the shadow that are at different depths must be shifted by different amounts. One shadow will be a skewed version of the other. On the bright side, if you mess this up a little bit, the viewer's brain will probably respond by inferring that the shadow is not cast onto a flat plane, but rather onto some other surface that makes the two shadow forms be consistent with each other. If in addition the shadow is blurred, appearing cast by a large light source, then the viewer's vision will not make much inference at all, beyond something like "oh yeah, that's a shadow, below and behind the subject".

--Rik

WojTek
Posts: 2860
Joined: Mon Nov 30, 2015 1:09 pm

Re: how to retouch stereo images

Post by WojTek »

Hello Rik,

Many thanks for the hints!

So, I have just tried but haven’t found any way
to set the different angle of the shadow
for each layer separately up to now.

Finally, I placed the sun in the upper left corner
and I set the angle of the shadow to 135 degrees.

Image
https://live.staticflickr.com/65535/530 ... 445d_o.jpg

Best, ADi

rjlittlefield
Site Admin
Posts: 24434
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: how to retouch stereo images

Post by rjlittlefield »

Is this more what you had in mind?
Riks3DShadow.jpg
I made this one in Photoshop using basic tools. I made a single instance of the shadow using select, fill, and blur in a new layer. I moved that instance to roughly where I wanted it for one eye, then made a second copy of the shadow layer and moved it horizontally to where I wanted it for the other eye. Each eye sees that the subject and shadow are offset differently, which makes the shadow appear behind the subject when viewed in stereo.

--Rik

Post Reply Previous topicNext topic