Unexplained ZereneStacker trouble

pittendrigh · Post by **pittendrigh** » Thu Apr 12, 2018 3:11 pm

I'm at a loss. I have a workflow that has worked well for several years that suddenly doesn't.

I have a Mint Linux AMD64 box.
I use studio lighting for bugs and trout flies.
entangle and Zerene to control a StackShot rail.
Nikon D700 with 105mm macro lens and optional extension rings.

I shoot a stack of perhaps 30 frames.
The run rawtherapee on an arbitrary example from the middle of the stack.
Edit that Nikon.NEF raw with rawtherapee and save the edit sequence to a file I arbitrarily name now.pp3

Then a command line rawtherapee call to apply the same editing changes to all other raw files in the stack.

rawtherapee-cli -o tifs -p ./now.pp3 -t -Y -c .

The above command creates a subdirectory named tifs and converts all raws to tif inside the the subdirectory tifs.

If I cd into the tifs directory and look at any one of the tif images it looks fine in gimp.

But if I now run ZereneStacker.bsh *.tif and try to stack the images (with any method) I get an all black non-image in the Zerene results pane.

I've looked at various preferences and can't see anything amiss. I have Zerene registered with a valid key. I must be missing something obvious. But I'm stuck.

rjlittlefield · Post by **rjlittlefield** » Thu Apr 12, 2018 4:25 pm

This description does not remind me of any problem that I know.

I suggest to build a test case that uses just two of the tiff files. Repeat the problem, then send an email to me via support@zerenesystems.com, including the LastNormalLog.txt file and snapshots of what you see on screen. Shoot the screen with a camera if you have trouble capturing an image using Linux tools. Also send the two tiff files, using DropBox or wesendit.com or whatever other large-file transfer facility you prefer to use.

When I get those materials I hope to be able to reproduce the problem.

One quick question. You mention specifically the results pane. Does the source pane look OK and it's only the results pane that looks black?

--Rik

pittendrigh · Post by **pittendrigh** » Thu Apr 12, 2018 5:59 pm

Ok will do. First thing tomorrow morning. With fresh brain cells.

FWIW before the the Zerene output I got just prior to black was the attached. This has to be ....... I just don't know what to say. The odd thing is the tif files this bizarre image was made from look fine, as do the NEF files the tifs came from. I'll put something together in in the morning. As promised. I appreciate the help.

rjlittlefield · Post by **rjlittlefield** » Thu Apr 12, 2018 6:10 pm

This looks a lot like what happens when even one severely underexposed source frame appears late in the sequence, with Preferences > Alignment > Brightness correction turned on. What happens is that the software blindly cranks up the brightness and contrast of an almost black frame, exposes a huge amount of noise in the process, and then the noise looks like really intense image detail so that frame gets selected. But I've never seen the result go black afterward; it has always stayed garish and speckly.

It's a hint, though. Check all your source frames and make sure that every one of them previews OK.

--Rik

pittendrigh · Post by **pittendrigh** » Fri Apr 13, 2018 7:22 am

This now has to be a rawtherapee-cli problem.

RawTherapee, version 5.3-669-g478e1c0 (this was a recent upgrade)
Using lensfun 0.2.8.0

........hmm. I wonder what lensfun is. I don't remember installing it or activating it.

Instead of using rawtherapee-cli -o tifs -p ./now.pp3 -t -Y -c . to make tif files from Nikon RAW I used Imagemagick in a loop (convert $file $file.tif). That defeats my purpose because I don't get to edit the raws before making the tif files but it does eliminate rawtherapee from the equation.

The tif files I got from the ImageMagick loop work just fine in ZereneStacker. Rawtherapee has worked for me for a long time. I realize now it is only since a recent upgrade that my workflow went haywire.

I suppose I could use UFRAW or DarkTable instead of rawtherapee. Figuring out exactly what is going on will be interesting. UFRAW is primitive. DarkTable would take a day or two of learning curve. What ever this latest and greatest version of Rawtherapee is doing needs to be fixed somehow.

rjlittlefield · Post by **rjlittlefield** » Fri Apr 13, 2018 8:33 am

Very interesting!

If possible, I would still like to receive samples of a couple of tif files that show this problem. It's a well known saying that TIFF actually stands for "Thousands of Incompatible File Formats", but at this moment I cannot think of what could cause the source images to look OK in Zerene Stacker while producing an output like you're getting.

--Rik

pittendrigh · Post by **pittendrigh** » Fri Apr 13, 2018 9:15 am

rjlittlefield wrote:Very interesting!

If possible, I would still like to receive samples of a couple of tif files that show this problem. It's a well known saying that TIFF actually stands for "Thousands of Incompatible File Formats", but at this moment I cannot think of what could cause the source images to look OK in Zerene Stacker while producing an output like you're getting.

--Rik

http://montana-riverboats.com/rawchaos/rawchaos.zip .... zip file with two raws and four subsequent tifs still uploading. Two such tifs made with rawtherapee. Two made with ImageMagick.

The ImageMagick tifs work fine in ZereneStacker. The rawtherapee ones don't, although they used to.

I'm trying to figure out how to use a script to read tif headers. See what differences there might be. I'm assuming the data blocks must be the same.......although? Maybe not!

pittendrigh · Post by **pittendrigh** » Fri Apr 13, 2018 10:40 am

....forgot to mention. The lighting for the two Nikon nikon.NEF files above is not good. This was just a debug session so I didn't take the time to get the strobes adjusted properly, nor the ISO F-stop settings.

rjlittlefield · Post by **rjlittlefield** » Fri Apr 13, 2018 11:20 am

On quick look, the problem is that your rawtherapee files have 32-bit floating point color. They don't produce any error messages when read by Zerene Stacker, but the results are nonsense, even in the source panel.

Try forcing 16-bit color and let me know what happens.

--Rik

pittendrigh · Post by **pittendrigh** » Fri Apr 13, 2018 11:49 am

rjlittlefield wrote:On quick look, the problem is that your rawtherapee files have 32-bit floating point color. They don't produce any error messages when read by Zerene Stacker, but the results are nonsense, even in the source panel.

Try forcing 16-bit color and let me know what happens.

--Rik

Ok that solved it. You can specify bit depth with the -b option in rawtherapee-cli

Not using it worked with older versions of the software. Using it now fixes my problem with this newer version of the software.

It's possible I could have figured that out on my own. But it would have taken days......if not weeks. I'm good to go now. Thank you!

NOTE: from imagemagick identify -verbose filename is a useful tool I didn't know until today.

rjlittlefield · Post by **rjlittlefield** » Fri Apr 13, 2018 8:17 pm

You're very welcome.

For what it's worth, the debug process at this end was pretty quick but had some element of serendipity to it. I first confirmed that my copy of Zerene Stacker, running on 64-bit Windows, saw black images also. Then I tried setting the option to use the "external TIFF reader", which changed the results but only slightly. Then I pulled an image into Photoshop to see if perhaps the color profile was weird. It was, and I had never seen the profile before, so then I thought to convert the file to a different profile, but oddly, Edit > "Convert to profile..." was grayed out. Still, I thought it might be informative to just re-save the file from Photoshop. That got me to a TIFF Options panel in which "32 bit (Float)" was pre-selected, and that was when the light dawned. Of course Photoshop had been showing me "RGB/32*" in the image title bar all along, and I just had not noticed it.

It is clearly a bug that this problem is not diagnosed by Zerene Stacker. I would have expected that any file format problem would eventually produce some sort of exception that would be reported as an "Uh-oh" unexpected condition if nothing else. I'll have to look into why that is not happening in this case.

--Rik