www.photomacrography.net :: View topic - Automated FocusStacker
www.photomacrography.net Forum Index
An online community dedicated to the practices of photomacrography, close-up and macro photography, and photomicrography.
Photomacrography Front Page Amateurmicrography Front Page
Old Forums/Galleries
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Automated FocusStacker
Goto page 1, 2  Next
 
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Equipment Discussions
View previous topic :: View next topic  
Author Message
masquerade



Joined: 08 Nov 2016
Posts: 9

PostPosted: Tue Nov 08, 2016 4:51 am    Post subject: Automated FocusStacker Reply with quote

Hi Everyone,

I have been making automated macro rails for a while. I wanted to share my experiences and findings through my blog.

I split the serie into 4 parts, which each part has a different approach. My latest build is the closest one in my mind.

The most important aspects I tried to achieve are a super-compact and easy to use design. I guess FocusStacker v4 has it and ready to be in the field.

http://www.ardakutlu.com/focus-stacking-adventures-part-i/

http://www.ardakutlu.com/focus-stacking-adventures-part-ii/

http://www.ardakutlu.com/focus-stacking-adventures-part-iii/

http://www.ardakutlu.com/focus-stacking-adventures-part-iv/
_________________
If it is not in frame, it does not exist
- Murnau
Back to top
View user's profile Send private message
Saul



Joined: 31 Jan 2011
Posts: 886
Location: Naperville, IL USA

PostPosted: Tue Nov 08, 2016 8:38 am    Post subject: Reply with quote

Welcome aboard, Arda !

Your Del-Tron stage works great, I'm very happy with it , very compact & versatile !

Looking forward to see V4 Wink

Cheers,
Saul
_________________
Saul
Studio and field setups
Back to top
View user's profile Send private message Visit poster's website
masquerade



Joined: 08 Nov 2016
Posts: 9

PostPosted: Tue Nov 08, 2016 10:20 am    Post subject: Reply with quote

Hi Saul,

Thank you very much. v4 is out and working Smile I am just making small fixes and tweaks. Once I gather all the materials, I will publish them at my website.


_________________
If it is not in frame, it does not exist
- Murnau
Back to top
View user's profile Send private message
Smokedaddy



Joined: 07 Oct 2006
Posts: 785
Location: Phoenix, Arizona

PostPosted: Tue Nov 08, 2016 12:12 pm    Post subject: Reply with quote

Arda in case you didn't notice, you have a private message. <g>
Back to top
View user's profile Send private message Visit poster's website
Chris S.
Site Admin


Joined: 05 Apr 2009
Posts: 2815
Location: Ohio, USA

PostPosted: Tue Nov 08, 2016 10:46 pm    Post subject: Reply with quote

Arda, welcome to the forum! Very Happy And thanks for sharing with us your macro journey.

I would respectfully disagree with you, however, that having stacking software handle raw files is a useful feature.

Shooting raw has value if--and only if--the photographer takes the time to tweak conversion parameters in robust raw conversion software. Stacking software that uses default values to perform this conversion provides only the illusion of a benefit. This throws away and advantages of shooting raw. A photographer, in such a workflow, would be better shooting jpeg, and carefully tweaking them during in-camera acquisition.

For focus stacking, I sometimes shoot jpeg and other times shoot raw. In my workflow, jpeg is for those cases when careful control of camera settings can place all useful visual data in a jpeg file. For situations where important visual data exceeds the range of jpeg, I shoot raw--and process resultant raw files with care, in order to compress useful visual information the capacity of a tiff or jpeg output. Vitally, this judgment cannot currently be done without thoughtful human input.

So when stacking software is contrived to accept raw files and convert them based on default values, this is adds little or no value, no matter how good it looks on a features checkoff list.

Cheers,

--Chris
Back to top
View user's profile Send private message Send e-mail
masquerade



Joined: 08 Nov 2016
Posts: 9

PostPosted: Wed Nov 09, 2016 4:36 am    Post subject: Reply with quote

Hi Chris,

I see your point. Maybe I couldnt explain it well, sometimes I think I couldnt explain myself clear since English is not my native language, I apologize for that in advance.

Let me try to rephrase it.
What I meant by "the ability to handle the raw files" is that Helicon's feature of reading .dng files. Compared to tif conversion from original raw files (.raw, .nef etc) conversion to .dng is very fast. Because it keeps most of the data same, but only converts the shell which enables it to be identified by the softwares. And the size is smaller then tif files so writing speeds to disk is faster.

There is no data loss when you convert a .raw file to a 16bit .tiff file. Actually most raw formats are 14 bit, so you put it into a bigger container and there is still room for more data. The key point is your range is preserved. There is no clipping assuming you didnt excess the limits of the original raw data.

The advantage of using dng workflow of Helicon is merely a speed and storage issue and since dng files are smaller sizes with the same data range, you can go for higher resolutions with less RAM.

About the .jpg pipeline, I respect that approach if speed, bandwith, amount of RAM etc is an issue for you.
For me, the main priority is quality and sharpness. You will surely lose some data on the way even if you shoot with a perfect lighting which will require no color correction. Because jpg is not a lossless compression, meaning even with the highest quality you will be losing something. For most of the time it may not be noticable or important, but I am not willing to take that risk.

On the other hand, I totally agree with you that one should pay the utmost attention to the lighting conditions, framing and color balance while shooting to minimize the post process (or even to eliminate it completely) Believe me, working in the post production field for ten years, I am well aware it is much better to prevent the mistake than try to patch it up. But we have to face it, everyone makes mistakes. If you somehow require post process, may it be color correction, re-framing, spot cleaning or even reconstructions of large pieces every bit of data is a plus.
_________________
If it is not in frame, it does not exist
- Murnau
Back to top
View user's profile Send private message
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7186
Location: Near London, UK

PostPosted: Wed Nov 09, 2016 5:26 am    Post subject: Reply with quote

Quote:
but I am not willing to take that risk.
A number of us have looked, and not see an improvement with the raw>tiff route. There's nobody I can remember saying it's a better route to go via raw than from a maximum quality Jpeg - UNLESS - you want to edit the raw file.

(I have a nagging suspicion that Canon's in-camera Raw>Jpeg converter isn't quite as good as Nikon's, but it's from anecdotal observations on my part.)
_________________
Chris R
Back to top
View user's profile Send private message
masquerade



Joined: 08 Nov 2016
Posts: 9

PostPosted: Wed Nov 09, 2016 7:23 am    Post subject: Reply with quote

I am not sure if I am missing something or dont understand what you mean. Raw images holds the whole range (either it is 14bit or 16bit) But jpg images can only hold 8bit information + it adds lossy compression.

In my pipeline I open .nef (raw) images either in photoshop or in lightroom and send them to zerene using the zerene-lightroom connectivity plugin (which uses tiff conversion). This way the 16bit depth information has been retained. Then I save the stacked image again as 16bit tiff file. This way I have a 16 bit stacked file which contains the whole range as has been shot by the camera. If the images was converted to a jpg file at any step, the range would be cropped to 8 bit and you cannot get it back.

Are we talking about if the raw files has an advantage over jpg files or if the raw pipeline in Helicon Focus is a good thing or unnecessary thing?

Can you elaborate what you mean by editing the raw file? If removing spots, applying primary and secondary color corrections etc. is editing, I did not even have a single photo which is not edited.
_________________
If it is not in frame, it does not exist
- Murnau
Back to top
View user's profile Send private message
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7186
Location: Near London, UK

PostPosted: Wed Nov 09, 2016 10:29 am    Post subject: Reply with quote

These posts of Rik's, of focus stacking workflow, and Raw file stacking, explain:
http://www.photomacrography.net/forum/viewtopic.php?t=27012

http://www.photomacrography.net/forum/viewtopic.php?t=25268

When I want to use Raw I want to use my choice of raw converter with whatever quality and bells and whistles it has. I have tried using raw source, and ex-camera tiff source, and ex camera fine Large Jpeg, with suitably adjusted lighting on subject, and found no significant difference in a stack output. YMMV.

Of course the output needs editing, but you don't edit every source frame. Spot removal etc can be done to the output just fine, I've found.

I won't swallow the common justification claim : "It's wrong to tolerate any loss" without a demo that the loss is detectable.
Workflow preference, and speed benefits, I accept. It's nice to have all the options.
_________________
Chris R
Back to top
View user's profile Send private message
masquerade



Joined: 08 Nov 2016
Posts: 9

PostPosted: Wed Nov 09, 2016 11:19 am    Post subject: Reply with quote

Quote:
Of course the output needs editing, but you don't edit every source frame. Spot removal etc can be done to the output just fine, I've found.


That is exactly what I mean. Since I dont do that for every shot and since I dont want to do that on a 8bit image, I have no other option then either using the dng or tiff pipeline. There is no turning back once an image turned into jpg.

On the second post link, Rik says actually the same thing that I mentioned:
Quote:
The downside of JPEG is mostly not from compression, but from the fact that sharpening, noise reduction, and color balance are baked into the images and cannot be adjusted much in postprocessing without introducing other problems. The fact that they have 8-bit color also limits flexibility to do curves adjustment in post.


What I try to say is going back to 8bit is not option for me. That is of course a choice, there is no rule about that. As a habit, as long as I dont have a pressing deadline or hardware issue, I never waste even the fraction of my color depth. Not at work, not at home.

I tried to point out in my blog that dng wrapping option in Helicon provides a much faster way of getting a 16bit unclamped stack output compared to tif pipeline in Zerene Stacker. I am not defending that everyone should follow that or another workflow. If you are not going to do any post processing, if you have a limited bandwith and/or hardware, if you dont mind the jpg compression and noise or if you dont have time for data transfers, of couse jpg pipeline stays as an option (which I will never practice Smile )
_________________
If it is not in frame, it does not exist
- Murnau
Back to top
View user's profile Send private message
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7186
Location: Near London, UK

PostPosted: Wed Nov 09, 2016 1:46 pm    Post subject: Reply with quote

Quote:
If you are not going to do any post processing, if you have a limited bandwith and/or hardware, if you dont mind the jpg compression and noise

Or, - if you try it and find those are theoretical rather than actual concerns.
16.8 million colors is usually enough. Smile
_________________
Chris R
Back to top
View user's profile Send private message
masquerade



Joined: 08 Nov 2016
Posts: 9

PostPosted: Wed Nov 09, 2016 2:14 pm    Post subject: Reply with quote

Apparently not enough for everyone. Otherwise everything would be jpg, there wont be anything like exr, dpx, tif.
I sometimes work with 32 bit rendered images for feature films. But you are right, no monitor can show that color depth even it was perceivable by the human eye (which is not). It is about having that room to work with, not seeing the whole range.

I used to think exactly like you do. In one night my whole idea changed, so I wont insist anymore Smile Thanks for reading and listening.
_________________
If it is not in frame, it does not exist
- Murnau
Back to top
View user's profile Send private message
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 18172
Location: Richland, Washington State, USA

PostPosted: Wed Nov 09, 2016 11:43 pm    Post subject: Reply with quote

masquerade, I mostly agree with your position about pixel depth.

However, regarding the dng workflow in Helicon Focus, I think you have some misconceptions.

It appears, from your description at http://www.ardakutlu.com/zerene-stacker-vs-helicon-focus-comparison/ , that you think Helicon is doing the stacking with raw sensor data. It's not.

When you feed Helicon Focus a .dng file, the first thing it does is to convert the image to 16-bit RGB TIFF, in a background process, using default conversion parameters. Then it stacks the 16-bit TIFFs, producing a 16-bit RGB output. When you save the output as DNG, it packs those RGB pixels back into a DNG wrapper, leading a casual observer to think "Ah, DNG in, DNG out, how efficient!"

Let me illustrate this with an experiment that I just now ran. What I did was to take three Canon raw files, type .CR2, and had Lightroom convert them to .dng . Then I fed the .dng files into Helicon Focus, stacked, and saved the output as .dng . While Helicon Focus was running, I also snapped a copy of its cache so that I could look behind the GUI.

Here's what I see when I look at the folder contents using Windows Explorer:





Quickly summarizing, in the input/output folder the original .CR2 files are about 18,017 KB, the .dng files produced by Lightroom are about 15,600 KB, and the .dng output produced by Helicon Focus is 53,125 KB, over 3 times larger than the .dng inputs. In the Helicon Focus cache folder, the TIFF files are even larger: 88,234 KB. That last number is completely consistent with uncompressed 16-bit RGB TIFF: 4752x3168 pixels at 6 bytes per pixel would be 88,209 KB, plus a bit for headers.

So... When your article talks about the speed advantages of raw workflow, it seems pretty badly misdirected. The speed advantages of Helicon Focus come from other causes.

As for why Zerene Stacker does not provide a raw workflow, the reasons for that are outlined in one of the FAQs:
Quote:
A longer explanation is that no stacking software really works directly with raw files.

The structure of data in a typical raw image file, one value per photosite with color implied by a mosaic Bayer filter pattern, is fundamentally incompatible with the image alignment process that is required for stacking.

Some stacking software deals with this aspect by accepting raw files at the level of the user interface, then converting them to some RGB format, typically TIFF, in a background process that is easy to overlook and may be difficult to optimize.

Zerene Stacker goes the other route, exposing the conversion process and encouraging users to deal with it as what we think it really is: a key part of the overall workflow that needs to have some attention paid to it in order to get best results.

Adding a bit more information to that last point in the FAQ... As the fellow who writes Zerene Stacker, I'm acutely aware that stacking methods can benefit from appropriate amounts of sharpening and noise reduction applied before stacking. And as a frequent stacker at many different magnifications, I'm also frequently reminded that "appropriate amounts" vary quite a bit from one stack to another depending on setup. As a result, I'm not at all inclined to provide a workflow in which raw conversion is done invisibly in a background process, using parameters that are at best "out of sight, out of mind". It's a policy decision, to not spend my development time encouraging users to take a shortcut that I think is a bad idea.

To the extent that Zerene Stacker competes with Helicon Focus at all, I think it does so on the basis of output quality versus human time invested. If the nature of someone's stacking is such that computer time is the deciding factor, then they should definitely opt for Helicon Focus, and I'll be happy to make that recommendation.

That said, it's also true that there are ways to make Zerene Stacker run quite a bit faster than the trial configuration that I'm guessing you used for your tests. But that's a topic for another day, and probably a different thread. If you're curious, you might look at http://www.photomacrography.net/forum/viewtopic.php?p=141699#141699 and the discussion continuing onto the following page.

--Rik
Back to top
View user's profile Send private message Visit poster's website
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7186
Location: Near London, UK

PostPosted: Thu Nov 10, 2016 4:32 am    Post subject: Reply with quote

Quote:
I used to think exactly like you do. In one night my whole idea changed,
Point missed, I think. Perhaps not fully expressed.

I'm not arguing against the generality of benefit of having more data to work with (though it's frequently asserted where untested and not real), but the stacking case is somewhat atypical.

As I said, if I want Raw, I'll choose the converter and conversion, not as Rik says leave it to someone else's decisions.

When stacking, then most of the time, you're working with a standard setup, and standard lighting. If you turn off all the "adjustments" that the camera thinks are a good idea, control the lighting, then what you get is well within the Jpeg "space".

This is taking into account that process time was an issue when I started stacking, but it's not a bother now and memory has become almost free.
_________________
Chris R
Back to top
View user's profile Send private message
masquerade



Joined: 08 Nov 2016
Posts: 9

PostPosted: Thu Nov 10, 2016 7:43 am    Post subject: Reply with quote

I am actually not defending that Helicon uses the raw pixel data. None of them use it. Raw data holds the color information with a gamma curve applied. When it is converted to tiff it becomes linear and if there is no compression, it is normal that the tiff file becomes much larger. And actually 16bit Tiff file can hold more data then the Raw file.

About the conversion, you probably mean the Bayer interpolation and White Balance, Saturation, Contrast etc. value conversions right? These are converted automatically when you shoot jpg. But if you shoot raw and convert it to tiff (as long as you dont push the white and black levels beyond edges) since you will have 4096 intensity levels instead of 256 per pixel, you can get the exact same image as you would shoot jpg. And also a room that you can play with.

What I tested was actually more then the abilities of the softwares the time spend during the data transfers mostly. As you can see and download the test results, both of my test results have the same data range. So there is nothing different between them in terms of color depth.

You can try it yourself. When you send the files using Zerene Stackers plugin with a tif export, it takes 3-4 times longer then the .dng pipeline suggested in Helicon's website.

And Helicon does the stacking faster too when you compare these two methods. It may be related with file format (that was my initial guess) but It may be not, I dont know. The only thing I know is if you want to have a 16bit result, dng pipeline is much more faster.
_________________
If it is not in frame, it does not exist
- Murnau
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Equipment Discussions All times are GMT - 7 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group