TIFF format
Moderators: rjlittlefield, ChrisR, Chris S., Pau
TIFF format
This is derivative thread from a prior (shown below) stacking thread that was diverging, so redirecting here.
https://www.photomacrography.net/forum/ ... 6&start=60
It seems that all TIFFs are not treated alike, and LR and PS can manipulate the generation of TIFF files from camera RAW in various ways as Robert O'Toole kindly pointed out.
The Nikon D800E (I assume the D800 and D500 and upcoming D850 also) produces 8 bit TIFFs, not 16 bit. Obviously these are smaller, about 1/2 the size of 16 bit TIFF files that LR and PS produce.
However the TIFFs that Nikon produces in the D800E have shown IQ comparable, maybe even slightly better, to those 16 bit versions generated by LR or PS converted from camera RAW files for the special chip subjects I tend to shoot. Robert pointed out that LR and PS generated TIFFS have lots of somewhat hidden variables, including a "Camara Standard" non-default value (the LR default is Adobe Standard, not Camera Standard).
I compared some TIFF files generated in the D800E camera (8 bit), the basic default LR Adobe Standard 16 bit and Camera Standard 16 bit TIFF files generated from D800E RAW files with LR default parameter settings (all 0) of the same subject with same lighting and camera settings. These files turned out to be different as you can see below.
Folks with TIFF experience and knowhow are encouraged to chime in and help. I had thought this was somewhat of a constant, but in Wayne's World context...apparently NOT!!
Best,
Mike
8 bit TIFF Directly from D800E
Converted by LR from D800E RAW with Adobe Standard 16 bit TIFF
Converted by LR from same D800E RAW file with Camera Standard 16 bit TIFF
https://www.photomacrography.net/forum/ ... 6&start=60
It seems that all TIFFs are not treated alike, and LR and PS can manipulate the generation of TIFF files from camera RAW in various ways as Robert O'Toole kindly pointed out.
The Nikon D800E (I assume the D800 and D500 and upcoming D850 also) produces 8 bit TIFFs, not 16 bit. Obviously these are smaller, about 1/2 the size of 16 bit TIFF files that LR and PS produce.
However the TIFFs that Nikon produces in the D800E have shown IQ comparable, maybe even slightly better, to those 16 bit versions generated by LR or PS converted from camera RAW files for the special chip subjects I tend to shoot. Robert pointed out that LR and PS generated TIFFS have lots of somewhat hidden variables, including a "Camara Standard" non-default value (the LR default is Adobe Standard, not Camera Standard).
I compared some TIFF files generated in the D800E camera (8 bit), the basic default LR Adobe Standard 16 bit and Camera Standard 16 bit TIFF files generated from D800E RAW files with LR default parameter settings (all 0) of the same subject with same lighting and camera settings. These files turned out to be different as you can see below.
Folks with TIFF experience and knowhow are encouraged to chime in and help. I had thought this was somewhat of a constant, but in Wayne's World context...apparently NOT!!
Best,
Mike
8 bit TIFF Directly from D800E
Converted by LR from D800E RAW with Adobe Standard 16 bit TIFF
Converted by LR from same D800E RAW file with Camera Standard 16 bit TIFF
- rjlittlefield
- Site Admin
- Posts: 23625
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
Rik,
The RAW files were all generated into TIFF files by LR for stacking in Zerene. So I did two, the first with TIFF files being generated by LR with Adobe Standard, then these TIFF files were stacked in Zerene, the next TIFF files were generated by LR from the same RAW files but Camera Standard was selected, then stacked in Zerene. Both stacked file sets from Zerene were saved in 16 TIFF format and then converted in PS to JPEG for uploading here.
The D800E In-Camera TIFF files are 110MB each and 342 files, the final Zerene created stacked 16 bit TIFF image is 216.9MB.
The D800E RAW images are 74.5MB each and 342 files. With LR 16 bit TIFF generation they grow to 216.9MB each, and the Zerene stacked 16 bit TIFF image is 216.9MB.
I normally don't use DropBox and don't think it would allow uploading this amount of data to Drop Box.
If you have a Nikon (or other) that creates a TIFF in camera, then this experiment is easy to repeat. Even if you don't, take any RAW image set and use LR to generate the TIFF files with Adobe and Camera Standard and stack the results.
Would be interesting to see what others find.
BTW I would do this except I don't have any recent file sets in RAW, since I've been using in camera TIFF for years. I might try a short session just to see, but with Irma bearing down on us here in the Tampa Bay area, it might not happen soon!!
Best,
Mike
The RAW files were all generated into TIFF files by LR for stacking in Zerene. So I did two, the first with TIFF files being generated by LR with Adobe Standard, then these TIFF files were stacked in Zerene, the next TIFF files were generated by LR from the same RAW files but Camera Standard was selected, then stacked in Zerene. Both stacked file sets from Zerene were saved in 16 TIFF format and then converted in PS to JPEG for uploading here.
The D800E In-Camera TIFF files are 110MB each and 342 files, the final Zerene created stacked 16 bit TIFF image is 216.9MB.
The D800E RAW images are 74.5MB each and 342 files. With LR 16 bit TIFF generation they grow to 216.9MB each, and the Zerene stacked 16 bit TIFF image is 216.9MB.
I normally don't use DropBox and don't think it would allow uploading this amount of data to Drop Box.
If you have a Nikon (or other) that creates a TIFF in camera, then this experiment is easy to repeat. Even if you don't, take any RAW image set and use LR to generate the TIFF files with Adobe and Camera Standard and stack the results.
Would be interesting to see what others find.
BTW I would do this except I don't have any recent file sets in RAW, since I've been using in camera TIFF for years. I might try a short session just to see, but with Irma bearing down on us here in the Tampa Bay area, it might not happen soon!!
Best,
Mike
- rjlittlefield
- Site Admin
- Posts: 23625
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
Sorry, I did not mean to imply uploading the entire stacks.
Instead I was thinking of a single representative TIFF and a corresponding NEF. Ideally these would be shot back-to-back with nothing changed except the image format. If they could be shot at the very same time, with both TIFF and NEF captured from the same exposure, so much the better.
The problem is, when I shoot RAW+TIFF and run the raw through my Photoshop and my Lightroom, with their respective settings, all the TIFFs end up being quite similar.
Here's a panel that I shot just now, D800E, in-camera TIFF at upper left, Photoshop's interpretation of the NEF at upper right, and Lightroom's interpretation of the raw at lower left.
Quite similar, though not identical. (As I said in the other thread, I'm never surprised when the colors come out different, only when they come out the same!)
The obvious conclusion is that there's something different between your experimental setup and mine. Of course there are lots of differences, so the key of the investigation is to figure out what the causal differences are.
It's a bit like running a behavioral experiment with lab mice. One lab's batch docilely solves the maze while another lab's batch simply runs away screaming in terror. Why is that? Is it the mice, the maze, the lights, the music, or the experimenters' different body odors?
--Rik
Edit: restore missing phrase, "TIFF and NEF captured from the same exposure".
Instead I was thinking of a single representative TIFF and a corresponding NEF. Ideally these would be shot back-to-back with nothing changed except the image format. If they could be shot at the very same time, with both TIFF and NEF captured from the same exposure, so much the better.
Um, yeah, for some definition of "repeat".If you have a Nikon (or other) that creates a TIFF in camera, then this experiment is easy to repeat.
The problem is, when I shoot RAW+TIFF and run the raw through my Photoshop and my Lightroom, with their respective settings, all the TIFFs end up being quite similar.
Here's a panel that I shot just now, D800E, in-camera TIFF at upper left, Photoshop's interpretation of the NEF at upper right, and Lightroom's interpretation of the raw at lower left.
Quite similar, though not identical. (As I said in the other thread, I'm never surprised when the colors come out different, only when they come out the same!)
The obvious conclusion is that there's something different between your experimental setup and mine. Of course there are lots of differences, so the key of the investigation is to figure out what the causal differences are.
It's a bit like running a behavioral experiment with lab mice. One lab's batch docilely solves the maze while another lab's batch simply runs away screaming in terror. Why is that? Is it the mice, the maze, the lights, the music, or the experimenters' different body odors?
--Rik
Edit: restore missing phrase, "TIFF and NEF captured from the same exposure".
Last edited by rjlittlefield on Tue Jul 17, 2018 8:33 am, edited 1 time in total.
Rik,
I'm not sure why my images are so different either, although pleased that the in camera created TIFF seemed OK (since I had been using them every since is "discovered" this new to me feature a few years ago). Whatever behind the scenes Nikon is using with the D800E seem to be good, at least for these chip images.
One nice thing about RAW is it's not "supposed" to be doctored "much", of course how much is much!! Like the old hair color product ad, only Nikon knows for sure!!
Robert pointed to this site, which has some interesting investigation going on. I only glanced, need to spend more time, but seems like a good reference. I like the part of "reverse engineering" to find out what happens to the files during various conversions. I guess some companies view these conversions as their Secret Sauce and don't document it for the public.
https://www.rawdigger.com/howtouse/deri ... mpensation
Did you try the different TIFF generation modes Robert mentioned? The Adobe and Camera Standard modes?
Back to hurricane prep!!
Best,
Mike
I'm not sure why my images are so different either, although pleased that the in camera created TIFF seemed OK (since I had been using them every since is "discovered" this new to me feature a few years ago). Whatever behind the scenes Nikon is using with the D800E seem to be good, at least for these chip images.
One nice thing about RAW is it's not "supposed" to be doctored "much", of course how much is much!! Like the old hair color product ad, only Nikon knows for sure!!
Robert pointed to this site, which has some interesting investigation going on. I only glanced, need to spend more time, but seems like a good reference. I like the part of "reverse engineering" to find out what happens to the files during various conversions. I guess some companies view these conversions as their Secret Sauce and don't document it for the public.
https://www.rawdigger.com/howtouse/deri ... mpensation
Did you try the different TIFF generation modes Robert mentioned? The Adobe and Camera Standard modes?
Back to hurricane prep!!
Best,
Mike
- rjlittlefield
- Site Admin
- Posts: 23625
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
- rjlittlefield
- Site Admin
- Posts: 23625
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
That's a pretty strong statement, considering that lots of people do use LR and PS to convert their NEF files and are happy with the results. My limited experience is as shown above -- minor differences are always present, but nothing major.Artiii wrote:LR and PS can not convert NEF files properly
What sorts of problems do you see?
--Rik
Color differences, and a little softer. Capture NX really best solution for this, but this was 5-6 years ago, softwares always get better, I never try again PS or LR for this.rjlittlefield wrote:That's a pretty strong statement, considering that lots of people do use LR and PS to convert their NEF files.Artiii wrote:LR and PS can not convert NEF files properly
What sorts of problems do you see?
--Rik
-
- Posts: 2627
- Joined: Thu Jan 17, 2013 9:34 pm
- Location: United States
- Contact:
Been so busy lately that I just caught this thread!Artiii wrote:Color differences, and a little softer. Capture NX really best solution for this, but this was 5-6 years ago, softwares always get better, I never try again PS or LR for this.rjlittlefield wrote:That's a pretty strong statement, considering that lots of people do use LR and PS to convert their NEF files.Artiii wrote:LR and PS can not convert NEF files properly
What sorts of problems do you see?
--Rik
Artii is right in that Adobe does not render Nikon DSLR colors very accurately compared to what you get out of the camera. The Standard camera profile is supposed to fix this, and its better, but still they fail with come colors. Nikon's own imaging apps do a better job with colors but the hardware was very buggy and slow last time I used it.
One fix to getting really accurate Nikon colors with Adobe RAW convertors is to load a custom color profile using a Passport:
http://xritephoto.com/colorchecker-passport-photo
I do own a CP and they are not to hard to set up and use.
Robert
-
- Posts: 2627
- Joined: Thu Jan 17, 2013 9:34 pm
- Location: United States
- Contact:
In process of doing a lens test of a new to me lens (thanks Ray) with D800E using in camera generated TIFF files.
I noted a few MB of difference in file size, from 112.6 to 115.0MB. If these were just pure representations of the actual pixel level AD converter results, all the files should be the same size (assuming no in camera compression) I would think.
Evidence of some file manipulations maybe?
Comments appreciated.
Best,
Mike
I noted a few MB of difference in file size, from 112.6 to 115.0MB. If these were just pure representations of the actual pixel level AD converter results, all the files should be the same size (assuming no in camera compression) I would think.
Evidence of some file manipulations maybe?
Comments appreciated.
Best,
Mike
- rjlittlefield
- Site Admin
- Posts: 23625
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
I find it troubling to see that sequence of words used to describe a TIFF file. They suggest no appreciation for how much processing is done to generate any TIFF file. At the very least, the sensor data has been demosaiced, gamma adjusted, and had its color profile translated from the sensor's native profile to some standard profile such as sRGB or Adobe RGB. In most cases, the image has also been sharpened and possibly noise-reduced. A typical TIFF image bears about as much resemblance to the original sensor data as a half-toned paper print does to an original piece of film.mawyatt wrote:If these were just pure representations of the actual pixel level AD converter results [emphasis added]
None of this bears on the issue of file size, however. That must be a matter of compression. As far as I know, the main image in Nikon TIFF files is not compressed, so that part should be exactly the same from one file to another. But the TIFF file includes other parts whose lengths can vary. Given that much variation in file size (from 112.6 to 115.0MB), I'm thinking there may be something like a JPEG thumbnail embedded in the TIFF, with a compression ratio that varies depending on image complexity.
If you can DropBox a couple of files, I'd be interested to take a quick look.
--Rik
Rik,rjlittlefield wrote:I find it troubling to see that sequence of words used to describe a TIFF file. They suggest no appreciation for how much processing is done to generate any TIFF file. At the very least, the sensor data has been demosaiced, gamma adjusted, and had its color profile translated from the sensor's native profile to some standard profile such as sRGB or Adobe RGB. In most cases, the image has also been sharpened and possibly noise-reduced. A typical TIFF image bears about as much resemblance to the original sensor data as a half-toned paper print does to an original piece of film.mawyatt wrote:If these were just pure representations of the actual pixel level AD converter results [emphasis added]
None of this bears on the issue of file size, however. That must be a matter of compression. As far as I know, the main image in Nikon TIFF files is not compressed, so that part should be exactly the same from one file to another. But the TIFF file includes other parts whose lengths can vary. Given that much variation in file size (from 112.6 to 115.0MB), I'm thinking there may be something like a JPEG thumbnail embedded in the TIFF, with a compression ratio that varies depending on image complexity.
If you can DropBox a couple of files, I'd be interested to take a quick look.
--Rik
"I find it troubling to see that sequence of words used to describe a TIFF file. They suggest no appreciation for how much processing is done to generate any TIFF file"
Wrong suggestion, I have the utmost appreciation for the signal processing taking place. Point I was trying to make with this reference to the pixel ADC is your comment below.
"None of this bears on the issue of file size, however. That must be a matter of compression. As far as I know, the main image in Nikon TIFF files is not compressed, so that part should be exactly the same from one file to another."
I have seen small variations before in TIFF file size before but this was larger and caught my attention. My thinking was the file size was dictated by the pixel ADC levels, color/exposure info and some overhead, and thus should be somewhat constant for a given camera body.
"At the very least, the sensor data has been demosaiced, gamma adjusted, and had its color profile translated from the sensor's native profile to some standard profile such as sRGB or Adobe RGB. In most cases, the image has also been sharpened and possibly noise-reduced."
How would any of these actions effect file size if the TIFF was created without any compression?
"I'm thinking there may be something like a JPEG thumbnail embedded in the TIFF, with a compression ratio that varies depending on image"
This makes the most sense to me, it explains the small varying file size apparently varying with image details. However, I was unaware that a small JPEG or other image specific file was embedded within the TIFF format.
You should be able to run a simple experiment to see if this follows with your D800. I'll see if I can upload to Drop Box if I'm still active.
The files in question were of a silicon wafer at 1X, thus lots of very fine detail across the entire image field. I was stepping at 50 microns to get the everything in focus somewhere after stacking. So maybe the in and out of focus areas cause an apparent detail variation within an individual image? But this suggests that something "knows" these details within TIFF, thus affecting the file size by way of compression or embedded image!
Best,
Mike
Ok this is getting even more interesting
I went back to the first wafer image collection in this series but with the files collected in RAW (I forgot to change the in camera settings to TIFF )
The RAW files were specified as uncompressed in the D800E, and vary from 77.4MB to 79.5MB. The TIFF files created in LR from these RAW files are all the same size, 216.9MB!! The final stacked rendering from Zerene is also 216.9MB!!
Best,
Mike
I went back to the first wafer image collection in this series but with the files collected in RAW (I forgot to change the in camera settings to TIFF )
The RAW files were specified as uncompressed in the D800E, and vary from 77.4MB to 79.5MB. The TIFF files created in LR from these RAW files are all the same size, 216.9MB!! The final stacked rendering from Zerene is also 216.9MB!!
Best,
Mike