TIFF format

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: ChrisR, Chris S., Pau, rjlittlefield

mawyatt
Posts: 2467
Joined: Thu Aug 22, 2013 6:54 pm
Location: Clearwater

TIFF format

Post by mawyatt »

This is derivative thread from a prior (shown below) stacking thread that was diverging, so redirecting here.

https://www.photomacrography.net/forum/ ... 6&start=60


It seems that all TIFFs are not treated alike, and LR and PS can manipulate the generation of TIFF files from camera RAW in various ways as Robert O'Toole kindly pointed out.

The Nikon D800E (I assume the D800 and D500 and upcoming D850 also) produces 8 bit TIFFs, not 16 bit. Obviously these are smaller, about 1/2 the size of 16 bit TIFF files that LR and PS produce.

However the TIFFs that Nikon produces in the D800E have shown IQ comparable, maybe even slightly better, to those 16 bit versions generated by LR or PS converted from camera RAW files for the special chip subjects I tend to shoot. Robert pointed out that LR and PS generated TIFFS have lots of somewhat hidden variables, including a "Camara Standard" non-default value (the LR default is Adobe Standard, not Camera Standard).

I compared some TIFF files generated in the D800E camera (8 bit), the basic default LR Adobe Standard 16 bit and Camera Standard 16 bit TIFF files generated from D800E RAW files with LR default parameter settings (all 0) of the same subject with same lighting and camera settings. These files turned out to be different as you can see below.

Folks with TIFF experience and knowhow are encouraged to chime in and help. I had thought this was somewhat of a constant, but in Wayne's World context...apparently NOT!!

Best,

Mike

8 bit TIFF Directly from D800E
Image

Converted by LR from D800E RAW with Adobe Standard 16 bit TIFF
Image

Converted by LR from same D800E RAW file with Camera Standard 16 bit TIFF
Image

rjlittlefield
Site Admin
Posts: 20766
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Mike, thanks for starting the new thread.

It will help if you can make available the raw file and camera-generated TIFF, say via DropBox?

--Rik

mawyatt
Posts: 2467
Joined: Thu Aug 22, 2013 6:54 pm
Location: Clearwater

Post by mawyatt »

Rik,

The RAW files were all generated into TIFF files by LR for stacking in Zerene. So I did two, the first with TIFF files being generated by LR with Adobe Standard, then these TIFF files were stacked in Zerene, the next TIFF files were generated by LR from the same RAW files but Camera Standard was selected, then stacked in Zerene. Both stacked file sets from Zerene were saved in 16 TIFF format and then converted in PS to JPEG for uploading here.

The D800E In-Camera TIFF files are 110MB each and 342 files, the final Zerene created stacked 16 bit TIFF image is 216.9MB.

The D800E RAW images are 74.5MB each and 342 files. With LR 16 bit TIFF generation they grow to 216.9MB each, and the Zerene stacked 16 bit TIFF image is 216.9MB.

I normally don't use DropBox and don't think it would allow uploading this amount of data to Drop Box.

If you have a Nikon (or other) that creates a TIFF in camera, then this experiment is easy to repeat. Even if you don't, take any RAW image set and use LR to generate the TIFF files with Adobe and Camera Standard and stack the results.

Would be interesting to see what others find.

BTW I would do this except I don't have any recent file sets in RAW, since I've been using in camera TIFF for years. I might try a short session just to see, but with Irma bearing down on us here in the Tampa Bay area, it might not happen soon!!

Best,

Mike

rjlittlefield
Site Admin
Posts: 20766
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Sorry, I did not mean to imply uploading the entire stacks.

Instead I was thinking of a single representative TIFF and a corresponding NEF. Ideally these would be shot back-to-back with nothing changed except the image format. If they could be shot at the very same time, with both TIFF and NEF captured from the same exposure, so much the better.
If you have a Nikon (or other) that creates a TIFF in camera, then this experiment is easy to repeat.
Um, yeah, for some definition of "repeat".

The problem is, when I shoot RAW+TIFF and run the raw through my Photoshop and my Lightroom, with their respective settings, all the TIFFs end up being quite similar.

Here's a panel that I shot just now, D800E, in-camera TIFF at upper left, Photoshop's interpretation of the NEF at upper right, and Lightroom's interpretation of the raw at lower left.

Image

Quite similar, though not identical. (As I said in the other thread, I'm never surprised when the colors come out different, only when they come out the same!)

The obvious conclusion is that there's something different between your experimental setup and mine. Of course there are lots of differences, so the key of the investigation is to figure out what the causal differences are.

It's a bit like running a behavioral experiment with lab mice. One lab's batch docilely solves the maze while another lab's batch simply runs away screaming in terror. Why is that? Is it the mice, the maze, the lights, the music, or the experimenters' different body odors?

--Rik

Edit: restore missing phrase, "TIFF and NEF captured from the same exposure".
Last edited by rjlittlefield on Tue Jul 17, 2018 8:33 am, edited 1 time in total.

mawyatt
Posts: 2467
Joined: Thu Aug 22, 2013 6:54 pm
Location: Clearwater

Post by mawyatt »

Rik,

I'm not sure why my images are so different either, although pleased that the in camera created TIFF seemed OK (since I had been using them every since is "discovered" this new to me feature a few years ago). Whatever behind the scenes Nikon is using with the D800E seem to be good, at least for these chip images.

One nice thing about RAW is it's not "supposed" to be doctored "much", of course how much is much!! Like the old hair color product ad, only Nikon knows for sure!!

Robert pointed to this site, which has some interesting investigation going on. I only glanced, need to spend more time, but seems like a good reference. I like the part of "reverse engineering" to find out what happens to the files during various conversions. I guess some companies view these conversions as their Secret Sauce and don't document it for the public.

https://www.rawdigger.com/howtouse/deri ... mpensation


Did you try the different TIFF generation modes Robert mentioned? The Adobe and Camera Standard modes?

Back to hurricane prep!!

Best,

Mike

rjlittlefield
Site Admin
Posts: 20766
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

mawyatt wrote:Did you try the different TIFF generation modes Robert mentioned? The Adobe and Camera Standard modes?
Yes. That makes a little difference in the colors, but nothing dramatic. The examples above were done in Camera Standard mode.

--Rik

Artiii
Posts: 35
Joined: Mon Oct 07, 2013 2:16 am

Post by Artiii »

LR and PS can not convert NEF files properly, I am not sure but I think they can`t read all the NEF parameters. I always use Nikon Capture NX2 for that.

rjlittlefield
Site Admin
Posts: 20766
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Artiii wrote:LR and PS can not convert NEF files properly
That's a pretty strong statement, considering that lots of people do use LR and PS to convert their NEF files and are happy with the results. My limited experience is as shown above -- minor differences are always present, but nothing major.

What sorts of problems do you see?

--Rik

Artiii
Posts: 35
Joined: Mon Oct 07, 2013 2:16 am

Post by Artiii »

rjlittlefield wrote:
Artiii wrote:LR and PS can not convert NEF files properly
That's a pretty strong statement, considering that lots of people do use LR and PS to convert their NEF files.

What sorts of problems do you see?

--Rik
Color differences, and a little softer. Capture NX really best solution for this, but this was 5-6 years ago, softwares always get better, I never try again PS or LR for this.

RobertOToole
Posts: 1613
Joined: Thu Jan 17, 2013 9:34 pm
Location: United States
Contact:

Post by RobertOToole »

Artiii wrote:
rjlittlefield wrote:
Artiii wrote:LR and PS can not convert NEF files properly
That's a pretty strong statement, considering that lots of people do use LR and PS to convert their NEF files.

What sorts of problems do you see?

--Rik
Color differences, and a little softer. Capture NX really best solution for this, but this was 5-6 years ago, softwares always get better, I never try again PS or LR for this.
Been so busy lately that I just caught this thread!

Artii is right in that Adobe does not render Nikon DSLR colors very accurately compared to what you get out of the camera. The Standard camera profile is supposed to fix this, and its better, but still they fail with come colors. Nikon's own imaging apps do a better job with colors but the hardware was very buggy and slow last time I used it.

One fix to getting really accurate Nikon colors with Adobe RAW convertors is to load a custom color profile using a Passport:

Image

http://xritephoto.com/colorchecker-passport-photo

I do own a CP and they are not to hard to set up and use.

Robert

RobertOToole
Posts: 1613
Joined: Thu Jan 17, 2013 9:34 pm
Location: United States
Contact:

Post by RobertOToole »

Mike,

Zero'd out settings really does make a big difference in ACR. You should look into the info in the link when you get a chance.


Robert

mawyatt
Posts: 2467
Joined: Thu Aug 22, 2013 6:54 pm
Location: Clearwater

Post by mawyatt »

In process of doing a lens test of a new to me lens (thanks Ray) with D800E using in camera generated TIFF files.

I noted a few MB of difference in file size, from 112.6 to 115.0MB. If these were just pure representations of the actual pixel level AD converter results, all the files should be the same size (assuming no in camera compression) I would think.

Evidence of some file manipulations maybe?

Comments appreciated.

Best,

Mike

rjlittlefield
Site Admin
Posts: 20766
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

mawyatt wrote:If these were just pure representations of the actual pixel level AD converter results [emphasis added]
I find it troubling to see that sequence of words used to describe a TIFF file. They suggest no appreciation for how much processing is done to generate any TIFF file. At the very least, the sensor data has been demosaiced, gamma adjusted, and had its color profile translated from the sensor's native profile to some standard profile such as sRGB or Adobe RGB. In most cases, the image has also been sharpened and possibly noise-reduced. A typical TIFF image bears about as much resemblance to the original sensor data as a half-toned paper print does to an original piece of film.

None of this bears on the issue of file size, however. That must be a matter of compression. As far as I know, the main image in Nikon TIFF files is not compressed, so that part should be exactly the same from one file to another. But the TIFF file includes other parts whose lengths can vary. Given that much variation in file size (from 112.6 to 115.0MB), I'm thinking there may be something like a JPEG thumbnail embedded in the TIFF, with a compression ratio that varies depending on image complexity.

If you can DropBox a couple of files, I'd be interested to take a quick look.

--Rik

mawyatt
Posts: 2467
Joined: Thu Aug 22, 2013 6:54 pm
Location: Clearwater

Post by mawyatt »

rjlittlefield wrote:
mawyatt wrote:If these were just pure representations of the actual pixel level AD converter results [emphasis added]
I find it troubling to see that sequence of words used to describe a TIFF file. They suggest no appreciation for how much processing is done to generate any TIFF file. At the very least, the sensor data has been demosaiced, gamma adjusted, and had its color profile translated from the sensor's native profile to some standard profile such as sRGB or Adobe RGB. In most cases, the image has also been sharpened and possibly noise-reduced. A typical TIFF image bears about as much resemblance to the original sensor data as a half-toned paper print does to an original piece of film.

None of this bears on the issue of file size, however. That must be a matter of compression. As far as I know, the main image in Nikon TIFF files is not compressed, so that part should be exactly the same from one file to another. But the TIFF file includes other parts whose lengths can vary. Given that much variation in file size (from 112.6 to 115.0MB), I'm thinking there may be something like a JPEG thumbnail embedded in the TIFF, with a compression ratio that varies depending on image complexity.

If you can DropBox a couple of files, I'd be interested to take a quick look.

--Rik
Rik,

"I find it troubling to see that sequence of words used to describe a TIFF file. They suggest no appreciation for how much processing is done to generate any TIFF file"

Wrong suggestion, I have the utmost appreciation for the signal processing taking place. Point I was trying to make with this reference to the pixel ADC is your comment below.

"None of this bears on the issue of file size, however. That must be a matter of compression. As far as I know, the main image in Nikon TIFF files is not compressed, so that part should be exactly the same from one file to another."

I have seen small variations before in TIFF file size before but this was larger and caught my attention. My thinking was the file size was dictated by the pixel ADC levels, color/exposure info and some overhead, and thus should be somewhat constant for a given camera body.

"At the very least, the sensor data has been demosaiced, gamma adjusted, and had its color profile translated from the sensor's native profile to some standard profile such as sRGB or Adobe RGB. In most cases, the image has also been sharpened and possibly noise-reduced."

How would any of these actions effect file size if the TIFF was created without any compression?

"I'm thinking there may be something like a JPEG thumbnail embedded in the TIFF, with a compression ratio that varies depending on image"

This makes the most sense to me, it explains the small varying file size apparently varying with image details. However, I was unaware that a small JPEG or other image specific file was embedded within the TIFF format.

You should be able to run a simple experiment to see if this follows with your D800. I'll see if I can upload to Drop Box if I'm still active.

The files in question were of a silicon wafer at 1X, thus lots of very fine detail across the entire image field. I was stepping at 50 microns to get the everything in focus somewhere after stacking. So maybe the in and out of focus areas cause an apparent detail variation within an individual image? But this suggests that something "knows" these details within TIFF, thus affecting the file size by way of compression or embedded image!

Best,

Mike

mawyatt
Posts: 2467
Joined: Thu Aug 22, 2013 6:54 pm
Location: Clearwater

Post by mawyatt »

Ok this is getting even more interesting :roll:

I went back to the first wafer image collection in this series but with the files collected in RAW (I forgot to change the in camera settings to TIFF :shock: )

The RAW files were specified as uncompressed in the D800E, and vary from 77.4MB to 79.5MB. The TIFF files created in LR from these RAW files are all the same size, 216.9MB!! The final stacked rendering from Zerene is also 216.9MB!!

Best,

Mike

Post Reply