Macro frame rates high enough to trap photons in flight

[Lou Jost]

https://www.youtube.com/watch?v=7Ys_yKGNFRQ

Probably this has been mentioned here before, since it is quite old. But I am posting it anyway because it is so incredible, it still makes my eyes wet every time I think about it.

More
https://www.youtube.com/watch?v=EtsXgODHMWk&t=166s

[CrispyBee]

I've seen this a while ago and I still wonder whether they recorded photons or "simply" a reaction of the liquid or (in case of the little figurines) surface to a laser impulse.

EDIT: and looking at the reflections from the bottle it does not look like travelling photos but like a light beam or laser beam was moved at a very narrow angle from one end of the bottle to the other, so I would rather think that the light source was slightly rotated during the recording. Note that the emphasis was on the speed of the recording and not recording light or photons as such.

If photons were recorded I'm sure they would not have behaved as such a perfect streak and due to the shape of the bottom of the bottle and in the refraction and all the "milk" molecules in the first video it would have dispersed way more and it wouldn't have formed clouds (that appear to move as agitated water tends to).

That would also explain why they used regular consumer plastic bottles with uneven surfaces, big labels etc and not something much better suited like a rectangular container or super clear glass/acrylic bottles.

[rjlittlefield]

so I would rather think that the light source was slightly rotated during the recording

The light source is stationary.

You're looking at a sampling system. It consists of a laser that emits very short pulses of light, a camera with a very short exposure window, and software to make sense of all the data. What you're seeing in the video shows the location of "the" pulse of light as "it" propagates through the system, except that in reality that view is constructed by sampling zillions of identical pulses with slightly different delays between pulse emission and camera exposure.

There is only a little discussion but several links to other publications at viewtopic.php?f=3&t=17421 (2012) and viewtopic.php?f=3&t=19145 (2013), both found by searching the forum for "trillion". I have vague memories of other discussions, but those may have been offline rather than in the forum.

--Rik

[CrispyBee]

so I would rather think that the light source was slightly rotated during the recording

The light source is stationary.

You're looking at a sampling system. It consists of a laser that emits very short pulses of light, a camera with a very short exposure window, and software to make sense of all the data. What you're seeing in the video shows the location of "the" pulse of light as "it" propagates through the system, except that in reality that view is constructed by sampling zillions of identical pulses with slightly different delays between pulse emission and camera exposure.

There is only a little discussion but several links to other publications at viewtopic.php?f=3&t=17421 (2012) and viewtopic.php?f=3&t=19145 (2013), both found by searching the forum for "trillion". I have vague memories of other discussions, but those may have been offline rather than in the forum.

--Rik

I was just looking at the way the light behaves, it's not projecting a symmetrical refraction on its surroundings - which is what I would expect if it was a "pulse" traveling through the bottle, even if it wasn't perfectly centred it would expect it to look and behave differently.
I still don't know why in both experiments they used these kinds of cheap plastic water bottles with labels on them. That just seems a bit.. strange. I would have expected them to use clean, clear and uniform containers, so the results aren't that 'iffy'.

As much as I want to "believe" we're seeing what they claim it to be, I just don't know why they chose to do some stuff the way they did it. I don't think it's fake but I think there may be more to it than what it looks like.

And just as I was looking for an explanation online - here's what I've found:

https://www.wired.com/2011/12/is-this-r ... er-second/

and

https://web.media.mit.edu/~raskar/trillionfps/

"Direct recording of reflected or scattered light at such a frame rate with sufficient brightness is nearly impossible. We use an indirect 'stroboscopic' method that records millions of repeated measurements by careful scanning in time and viewpoints. Then we rearrange the data to create a 'movie' of a nanosecond long event."

So it's not a slow-motion video of photons. It's an "imaging technique" that at best approximates to something like Rayleigh scattering (though using "milk" as a medium to increase the effect and visibility) and afterwards the individual frames are manually and deliberate sorted so they produce a video that suggests a light impulse in motion.
But since we don't know what the individual frames they used looked like or how exactly the camera functions (what kind of shutter (if any) was involved)... it's very difficult (if not impossible) to say what they really captures and how much they "goal" influenced the result.

[rjlittlefield]

But since we don't know what the individual frames they used looked like or how exactly the camera functions (what kind of shutter (if any) was involved)... it's very difficult (if not impossible) to say what they really captures and how much they "goal" influenced the result.

If you want to understand more about how the technology works, then I suggest to Google search on Andreas Velten femto-photography and drill down into the publications and conference proceedings. A good place to start is with https://en.wikipedia.org/wiki/Femto-photography and its link to http://giga.cps.unizar.es/~diegog/fiche ... /femto.pdf .

If you're especially short on time, visit https://dl.acm.org/doi/10.1145/2461912.2461928 and download the video at https://dl.acm.org/action/downloadSuppl ... =tp107.mp4 . The first four minutes is a good summarization. This video is supplementary material for an article published in one of the ACM publications back in 2013. On review, I think this material is probably the discussions that my earlier post mentioned vaguely remembering.

So it's not a slow-motion video of photons.

That depends on what meaning you impose on the words.

Certainly their system is not tracking individual photons. Quantum mechanics says that concept doesn't even make sense, because photons essentially take all possible paths between where they're emitted and where they're observed.

But if it's OK to say that a properly strobed flash makes slow motion video of an engine running, then I don't have much trouble with hearing that a properly strobed emitter/detector makes slow motion video of a light pulse propagating.

--Rik

[CrispyBee]

So it's not a slow-motion video of photons.

That depends on what meaning you impose on the words.

This isn't about me not understanding the technology (which I do) - it's about partially nonsensical claims that make it seem like something it's not and creating misleading visualisations and descriptions and (also quite important) not mentioning the actual setup and/or changes to the setup.

Again I would point you to this article:
https://www.wired.com/2011/12/is-this-r ... er-second/
and the data sheet here - which is the only actual piece of equipment they mentioned in their paper: https://pdf.datasheetcatalog.com/datash ... 976-04.pdf is also very interesting and revealing when it comes to interpreting the actual output they would have had to work with (which by itself is very impressive considering all their caveats with noise and the high threshold value of these xT datapoints) and this part of the paper:

"In order to visualize all light transport events as they have occurred (not as the camera captured them), we transform the captured data from camera time to world time, a transformation which we term time unwarping."

.

I agree that it's an interesting concept and maybe useful for... something but it does not record a slow motion video at trillion frames per second, it can't trap photons in flight and the visualisation (especially from the first linked video) is very dubios at best and the title is simply put wrong and misleading.

Your analogy with the train engine is problematic as we know how an engine moves as we can observe it in real time and can even slow it down to verify it. And if a video of a moving engine is shot using strobes synced up to the shutter speed - well then it's basically just a demonstration of something we already know - but we're getting a full picture with every strobe and not just a potential high noise low signal slice that we then run through an algorithm to hopefully then cobble together into an animation overlaid with actual images to simulate the movement of a motor.

However in this case it's not the same thing. We're getting a video that suggest something that may or may not actually work the way it's depicted.

[rjlittlefield]

I've let this sit for a couple of days, during which time I've re-read all the material and tried to imagine how things look to other people.

That imagining part is hard, because I've been familiar with the basics of the technology for over 10 years.

As a result, I naturally interpret presentations in terms of what I already know was done, as opposed to hearing the words cold and trying to figure out what they might mean.

But I'm curious.

I agree that it's an interesting concept and maybe useful for... something but it does not record a slow motion video at trillion frames per second, it can't trap photons in flight and the visualisation (especially from the first linked video) is very dubios at best and the title is simply put wrong and misleading.

To risk summarizing your concerns, at this point it sounds to me like (A) you're grousing about the language and (B) you still have some skepticism about whether the data acquisition and computing was done correctly.

So let me ask a question:

On the assumption that the work was done correctly, can you tell us what title and summary you would be happy with?

--Rik

[CrispyBee]

On the assumption that the work was done correctly, can you tell us what title and summary you would be happy with?

--Rik

Maybe something like this:
Using the principles of strobes and extremely fast exposures they were able to capture individual signals, which - after deliberate sorting and interpolating - could be used to create a visualisation of what a moving impulse of light might look like.


Imagine someone told you they are working with a 500 layer sensor to merge visual data of different points in time and with varying spacial refractions of photons to create ultra-high-resolution full-depth macro images with zero noise at ISO6400 and it only takes them 1 minute.
And they're tactually just stacking 500 images at 5MP with multiple aggressive noise-reduction passes - and they only count the time it takes to render the final image...

[rjlittlefield]

Imagine someone told you they are working with a 500 layer sensor to merge visual data of different points in time and with varying spacial refractions of photons to create ultra-high-resolution full-depth macro images with zero noise at ISO6400 and it only takes them 1 minute.
And they're tactually just stacking 500 images at 5MP with multiple aggressive noise-reduction passes - and they only count the time it takes to render the final image...

I agree, that would be annoying.

But I don't see that any of the papers or videos did anything remotely like that.

From my standpoint, they were all perfectly clear about how long the data acquisition and processing took. The YouTube guys even made a special joke about it, noting before showing the very first result that it took 8 hours to process and that one of the guys grew a longer beard and the other had a haircut while waiting for the results.

As for how much to trust the results, I note that the second demo in the YouTube video, the one at 100 billion "fps", is a configuration for which simulation works very well and the experimental observations match the simulated results. This strikes me as being very much like testing an oscilloscope by connecting it to what we have designed to be a fast square wave. If the display shows suitably fast rise and fall times with minimal overshoot, we conclude that the scope is working correctly, and after that we have good confidence in its results when looking at more complex waveforms that we actually want to learn about. To be honest, I have much more faith in their visualization of light propagating through a Coke bottle and its environment, than I do in any human's a priori expectation of what the results would be.

So I am puzzled. It's almost as if you and I were watching and reading completely different materials.

But no mind. Moving forward...

I did a bit more digging.

In the YouTube video, the guys were visiting Dr. Peng Wang ("Postdoctoral scholar") at the Compressed Ultrafast Photography lab at the California Institute of Technology. That provides a lot of good handles for searching.

As we might expect, it turns out that the CUP group has their own web page: https://coilab.caltech.edu/research/compressed-ultrafast-photography-cup .

That page begins with the following text:

Ultrafast imaging technologies are essential to capture transient phenomena in biology, physics, and chemistry. Currently, various pump-probe approaches are the mainstream in ultrafast imaging. However, these schemes require the targeted phenomena to be precisely repetitive and thus are inapplicable in imaging many single-event ultrafast phenomena, such as optical rogue waves, nuclear explosions, and scattering in dynamic biological tissue.

Compressed ultrafast photography (CUP), as a single-shot ultrafast computational imaging modality, overcomes this limitation. Synergistically combining compressed sensing and streak imaging, CUP can image non-repetitive transient events at 100 billion frames per second. In addition, akin to conventional photography, CUP is receive-only. Avoiding specialized active illumination, CUP is perfectly suited to imaging a variety of luminescent processes, such as fluorescence and scattering.

I will happily grant that those words give me the feeling of advertising. But that's a stylistic issue, which I try to keep separate from what I think about their technology.

Regarding the technology, their web page also has a list of recent publications. One of them is this:
Wang, P.; Liang, J.; Wang, L. V.; "Single-shot ultrafast imaging attaining 70 trillion frames per second," Nature Communications 11(1) 2091 (2020)
According to the abstract:

Real-time imaging of countless femtosecond dynamics requires extreme speeds orders of magnitude beyond the limits of electronic sensors. Existing femtosecond imaging modalities either require event repetition or provide single-shot acquisition with no more than 10^13 frames per second (fps) and 3 × 10^2 frames. Here, we report compressed ultrafast spectral photography (CUSP), which attains several new records in single-shot multi-dimensional imaging speeds. In active mode, CUSP achieves both 7 × 10^13 fps and 10^3 frames simultaneously by synergizing spectral encoding, pulse splitting, temporal shearing, and compressed sensing—enabling unprecedented quantitative imaging of rapid nonlinear light-matter interaction. In passive mode, CUSP provides four-dimensional (4D) spectral imaging at 0.5 × 10^12 fps, allowing the first single-shot spectrally resolved fluorescence lifetime imaging microscopy (SR-FLIM). As a real-time multi-dimensional imaging technology with the highest speeds and most frames, CUSP is envisioned to play instrumental roles in numerous pivotal scientific studies without the need for event repetition.

The full paper is downloadable as PDF, with videos available as supplemental materials on the Nature Communications website.

Yes, the method has certain limitations, some of which do not correspond to our naive intuitions about how ultrafast photography might work.

I have no idea whether you would find this work interesting or worthy, but other people might so I thought I should mention it.

--Rik