Motion Control and Focus Stacking

[JohnB]

This is my first post. I am the latest member at the moment :)

I have been experimenting with camera motion control for a while now and have recently started to introduce focus stacking within this process to enable enhanced depth of field. What comes out of his process is HD video.

I essentially take multiple frames (each of which is stacked), indexing the camera in space after each frame. I do this many times over a keyframed trajectory derived by math. When the stacks are processed and then the frames are rendered say at 24fps you get a video with movement and enhanced DOF. This process works but what I now need to do is hone my skills in macro work to improve quality.

I have just found this site and I believe that I have come to the right place for advice. I am looking forward to interacting.

Anyway I just thought I would make my first post.

Here is a link to the sort of thing I am interested in

https://vimeo.com/78995282

It's just a short sample.

Excuse the quality .... as I say .... I have a lot to learn. Any comments good or bad are welcome. You only learn by listening :D

[TheLostVertex]

It looks very nice. What is your method and set up for controlling motion here?

I noticed that it appears like when the object gets closer, it starts to get out of focus slightly. Is this intentional? Are you resetting the camera to the same focus position for each frame while changing the distance to the object?

A minor aside, I found the cut to black at the end of each loop to be distracting when trying to view the cycle.

[rjlittlefield]

John, welcome aboard!

This looks very promising.

For me the cut to black was helpful, not distracting, because in this prototype video it clearly marked the break between two sections that otherwise would not have fit together smoothly anyway.

However I found the motion to be too quick to see much in the individual frames.

What sort of applications are you thinking of, long term?

--Rik

[JohnB]

It looks very nice. What is your method and set up for controlling motion here?

I noticed that it appears like when the object gets closer, it starts to get out of focus slightly. Is this intentional? Are you resetting the camera to the same focus position for each frame while changing the distance to the object?

A minor aside, I found the cut to black at the end of each loop to be distracting when trying to view the cycle.

Hi, I use keyframing. In this test I was running pan/tilt/slide. The set up made the camera move towards and slightly past the object holding position on it. The controller is and software is my own design. It drives steppers via pulse count. Focus stacking is via a script within the camera. I trigger the script after the moco system indexes to index the lens motors.

The slight out of focus is my bad set-up :) I am only triggering 5 layers at the moment. I will post some alternative footage to show the movement dynamic later to better explain.

John, welcome aboard!

This looks very promising.

For me the cut to black was helpful, not distracting, because in this prototype video it clearly marked the break between two sections that otherwise would not have fit together smoothly anyway.

However I found the motion to be too quick to see much in the individual frames.

What sort of applications are you thinking of, long term?

--Rik

Thanks, The forum looks like a gold mine :) I have some other slower footage. I will post later and explain more :)

[JohnB]

Thanks once again for the replies. I'm still getting used to the forum protocol so please bear with me.

As mentioned in my first post I am keen to refine the capture of motion controlled macro HD video (multiple single frame capture aka stop motion/timelapse). The focus stacking of individual frames I think should help enhance impact.

I am able to deal with the motion control side of things but focus stacking is new to me and I am experiencing some artefacts in my rendered frames. Early days .... but any help/input re understanding what is going on would be much appreciated. I am also happy to discuss my technique.

Anyway to follow on from my last post here is a bit more test footage. It shows motion control linked with "within camera" focus stacking.

https://vimeo.com/79324502

Note: not sure how to embed video

Not bad ... but ... as you will see ..... I have artefact issues at some of the boundaries ... (see purple/yellow boundary flickering) . It seems to be some sort of shadowing effect. My question .... Is this common and why does it occur?

I am using Helicon on basic "do stack" settings over 5 layers and processing in batch mode. As I say any help/comments would be appreciated

Anyway ... to also provide some background ... this is the kit I used to derive the footage:


© 2013 KeyframeMoCo MILC by SiriusJohn, on Flickr

I have other similar pan/tilt arrangements to carry DSLR's with lens mounted stepper based focus control. This is the portable version.

The system also works in video mode (realtime movement). The shot below is it working in this mode (so not relevant here) but I include it as the picture shows the controller I use:


© 2013 KeyframeMoCo - Video Tests by SiriusJohn, on Flickr

Hope this is of interest. Any help would be much appreciated.

On the application front ... I am not sure yet. I am an engineer and I just like making things work. I am also very interested in making a short film about plants and their structure. There is something quite wonderful about the symmetry. When you combine it with movement it is quite mesmerising.

Happy to discuss as I say. Any help re the artefacts would be much appreciated. Off to read some forum posts now

John

[TheLostVertex]

Thank you for the slower video, it helped me a lot in viewing the footage in more detail. This looks quite nice.

Not bad ... but ... as you will see ..... I have artefact issues at some of the boundaries ... (see purple/yellow boundary flickering) . It seems to be some sort of shadowing effect. My question .... Is this common and why does it occur?

I am not very familiar with helicon focus, but it would look like you are using an incorrect radius/smoothing value for these images. In turn, it is selecting incorrect regions from other images in the stack, causing out of focus spots.

On the application front ... I am not sure yet. I am an engineer and I just like making things work. I am also very interested in making a short film about plants and their structure. There is something quite wonderful about the symmetry. When you combine it with movement it is quite mesmerising.

That sounds like a great use. I also love the structure of plants, always fascinating to me. If you havent seen the title 'Kingdom of Plants 3D' I would recommend it. It has some great timelapse and videography in it. And the making of portion is also quite entertaining.


I am guessing you are scripting the the focus stepping with CHDK? Also, how are you currently programming keyframes in for the motion?

[rjlittlefield]

I have artefact issues at some of the boundaries ... (see purple/yellow boundary flickering) . It seems to be some sort of shadowing effect. My question .... Is this common and why does it occur?

Yes, artifacts like this very commonly occur. But they are not so commonly seen because they are very difficult to spot in single frames. I usually spot them when I flash back and forth between results of processing the same stack with different methods or parameter settings.

The artifacts happen because in regions of the scene that do not contain definite texture, stacking software may make decisions based on essentially random noise. The randomness is different from stack to stack, so the decisions made in one stack may be different from those made in another. When the random decisions result in selecting pixels that are little brighter in one frame, darker in the next, then lighter again in the one after that, you get flickering.

This problem may be difficult to correct. I am not aware of any software that specifically tries to maintain consistency between multiple stacks.

In Helicon Focus, you might try using Method C rather than the A or B that I suspect you're using right now. I think that Method C is a little less inclined to make changes to large areas based on noise, but this is not something that I've ever specifically checked.

You might also try using Zerene Stacker instead of Helicon Focus, and in particular Zerene's DMap method with Options > Preferences > DMap Settings selected to "Use preset contrast threshold" with a specified Level that is set fairly high so that the software is limited to making image-based decisions only where it can do that reliably. I wrote Zerene Stacker and I'll be very happy to help guide you through this process, but I'll be the first to confess that I don't know how well it will work for this application. There will inevitably still be some random decisions where the image content falls just barely above or below threshold, and I don't know how those will affect the results.

--Rik

[JohnB]

Steven/Rik, thanks for the replies …. much appreciated. It made me spend a bit more time on my software setup. I'm getting much more feel for what is going on within the process now. I am also realising what I need to do to improve my workflow.

Anyway here is the latest test >> https://vimeo.com/79388747

and the flickering is gone :D Thanks

Now all I need to deal with is the out of focus bits. This I'm pretty sure is due to my setup though. These tests are the first I have done and I am using a fixed series of layers i.e. I set my script to capture at say 1cm, 2 cm, 3cm, 4cm and 5cm. The camera indexes through these focal distances when triggered by my controller. The problem is that I am moving the camera quite a bit so the layers are shifting. Also the restriction to 5 layers may be resulting in inter-layer out of focus layers (if you know what I mean). These are moving and appearing at times. If you look closely you can see them. What would be much better is to make the camera spot focus when triggered and then place the layers across/through the object accordingly (get_focus and set_focus and a bit of maths). I will probably try this next when I have time.

Anyway ….. I thought I would report….. and say a hearty thanks for the help/input …. this is a cool forum :D

Excuse the combined reply but, as I say, I am not sure re protocol on the forum … to answer questions ….

Steven, Thanks again for the post. After I read you post I went off an watched part of the Kingdom of the Plants video on Youtube …. brilliant ,,, thanks … It may end up being a Christmas present to myself :)

On the script front … yes I am using CHDK (simple Ubasic) and triggering via USB 5V input. I intend to move to CHDKPTP though once I get it compiled on my controller. I have also tried gphoto2 on compacts and messed around with lens indexing on DSLR's (lens mounted steppers) both work. On the keyframing front the controller takes care of it. I simply point the camera on the object (across all axes/control elements) re the key frames. Input is sequential and I define inter keyframe timing.

This is a pretty old "proof of concept" video and its pretty boring but it describes input >> https://vimeo.com/65244330

The way it works means that it can control any moving element that is stepper based e.g.. pan/tilt/roll/xyz linear/focus/zoom etc. You simply tell it where it need to be at certain points in time … it does the rest and works out the frame positions it needs to deliver the rendered move. There are many programme variants (SD cards). The above video shows pan/tilt/zoom.

Here is pan/tilt/focus >> https://vimeo.com/63927639

Hope this explains. Let me know if you want further info. No problem discussing … The only area I get sensitive about is the core key framing code. How I do it is a bit close to my heart (2 years developing) I use an RPi by the way to do the core processing. --John

Rik, Thanks for the reply (I am doing replies in the order of posting :) ).

The above video was done in Method C based on your suggestion so many thanks ….. but I have also now done trials adjusting radius and smoothing. Net result is that I'm getting a feel for how processing works …. so I'm happy. I have now downloaded Zerene and will experiment. It looks really good …. much more control …. I can see that the technique is quite complex. I would very much like to share and discuss if this is OK with you. Help would be very much appreciated.

I may not get much time in the next few days due to other demands/commitments but would it be OK to tick tack when I start to experiment? I would like to understand the process/algorithms a bit more. This might inform direction --John

[rjlittlefield]

Excuse the combined reply but, as I say, I am not sure re protocol on the forum

We don't really have protocols for stuff like formatting and segmenting. What you've done here is fine.

I would very much like to share and discuss if this is OK with you.

Sure, glad to help. If you like, you can contact me offline about Zerene stuff as support@zerenesystems.com .

--Rik

[JohnB]

Not sure if there is interest but here are the latest results.

Set-up: Canon G15. KeyframeMoco driven (pan/tilt/slide - 2 key frames required to define move), CHDK script, 10 layer stack manipulating the internal lens motors triggered by controller via USB. Stack reset for each frame acquisition to allow for rig indexing. Offsetting to spread the stack across subject.

Video link >>> https://vimeo.com/80502729

Gradually getting better and I understand the issues more now. The ultimate objective is full MoCo macro with enhanced depth of field.

I want to move around those seed pods and flower heads :D

JohnB