First post/Photogrammetry with Stacked images

[CarlW]

I am not sure if this is an appropriate topic, or which section it is best to post this message, but wondering if there are others in this forum that are using their stacked macro images to creating a 3D mesh with Photogrammetry software?

[ChrisR]

Welcome Carl
Yes that's fine. In fact, what is it?! It looks like a sculpture mimicking nature. Or is it the other way round?
Yes some members have done some 3D work as you describe.

(I enabled "BBCode" for your posts (a Profile setting), otherwise images don't show.)

[CarlW]

Good call Chris. It is destined to be a sculpture. Many years ago I found a small chunk of wood from a red oak tree. These patterns can be found underneath the bark. This piece had more complex patterns than average. I used a Pentax K-50 and their 100mm macro 1:1 lense and took hundreds of photos from different angles. 31 stacked photos were added to 3DF Zephyr to end up the 3D point cloud, mesh and texture. After some work editing the mesh, i will mill the pattern on a CNC out of wood or machinable wax. This can then be reproduced by making a rubber mold. I am currently trying to increase my magnification so I can capture higher detail.

[ray_parkhurst]

I am not sure if this is an appropriate topic, or which section it is best to post this message, but wondering if there are others in this forum that are using their stacked macro images to creating a 3D mesh with Photogrammetry software?

Some of us are just using the depth map information from a normal stacking session in order to render perspective views. For my coin work, this is a good method because coins don't have any retro (or even vertical) elements. All elements of the coin surface can be seen from a straight-on view, so no details are lost under others, unlike natural subjects like you're showing here.

It seems a daunting task to shoot and render such a subject so that the retro pockets and details will be well-modeled! With your actual final sculpture, how much retro/inner detail are you planning to recreate? Personally, I'd worry about the rubber mold elements breaking off during release since they will be so finely detailed, but perhaps you have methods of mold and release that will preserve these elements? I would guess a non-retro cast might not look realistic enough.

Edited to add: I say "no details are lost" doing coins with only straight-on view, but indeed the details on the edges of devices have much lower resolution than those on the flat surfaces. The more vertical the edge, the less detail since fewer pixels are available to describe the surface. The software renders these by "stretching" the available pixels, so the sides of the features look striated. Luckily, this is indeed what the sides of the features actually look like due to the nature of the minting process, where the metal must flow down into the die in those areas, and the metal gets physically stretched as it fills the voids, so the look is fairly natural.

[CarlW]

I heard about the depth map method several months back, but never had time to investigate. That is partly why I am posting here. Any suggestions on how to go about that process would be appreciated. Mr. Littlefield said I may need to upgrade my Zerene Stacker software, which is no problem.

The process does take some time, but after having tried multiple expensive scanning options, I have achieved what I wanted with a middle of the road camera, and 2 affordable software programs.

Silicon mold material is capable of capturing very fine detail. I just finished a vacuum chamber to assist with this. The release will not be a problem since the model would first be milled on a 3 axis CNC. This process of cutting would eliminate any undercuts that might be present in the original capture, and make the mold release very easy. There are very soft silicons, that can handle difficult undercut situations. I have this issue in mind when choosing my subject to avoid undercuts.

I am not trying to reproduce this at the original scale. The idea is to capture tiny things in nature that many people may not see or appreciate, and recreate them at a much larger scale. I really enjoy patterns and textures both natural and man made, and have been developing CNC skills to be able to reproduce those successfully.

I have just in the last 2 weeks figured out how to capture what I wanted to capture after over a year and half of looking for solutions my budget would allow. I look forward to hearing more about the depth map technique so I may be able to improve upon my ability to capture textures that are more subtle.

[ray_parkhurst]

I use Helicon for my 3D, but I assume Zerene can do the same or better. With Helicon, you can export 3D models, though I have no idea how to import them into software which would allow 3D printing or milling. The process of generating the model is extremely simple, but for it to be accurate the steps need to be well-controlled or the model will be distorted. This is not an issue with 2D rendering of course, which can tolerate a lot of variation in steps as long as the in-focus pixels are captured, but for 3D you need a regular and constant step size.

You can see some of the 3D/perspective renderings I've done at the image gallery on my web page:

http://www.macrocoins.com/image-gallery.html

Edited to add: the process for outputting the 3D model is:

- shoot the stack
- render the stack using depth map method
- in Helicon, start the 3D module
- output the 3D model

[CarlW]

If possible I would like to use depth map information from Zerene Stacker to create 3D information. It does not have the ability to create a 3D file like Helicon Focus. The normal Photogrammetry process does depth map processing in some form from the photos, but not sure how I would introduce depth map information from stacking software into photogrammetry software, or if it is even possible. I am not sure, but do not believe Helicon Focus can process information from multiple angles so it appears limited in terms of true 360 degree 3D information. Maybe multiple 3D files could be combined in other software, similar to how 3d scanning equipment stitches together information from multiple angles. I think standard photogrammetry software tracks pixels that are consistent from multiple angles to create some of the 3D information. While this seems to work reasonably well, it seems like the accuracy of the information from the stacking process would be far more precise. I would enjoy hearing about other methods that may work. I may end up giving Helicon Focus a try.

[ray_parkhurst]

If possible I would like to use depth map information from Zerene Stacker to create 3D information. It does not have the ability to create a 3D file like Helicon Focus. The normal Photogrammetry process does depth map processing in some form from the photos, but not sure how I would introduce depth map information from stacking software into photogrammetry software, or if it is even possible. I am not sure, but do not believe Helicon Focus can process information from multiple angles so it appears limited in terms of true 360 degree 3D information. Maybe multiple 3D files could be combined in other software, similar to how 3d scanning equipment stitches together information from multiple angles. I think standard photogrammetry software tracks pixels that are consistent from multiple angles to create some of the 3D information. While this seems to work reasonably well, it seems like the accuracy of the information from the stacking process would be far more precise. I would enjoy hearing about other methods that may work. I may end up giving Helicon Focus a try.

Helicon Focus only uses the information from one stacking session to render the 3D image. You'll need to use another program if you need to include images taken from other angles. It might be worth trying a single-stack rendering/model to see if it is acceptable for the edges of the features, and complete lack of retro info.

[CarlW]

I do not know my exact magnification for this third try, but guess around 1.75x.
At least a couple of thousand shots were taken, and stacked with Zerene Stacker. I ended up with 63 final shots to process in 3DF Zephyr photogrammetry software.

[JH]

Hi CarlW
This is interesting and thanks for the PM you sent about using depth maps.

First; I have not tried to stich photos and their corresponding depth maps http://www.gigamacro.com/blog/gigapixel ... ase-study/

Second; here is my limited opinions about the advantages and disadvantages with depth maps.

Depth maps works with non-reflective, non-transparent subjects with no bristles, hairs and no hidden surfaces.

The main advantage with depth map is that you only need one stack. This makes it possible to do 3D models of objects that do not last long enough for more than a few stacks or subjects that is difficult to photograph from different angels.

The main disadvantage as mentioned by ray_parkhurst is that the details of some stepper areas looks stretched. If you look at this 3D model https://skfb.ly/6xQys you can easily see the stretched areas. For your purpose – making a 3D subject – this might not be a huge problem because the effect on the 3D surface is less obvious.

Problems with steps, waves, spikes needs smoothing and retouching.
depth map before retouching: https://skfb.ly/6sWnT
depth map after retouching: https://skfb.ly/6sW96

There is also a question about resolution. I am not sure about the calculations behind the depth map, but I use to think of every picture as a step in a staircase that goes from black to white with no more resolution/steps than the number of pictures in the stack up to the number of grayscales. Even at a quite high magnification of 5x the Z dimension will have quite a lot less information than the X/Y dimension. For the Mitutoyo 5x NA 0.14 objective Edmund optics lists the resolving power to be 0.002 millimeters (2.0 micro meters). A table published at Zerene Stacker list the DOF to be 0.028 mm (more than 10x difference)
https://www.edmundoptics.com/f/Mitutoyo ... ves/12298/
https://zerenesystems.com/cms/stacker/d ... romicrodof


Best regards
Jörgen Hellberg

[CarlW]

If there is anybody that would like me to process multiple views of a macro object with my photogrammetry software let me know. It would be interesting to see a 10x subject in 3D.

[BugEZ]

How many views do you want of a subject? How much angular separation between them?

Keith

[CarlW]

So far I have done one at 43 views and another at 60 views. A lot of photos if you are doing large stacks. Precise step angles are not necessary in my experience. I probably took 20 to 25 views per rotation. The angle could vary from near overhead to as shallow as say 30 degrees from the subject plane. Exactly how many views it takes to create a very dense point cloud and accurate model is likely very dependent on the subject. Obviously the more the better, but I have not experimented with seeing how few images can be used and still get a good result. For my first attempts, I have picked one central feature, and done my best to have it central in the frame for every shot. This appears to help the software assemble the topology of the subject fairly easily. I am also working on a subject that is not a true 360 degree view, so all of the subject is in every shot, as opposed to having to circumnavigate a complete object. Photogrammetry processing does best with subjects that have little or no reflections or highlights. One potential hurdle that makes macro subjects a challenge is that the camera is usually stationary, and the subject is moved to speed up the process. The potential trouble is lighting shifts that can happen as the subject is rotated. Shadows and highlights confuse the software. I do not know all the details about how the software works, but gather it searches for pixel patterns and features it can track. If those features are changing with lighting shifts it can reduce the model accuracy, or in bad cases fail to assemble a decent model. BugEZ, if you would care to discuss some subjects ideas let me know.

[BugEZ]

CarlW,

CarlW wrote

...I have done one at 43 views and another at 60 views...

If each view were a stack, that would be a lot of stacks. A labor of stacking love for sure!

Many of the insects I photograph have shiny bodies. Another problem.

I will ponder this a bit.

Keith