Attempt to recreate the AMU Shooter's View Video

Ever since I saw the AMU video from the shooter's point of view (link) I've wanted to be able to do the same thing. I've had most of the parts for a while and finally put them all together over Christmas break.

See Through The Eyes of a Champion version 1 (hackaday.io)

The idea is to use VR/FPV tech to create "shooting goggles". The camera is in front of the VR headset and the shooter sees the video display the same as FPV drones. The session can be recorded, shared and analyzed. We can all see what a shooter actually sees and does, rather than trying to describe it. FPV technology is pretty incredible, their resolution is quite good and their latency is typically in the low 10's of milliseconds in very small and lightweight packages.

The result has to be considered mostly a failure. The videos are good enough to review and analyze, but shooting using the system does not feel natural. Even with some optimizing (lens focal length, aperture, brightness, contrast etc.) shooting with this setup does not feel natural and the resolution just isn't good enough. I might try again if the cost of 4k cameras and displays come down, but right now they cost too much just to fool around. 

However, if/when I have time there will be a version 2, I want to gut the goggles so there is nothing in front of the shooter's eye except a beamsplitter (link to description). That way the shooter sees the same as normal (a little dimmer) and mount the camera to the side. I don't know when I'll have time to do that, and the mechanical system to hold everything will be more of a challenge so will take some time to work out, but I wanted to share in case it inspires someone else to get it done before I do.

Cool idea!

Rayban (and others im sure) sell glasses frames with cameras integral to the frame.

I did some work to recreate exactly what a shooter sees to show the effect of focal point and aperture.  Not just the principle, which is easy to show by exaggerating the camera setting, but really what the eye sees.

The human eye is 20 to 25mm in diameter (or about 1 inch), so I used a 24mm lens on my Nikon.  I set up a 1911 longslide, with the rear sight 24" from the lens, same as it is from my eye.  I set up a target at 50 feet.

f-stop is the ratio of focal length to aperture diameter.  In bright light, the human pupil is about 1/8" in diameter, so in bright light the eye is an f8 lens.  Indoors, in dim light, the pupil can expand to about 3/8", so that is around a f2.8 lens.  If you use an aperture that is 1/16" you get a f16 lens, etc.  This obviously affects your depth of field.

Focus to get the best depth of field, and balanced blur between the sights and the target, should be set at the hyperfocal distance of the sight, which is at a point 2x the distance from the sight.  In my experience, basing this off the rear sight in a pistol, or the front sight in a rifle.

For the shooter using iron sights on a pistol, the best focus is usually achieved with a +0.75 diopter lens, which will put their relaxed focus at 1.33m.

I have attached a collage matrix showing the degree of blur and the focal balance for all different aperture and focus settings.  Unfortunately, even on my Nikon D700, when it is set up to mimic the human eye, the sensor resolution is only about half that of the human eye.

chiz1180 wrote:Rayban (and others im sure) sell glasses frames with cameras integral to the frame.

I looked at a couple of similar ideas, one with the camera on the outer edge of the lens and another with the camera at the center top of the lens. In neither case did the camera "see" the dot. It was too far out from the line of sight of the eye.

shootingsight wrote:I did some work to recreate exactly what a shooter sees to show the effect of focal point and aperture.

That's great work!

Steve's camera view shooting the dot is a great example of what shot calling is all about for those not understanding that.

One interesting thing, when I compare what I perceive vs what I see in the video, My mind filters out the very high frequency shaking. I see the dot moving around in the middle and the big deviations, but not the very fast quivering of the dot.

I focus on the target when shooting. I raise above the bull, check that the dot is centered in the tube, then shift my eye to the center of the bull and wait for the dot to come down. The settling process is largely automatic. My conscious mental focus is on the target.

As to DA/SA's comment, my shot calling was pretty good except for the 45 "oopsy" shot in the white. I called it 9 ring left and a little high. Looking at the video, It shows the dot moving very quickly up in the last frame or two before the shot. Clearly I did something at the last moment to move the gun a lot. I also need to record at a higher frame rate. This was recorded with my iPhone at 30fps.

shootingsight wrote:
The human eye is 20 to 25mm in diameter (or about 1 inch), so I used a 24mm lens on my Nikon. 

My understanding is a 50mm lens is the analog to the human eye perception with a full size 35mm sensor.  I'd bet it is some ratio of lens to sensor size. I know if I hang a legacy film lens on my 3/4 size sensor camera I get a 1.5x zoom from crop effect, but in terms of compression it is still a 1x (I think).

As to the original post, I think the beam splitter with a small camera would be the way to do it.

You need a GOFUNDME account.

Merick wrote:

shootingsight wrote:
The human eye is 20 to 25mm in diameter (or about 1 inch), so I used a 24mm lens on my Nikon. 

My understanding is a 50mm lens is the analog to the human eye perception with a full size 35mm sensor.  I'd bet it is some ratio of lens to sensor size. I know if I hang a legacy film lens on my 3/4 size sensor camera I get a 1.5x zoom from crop effect, but in terms of compression it is still a 1x (I think).

As to the original post, I think the beam splitter with a small camera would be the way to do it.

Yes you are correct. 35mm film is approximately 50mm diagonally corner to corner. Most of the digital cameras have either 1/3" or 1/2" sensors, measured diagonally, so 8mm and 12-13mm lenses create "normal" vision for them respectively. I can say that walking around (or even just loading magazines) wearing this setup is not too bad with an 8mm lens but is very weird using a longer or shorter lens. I got a few attacks of vertigo doing that. 

The problem with using a "normal" focal length lens is that there is not enough resolution to see the target and sights. It looks very different through the camera vs in real life. That is why I think the best solution is to abandon this path and go to a beam splitter.

james r chapman wrote:You need a GOFUNDME account.

I'm OK as far as the cost of doing this. I need GOFUNDME to give me more time

50mm lens is equivalent to the human eye in 35mm film, because they both have the same field of view.  But 50mm diagonal is much bigger than the retina of the human eye.

When you look at pixel spacing on the human retina, and compare that to pixel spacing on a CCD sensor, the human eye has about 2x the resolution of most camera sensors.  So to match resolution, I could have used a 50mm lens for my pictures, but with the magnification, the representation of depth of field might have changed.

Does Fronek ( or his son) know anything about the original equipment? I've heard it's gone now

SteveT wrote:

chiz1180 wrote:Rayban (and others im sure) sell glasses frames with cameras integral to the frame.

I looked at a couple of similar ideas, one with the camera on the outer edge of the lens and another with the camera at the center top of the lens. In neither case did the camera "see" the dot. It was too far out from the line of sight of the eye.

I was thinking along the lines of canibilizing the frames to get the camera and integrate it into an occluder or similar. Though at the end of the day something like this shows what it looks like, not how to process what you are seeing.

What are you trying to achieve?  If it is just to see how the aim settles in, you could come up with a camera that allows you to put various styles of digital crosshair on the image, add a telephoto lens, and attach it under the barrel.  You could easily position the crosshair to locate at the same point as the sights, so while you cannot see exactly what the shooter sees in terms of sight alignment, you could see where he is aiming.

Actually seeing his sights by co-witnessing his vision is more difficult because the eyes move relative to the head, so if you attach the camera to his head, it will not always align with his eye.  Rather than use a beam splitter, take a mirror and drill a hole in it for him to look through.  This would get him to consistently look through the same spot on the mirror, and would prevent the shooter from losing light.

shootingsight wrote:What are you trying to achieve?

Well first of all, it's cool and as far as I know no one has recreated the system the AMU used so we only have that one video floating around. I'd like to see more videos like that.

I've had some shooters ask questions about what they are looking for when they shoot and especially when they dry fire. After some discussion I realized they weren't sure what was the difference between normal area wobble and trigger induced movement. It would have been helpful to pull up a few videos and say "this is what it should look like and that shows a trigger problem". (yes Ed, I know we should focus on the right way to do it, not the errors)

I think it might be enlightening for marksman / sharpshooters to see how much movement masters and high masters have. At a Zins clinic many years ago someone had a laser on a gun and we all took turns holding on target. Brian Zins did not have the smallest hold area. Half the class had holds that spent at least a few seconds inside the 10 ring. The difference between Brian Zins and the rest of us isn't that he is a machine vice. It's that he has learned what to look for and how to pull the trigger.

It might be helpful to record beginners and lower level shooters to see what they see. I know when I was coming up, I wasn't able to observe myself shooting and be able to describe it to a better shooter. I've also had discussions with shooters who describe a certain hold or wobble pattern and asked my advice about what to do about. Words are useful, but they aren't as good as seeing the pattern itself.

I think shooting with this might be fun, almost like a video game. Maybe some of the run-and-gun people in the local club will come over to the bullseye range just because of the novelty of it.

Lastly, and this is probably not something I'll get to anytime soon but it's in the back of my mind, it might be interesting to make recordings dry firing and live firing over time and do some analytics and machine learning. Do it using different grips or trigger configurations. Maybe it's possible to see patterns I don't notice in real time.

There are undoubtedly other ways to accomplish similar things. A Scatt or Mantis is probably more valuable for most shooters. I've put lasers on pistols and recorded it with a camcorder. A small camera on the gun can show hold patterns. That's basically what a Scatt is. I've thought about putting accelerometers on the gun to recreate hold patterns like the Mantis system does.

First person suggests the camera looks where your eye looks.  What happens if you say 'screw that', the eye will look where the monitor is.

I'm standing at the line, but facing left.  there is a camera next to my head on the right side, pointed to the right, ie at the target.  THere is a monitor in front of me, so I am facing across the line, but am aiming the pistol to my right, at 90 degrees to the way I am looking.  Now I look at the monitor to get my sight and target alignment.  I am seeing what the camera sees, and I line up my sights and my target based on the video monitor, not because I am looking at the target.

Maybe this weekend I'll put it together for dry firing.

I am sure that will work. I want to record myself and other shooters with as little change as possible, using the same unmodified gun,  natural stance and natural process. Putting a set of goggles on with no wires seems to be the least intrusive method.

I see a challenge with goggles in that you want to control your focal distance.  The integrated cameras do auto-everything.  If you use a mirrorless Nikon, you can select the lens you want for the proper magnification, and set the focus to manual.  You might get away with using a loupe to look at the viewfinder screen, but viewfinder resolution is lower than sensor - that is why I thought of using a monitor.

In thinking about it, the change is not huge.  You already aim off to your right, so position would not change, you would just have your head face forward, toward the monitor, instead of looking to the side to line up with the sights.

In the end, I agree that an effective video needs to be truly coaxial with your vision, so you can see sight alignment.  Most First Person cameras I see place the camera at your temple.  Probably OK for most applications, but won't show correct sight alignment.

A key question is if you need to have true optics coaxial with the recorder, by using a beam splitter type arrangement, or whether you can aim based on the electronic image, in which case getting coaxial is easy, you just need to figure out how to present the electronic image to the shooter - external input to VR goggles, or an off-axis monitor.

I recall that Ed Hall had some knowledge of how this was/is done. Ed? Care to add your wisdom?

They used a commercial bought system with an HUD visor that had a camera mounted on the front which fed the visor and had an external video connection so the instructor could monitor/record what the shooter saw.  I've never been able to find the system online, but it was called something like FTVS.  The USAF Team used to have a videotape of the Air Force Team members using the system along with Steve Reiter and a couple others.  Maybe Froneck can ask Adam what the system was called and other particulars.

Might already be known, but there is a company called aim cam that is somewhat similar. Haven’t done any research on them, but I remember seeing them somewhere. at SHOT show years back if I recall. 
www.aimcam.com
They offer up to 3K resolution. 
https://www.aimcam.com/products/aimcam-pro3k
Might not be precisely what you’re looking to achieve SteveT. But perhaps might give you some more ideas?  Or be able to build off of an established platform through some type of an agreement. 
-Rob

That there Aimcam looks like it would get the job done! 
Who owns one? Waiting for a product review!

From what I see in the aimcam video and images, this is not the same.  It looks like the camera is mounted above the line of sight instead of providing the line of sight.  No matter how close you get, if you aren't at the line of sight you're not seeing what the shooter sees.  The point of the AMU video is to see the actual sight alignment and dynamic sight picture.