geoVideo is not new...?

GeoIntelligence

GIS and Video: A Growing Courtship

April 12, 2011By: Art Kalinski

---

Years ago one of my favorite TV programs was on the Discovery Channel, a British program called “Connections” hosted by James Burke. The program was a fascinating combination of science and history that showed how seemingly unrelated events in history resulted in the modern inventions and systems of today. One example I remember was how coffee’s popularity in the 1300s resulted in modern digital computers. Seems like a stretch, doesn’t it?  Well, in the 1300s importing coffee and cocoa was very profitable for Dutch merchants. However the occasional loss of a ship and cargo could be a catastrophic financial loss for some merchant families. So sitting in the coffee houses of Amsterdam, merchants came up with the idea of pooling their risks and profits so any loss would be spread among many merchants and thus was born the insurance industry.

As the industry grew and expanded into life and casualty insurance, so did the need for actuaries, calculators, and ultimately computers. Most believe that it was the insurance industry of the '50s and '60s that funded major work in mainframe digital computers. It’s fun to see the same kind of evolution and connections occurring today in seemingly unrelated current technologies. GIS and CAD are merging in the complex but elegant 3D BIM models. (See my August 2008 column “BIM, Son of CAD and GIS” and Eric Gakstatter’s column last month.) I now see that same merging of GIS and video, especially in military applications.

I recently attended a full-motion video (FMV) conference put on by the Institute for Defense & Government Advancement (IDGA). The Washington, D.C., conference was geared mostly toward military users and developers. It was only a few short years ago that most in-theater video was low-quality back and white “gun camera” video not much better than WWII battle footage. Everyone was clamoring for more imagery and better video. Video is now being captured by almost every air platform found in-theater with quality moving toward HDTV and better. The array of capture hardware is growing exponentially from large, wide area persistent surveillance systems such as the pending football-field-size high-altitude dirigible theLong Endurance Multi-Intelligence Vehicles (LEMV) by Northrop Grumman to the extremely small nano Humming Bird by AeroVironment, Inc. With a wingspan of 6.5 inches it weighs just 19 grams, has a motor, video camera, network communications, and a battery which looks like ahumming bird.

A Flood of Video

So now there is a flood of video, and as the old saying goes, ”Be careful what you wish for” because users are literally drowning in video content. The new problem is how to catalog and index the video to gain actionable intelligence from the deluge of content. Many people are working on the issue such as this example from RAM Systems, LLC of Canada, but several solutions were presented at the FMV conference with a GIS-based approach.

A technique that seemed promising was geo-referencing video to base maps. David Kirk, principal engineer for Information Systems Laboratories (ISL), talked about “Automatically Producing Geo-registered Full Motion Video (FMV) in Real Time.” There have been crude single Lat/Long links that tied video clips to base maps but this new method was an attempt to orient and pin the video to several points on the ground and could be a first step toward a more robust cataloging of video to accurate geography.

Another presentation, “Automated Road Sign Reading and Real-Time Vision Systems for Ground Vehicles” Dr. Mitchell Rohde, Chief Operating Officer of Quantum Signal, LLC, had an interesting twist. I thought that this was just going to be a simple sign recognition effort, but the twist was using signs and other physical features as an alternative to GPS, sort of a visual feature odometer. We humans do this instinctively as we walk down a hall or down a street. We judge our speed and distance by the passing of features. Dr. Rohde’s system reads mile post markers, exit signs, etc., and does a comparison to the GIS data.

I had an interesting thought that this technique might be a possible solution to track firefighters in a building by combining GPS, inertial guidance, and this “visual odometer.” The best capability I’ve seen to track people inside buildings was theNaviSeer device, which was good but not perfect. (See my September 2010 column.) I asked Dr. Rohde if he had considered this technique inside buildings. He hadn’t, but one of the attendees indicated that the Naval Research Lab had been testing something similar. Cracking the personnel tracking issue would be a big breakthrough.

Other techniques that were discussed were not directly GIS related but will be familiar to us in the GIS community, that painful word “metadata.”  With more than 24 million minutes of video collected by Predators alone, military video libraries are of limited use because analysts often have no way of knowing exactly what they have, or any way to search for information that they need. The military is learning a solution from a surprising source: NFL broadcasters. They showed military analysts how they can quickly find and show replays and even key plays years before for comparison. This is done by meta-tagging video segments as they are captured that identify the key players, events, etc., for a very rapid data search even years later.

The Vision for the Near Future

Picture a collection system that automatically captures and catalogs key visual events as they occur for automated comparison and correlation. As an example, an analyst may have video clips of IED explosions that correlate to previous activity, such as a vehicle that stops at a particular farm then drives down a road that later has a detonation. Viewing and linking these geographic and temporal events to create “connections” may be the critical predictor of future activity and provide for a safer environment for our troops.