Billion Dollar Clutter?

White House ‘Big Data’ Push Means Big Bucks for Drone Brains

from Danger Room by Robert Beckhusen

Illustration: DARPA

The military has a data problem. More specifically, it has a 

too-much-data problem. Analysts have to sort through massive 

amounts of information collected by orbiting surveillance drones 

and satellites, or finding the data trails left behind by spies inside 

defense networks. Sorting through all this data is also necessary 

for making unmanned vehicles more autonomous.

Bring on the White House’s new “big data” research initiative. 

Announced this morning, the plan aims to invest “more than $200 

million” in six government agencies to develop systems to 

“extract knowledge and insights from large and complex collections of digital data,” 

according to a White House statement (.pdf). That means anything too 

large for normal software to handle, meaning data sets of at least 

dozens of terabytes, at minimum. The biggest beneficiary of all this 

could be the Department of Defense.

The Pentagon already spends hundreds of millions annually on 

“big data”-esque problems. The initiative announced today could add to 

that kitty up to $60 million per year for new research projects. That 

includes a $25 million yearly sum for a new Darpa data mining 

program called XDATA, which is broadly defined as a tool to 

analyze large amounts of meta-data and “unstructured” data like message traffic. 

(In comparison, the Department of Energy is receiving only $25 million 

in funding for a new data mining institute and the National Science 

Foundation is being granted $13.4 million.)

Where is all the rest of that defense research going? Several places, 

and a lot of it for helping drones crunch the massive amounts of information 

pulled from sensors.

“The Department of Defense if placing a big bet on big data.” Zachary 

Lemnios, the Pentagon’s research and engineering chief, told reporters 

on Thursday. “We are within sight of a new generation of systems that 

understand and interpret the real world with computer speed, computer 

precision and human agility. These systems will not only be central to 

helping our commanders and analysts make sense of the huge volumes 

of data our military sensors collect, they will also support multiple 

missions.

Some of these systems, like Darpa’s Mind’s Eye, seeks to develop 

“visual intelligence” in aerial sensors, which would give military computers 

he ability to connect visual data with pre-written cues. Effectively, that 

could mean giving drones the tools to spot enemy soldiers automatically. 

Other programs likely to benefit include the Insight program, which can 

help drones spot potential threats on the battlefield.

That information is “growing rapidly in both volume and complexity,” Darpa 

acting director Ken Gabriel said. “From scraps of paper to hard drives, to 

overhead imagery and intercepts — the data collected is often imperfect, 

incomplete and heterogeneous. This trend is further accelerated by the 

proliferation of various digital devices and the internet. All of which is 

used by our adversaries to operate and hide in this data terrain. The sheer 

volume of information itself is creating a background clutter.”

Clutter so thick, even a quarter-billion dollars in investments may not be 

enough to cut through.