The US National Security Agency (NSA) has built a search engine in the style of Google to help employees within the NSA and other intelligence agencies to search for information within its vast databases of metadata.
Named ICREACH, the "one-stop shopping tool" was revealed in the latest documents released as part of the Edward Snowden leaks. More than 20 US government agencies are able to use the "Google-like" engine to access over 850 billion records about phone conversations, emails and internet messages.
The latest revelation comes after classified documents were obtained by The Intercept. Documents describe ICREACH as a "large-scale expansion of communications metadata shared with [intelligence community] partners".
Documents show that the tool has been operational since a pilot launched in May 2007, with data being shared with more and more agencies as time moved on.
According to information leaked from a top-secret memo, the development of ICREACH represented a landmark moment, even within the intelligence gathering community.
"The ICREACH team delivered the first-ever wholesale sharing of communications metadata within the US Intelligence Community," said one of the leaked documents.
"This team began over two years ago with a basic concept compelled by the IC's increasing need for communications metadata and the NSA's ability to collect, process and store vast amounts of communications metadata related to worldwide intelligence targets."
ICREACH was designed to be the largest system ever developed for internally sharing intelligence data within the US. However, the data stored within it isn't limited to US citizens, with information about foreigners' communications also stored within the system.
In both domestic and foreign cases, the vast majority of individuals who have data about them stored within ICREACH aren't suspected of committing any crimes whatsoever.
Legal experts have reacted with shock to the revelations surrounding ICREACH, with concerns that authorities are abusing their access to metadata.
"To me, this is extremely troublesome," Elizabeth Goitein, co-director of the Liberty and National Security Program at the New York University School of Law's Brennan Center for Justice, told The Intercept.
"The myth that metadata is just a bunch of numbers and is not as revealing as actual communications content was exploded long ago. This is a trove of incredibly sensitive information," she added.
Last month, American mathematicians voiced their concerns about the mass collection of data by the NSA, stating it to be both ineffective in achieving the stated goal of "preventing terrorism" and dangerous for democracy.
Sometimes, the power of the mainframe is the most cost effective answer. Computing's Peter Gothard puts Computing's readers' questions on the future of the mainframe to IBM's Z13 expert Steven Dickens.
This Dummies white paper will help you better understand business process management (BPM)