Top-secret US intelligence documents leaked by whistleblower Edward Snowden have revealed details of a key signals intelligence program used by the Australian intelligence community to harvest internet and telecommunications traffic across the Asia-Pacific region.The system known as X-Keyscore allows the US National Security Agency and international partners including Australia to monitor ''nearly everything a typical user does on the internet'', according to the leaked documents published by Britain's Guardian newspaper.
According to classified intelligence training materials, X-Keyscore allows the NSA and its allied partners to comprehensively monitor the emails, web browsing, internet searches and social media use of targets.
This includes ''real-time target activity [tipping]'' and a ''rolling buffer of three days of all unfiltered data'' with the ''full take'' stored at collection facilities - enabling analysts to retrospectively access the communications of newly identified targets.Significantly, all the secret documents are classified as available to personnel from ''5-eyes'' intelligence partners: the US, UK, Canada, Australia and New Zealand.Australian intelligence sources recently confirmed to Fairfax Media that Australia's electronic espionage agency, the Defence Signals Directorate, was a ''full partner'' in the program.It is claimed that by 2008 more than 300 terrorists had been captured thanks to intelligence from X-Keyscore.However, Australian intelligence sources emphasise the reach of the system for diplomatic, political and economic intelligence collection on ''targets of interest across the whole Asia-Pacific'' - including China, Indonesia, Malaysia, India, Sri Lanka and Pakistan.Documents previously disclosed by Mr Snowden identified Australian signals intelligence facilities at Geraldton in Western Australia, Shoal Bay near Darwin, HMAS Harman near Canberra and the US-Australian Joint Defence Facility at Pine Gap near Alice Springs as contributors to the global collection of internet and telecoms traffic under the X-Keyscore program.Fairfax recently reported the construction of a new data storage facility at HMAS Harman to support the surge in data collection by Australian intelligence agencies.Last week the US House of Representatives only narrowly voted down a proposal to defund the NSA's ability to collect electronic information, including phone call records.But President Barack Obama has been forced to declassify aspects of the surveillance programs.Democrat senator Dianne Feinstein - the chairwoman of the US Senate intelligence committee and a staunch supporter of the wide-ranging electronic surveillance - says she is looking into reforms including greater public reporting of phone interception and meta-data collection statistics.Australian diplomat cables show close consultation between US and Australian officials about Mr Snowden's disclosures. However, most details have been redacted on the grounds they would reveal confidential discussions or ''comment and analysis of any implications of Snowden's breach for Australian communications systems''.Read more: http://www.theage.com.au/technology/technology-news/spy-program-tracks-nearly-all-web-use-20130801-2r2dc.html#ixzz2aiyVeF9e

NSA training guide shows how analysts like Snowden read American email exchanges and chatroom history without a warrant

  • Training manuals on the NSA's web spying system were released today
  • X-Keyscore, the system used to mine information online, is described by the NSA as their 'widest-reaching' database
  • The manual explains how analysts were able to spy on Americans without a warrant
  • For 2012, the database had 41billion records stored during a single 30-day period
  • The Senate Judiciary Meeting is having a hearing on the surveillance program today
  • Obama Administration will declassify documents on its telephone spying program
By Daily Mail Reporter and Associated Press
PUBLISHED: 16:45 GMT, 31 July 2013 | UPDATED: 04:43 GMT, 1 August 2013
With only thin justifications, NSA agents have been spying on Americans online without a warrant- reading emails, chats and browsing histories. Today, The Guardian broke down the process by which NSA analysts tap internet activity by publishing a training guide used to introduce new analysts to X-Keyscore. Described as the 'widest-ranging' database, X-Keyscore allows analysts to wiretap basically anyone. Snowden was referring to the system when he said in June: ‘I, sitting at my desk can wiretap anyone, from you or your accountant, to a federal judge or even the president, if I had a personal email.’
How they did it: New documents published in The Guardian today show how NSA analysts like Edward Snowden spied on Americans without a warrant
How they did it: New documents published in The Guardian today show how NSA analysts like Edward Snowden spied on Americans without a warrant

Massive reach: X-KEYSCORE is a program involving over 700 servers worldwide that supposedly hold basically everything that every casual user does on the internet
Massive reach: X-KEYSCORE is a program involving over 700 servers worldwide that supposedly hold basically everything that every casual user does on the internet

The manual explained in detail how analysts could side-step warrants and spy on ‘nearly everything a typical user does on the internet’ – including the web activity of Americans.These revelations come in light of a congressional hearing taking place today which Representative Alan Grayson said would be ‘an ad hoc, bipartisan hearing on domestic surveillance.’Senior intelligence officials from the Department of Justice, NSA, Office of National Intelligence, and the FBI, as well as critics, will be speaking about the surveillance program.
Hearing: The Senate Judiciary Committee is holding a hearing today on the surveillance programs carried out by the NSA
Hearing: The Senate Judiciary Committee is holding a hearing today on the surveillance programs carried out by the NSA

Legally, the NSA is not allowed to spy on U.S. persons without a Fisa warrant. But X-Keyscore provides a loophole for this law.If an American had contact with a foreign target, that allows the NSA to track them without a warrant.Analysts just provide a justification for the tapping, usually from several excuses, some included in a drop-drown menu, and they can start surveillance.No court official or NSA official signed off on the individual justifications. 'It's very rare to be questioned on our searches,’ Snowden said in June. ‘and even when we are it's usually along the lines of: let's bulk up the justification'.The NSA denies this claim, issuing a statement to the newspaper today that there are ‘multiple technical, manual and supervisory checks and balances within the system to prevent deliberate misuse from occurring.’And once tapped, the type of information searchable is startling. All they need is simple information such as an email, IP address or phone number to access web activity. 
Manual: An excerpt from the X-Keyscore training guide shows how simple log-in information allows analysts to access a wide-range of information online
Manual: An excerpt from the X-Keyscore training guide shows how simple log-in information allows analysts to access a wide-range of information online

By entering a person’s email and a justification, an analyst can read all email exchanges, narrowing down by time range.
Justification: Analysts can read personal email exchanges with an address and a simple 'justification' for the surveillance
Justification: Analysts can read personal email exchanges with an address and a simple 'justification' for the surveillance

Due to Facebook’s corroboration with the agency, analysts could read chats and private message on the social networking site by entering the target’s username and a date range.They could perform reverse searches as well, entering a specific website and gaining information on all the IP addresses that visited it. The sheer amount of information being collected online is so large that most information can only be stored for three to five days.To store ‘interesting data’  for longer, the agency built a database that can hold some documents for as long as five years.According to an NSA report published in 2007, 850billion ‘call events’ had been collected and stored, and about 150billion internet records. They reported that each day 1-2billion new records were added.In 2012, there were 41billion total records in the X-Keyscore system for one 30-day period.While the agency has tried to keep U.S. communications separate from foreign communications, they have admitted that even strictly U.S. communications can travel on foreign systems and it’s sometimes hard to identify the national origin of of a communication. 
Domestic surveillance: Drop-down menus give analysts several justifications to monitor Americans based on a foreign target connection
Domestic surveillance: Drop-down menus give analysts several justifications to monitor Americans based on a foreign target connection

In their statement, the NSA continued to defend their broad surveillance: ‘These types of programs allow us to collect the information that enables us to perform our missions successfully – to defend the nation and to protect U.S. and allied troops abroad.’
Wide-ranging: X-Keyscore allowed analysts access to 'nearly everything a typical user does on the internet'
Wide-ranging: X-Keyscore allowed analysts access to 'nearly everything a typical user does on the internet'

Hoping to stop some of the opposition in congress to domestic surveillance, the Obama administration announced today that it will declassify a set of documents on its telephone spying program. The documents will provide little solace, however, to Americans hoping to understand the legal analysis that underpinned the widespread surveillance.
And the redacted documents show only in broad strokes how NSA officials use the data.One particular type of analysis, called "hop analysis" is hinted at but never fully discussed. That allows the government to search the phone records of not only suspected terrorists, but everyone who called them, everyone who called those people, and others who called them, as well.With that authority, the government can search the records of millions of people in an investigation of one person.
Read more: http://www.dailymail.co.uk/news/article-2382011/NSA-training-guide-shows-analysts-like-Snowden-read-American-email-exchanges-chatroom-history-warrant.html#ixzz2aj1JLxJE
Follow us: @MailOnline on Twitter | DailyMail on Facebook