“The raw volume of the proposed collection is enormous,” wrote Judge Colleen Kollar-Kotelly, who was then the chief judge on the secret surveillance court. The government censored the date of her ruling in the publicly released document, and many sections — including a description of what she had been told about terrorism threats — were heavily redacted.
The ruling was among a trove of documents that were declassified and made public by the Office of the Director of National Intelligence in response to Freedom of Information Act lawsuits, including those by the American Civil Liberties Union and the Electronic Freedom Foundation.
Many of the documents have historic significance, showing how Bush administration surveillance programs that were initially conducted without court oversight and outside statutory authorization were brought under the authority of the surveillance court and subjected to oversight rules. The documents also included reports to Congress, training slides and regulations issued under President Obama.
The Bush administration temporarily shut down its bulk collection of email logs after Justice Department lawyers raised legal concerns in March 2004. Judge Kollar-Kotelly declared the collection lawful in July 2004, according to documents leaked by Edward J. Snowden, the former N.S.A. contractor.
The email metadata — information like the identities of senders and recipients and the and the dates of messages, but not the content — was used in searches of unknown associates of terrorism suspects. The Obama administration has said it shut down the email metadata program in 2011 for “operational and resource” reasons.
Several other court documents released on Monday indicated that the program had difficulties with collecting Internet communications beyond the scope of what the court had authorized. Redactions made it difficult to understand the specifics of the problems, but an accompanying statement offered more details. At one point, it said, the government had shut down the program for several months “because of the significance and complexity of these incidents.”
The New York Times reported in 2009 that the N.S.A. had intercepted private email messages and phone calls of Americans on a scale that went beyond broad legal limits. A statement released on Monday said that an excess collection problem in 2009 was the result of “longstanding compliance issues associated with N.S.A.’s electronic communications and telephony bulk metadata collection programs” and that the N.S.A. “recognized that its compliance and oversight structure had not kept pace with its operational momentum.”
In a statement, James R. Clapper Jr., the director of national intelligence, said that with the new releases, nearly 2,000 pages about surveillance matters had been declassified since President Obama instructed him in June to “make public as much information as possible about certain sensitive programs while being mindful of the need to protect sensitive classified intelligence activities and national security.”
“Release of these documents reflects the executive branch’s continued commitment to making information about this intelligence collection program publicly available when appropriate and consistent with the national security of the United States,” he said.
The trove also included the Bush administration’s 2006 application for initial approval by the surveillance court to collect bulk logs of all domestic phone calls under a provision of the Patriot Act that allows the collection of business records deemed “relevant” to an investigation, another program it had previously undertaken unilaterally. The call record program is still active.
“Here, the government’s interest is the most compelling imaginable: the defense of the nation in wartime from attacks that may take thousands of lives,” said the Justice Department brief, which was signed by Alberto Gonzales, who was then attorney general. “On the other side of the ledger, the intrusion is minimal” into privacy concerns because the calling logs did not include any content of communications.
The documents show that as early as 2006, an inspector general review recommended tighter controls over the bulk telephone metadata program to reduce the risk that they would violate the limits on the collection of data. In 2009, the court would sharply rebuke the N.S.A. for violating its own procedures and misleading the nation’s intelligence court about how it used the telephone call logs.
Jameel Jaffer, a senior lawyer with the A.C.L.U., argued that the release of the documents demonstrated what he argued were structural problems with the surveillance court, which decides major issues.
“This a reminder a lot of the most important and far-reaching decision of the past decade was issued by this court, which meets in secret and hears only from the government and doesn’t publish its decisions,” Mr. Jaffer said.
The full scope and details of any revelations in the documents were not immediately clear because of the large volume of materials and the late hour at which they became available. It appeared likely to take days for journalists, privacy advocates and other close watchers of surveillance policy issues to finish scouring the trove.