The Pentagon in 2003 wanted to create a database on every American

Ten years before the recent global panic over the U.S. government’s domestic spying program, the Pentagon solicited contractors for a searchable database of people’s lives.

In 2003, the Department of Defense’s Defense Advanced Research Projects Agency (DARPA) hoped to create a database that would amass everything about the life of a person participating in the project, ranging from GPS coordinates to every email and phone call sent and received.

The participant would wear a camera, microphone and sensors to record the minutia of everyday life. The program, called Lifelog, would act as a person’s digital diary.

Lifelog was a feature of DARPA’s Perceptive Assistant that Learns (PAL) program, which was meant to create intelligent digital personal assistants.

DARPA hoped the database would further research in artificial intelligence. It would take the data gathered from consenting participants and map out relationships and events found in the data.

After privacy advocates voiced concerns that effectively brought Lifelog to a halt in 2004, DARPA rebooted its attempt to create PAL technology several months later by tailoring the project with a military focus in order to assuage critics.

Private sector versions of PAL technology also emerged on the market in later years, Apple’s Siri being a direct descendant of DARPA’s PAL initiative.

DARPA would not confirm any connection between the National Security Agency’s surveillance database technology, such as PRISM, and DARPA’s PAL program.

The revelations about the U.S. government’s counter-terrorism surveillance apparatus, however, illuminate how both governments and corporations effectively collect data on the commercial and personal activities of everyday life.
http://dailycaller.com/2013/07/01/pentagon-wanted-searchable-database-of-peoples-lives-in-2003/

Outsourcing US national security to private corporations:
Not all federal employees are CIA officers. No statistics of how many took advantage of the policy were forthcoming, but the policy itself was disturbing enough to lead to questions before Congress.

Why is this allowed? Defenders hold it is important to stop the "brain drain", where the best and brightest depart the agency for better pay. Supporters argue the level of compensation at CIA doesn't stack up against other federal or private-sector employment. This "discrepancy" leads talented employees to look elsewhere – for roles in which their skills fetch better prices. The CIA thus finds itself in competition with the private sector to keep personnel on whom it has expended a great deal of money and effort in training.

Demand for intelligence work has ballooned since 9/11, though government belt-tightening in recent years has had an effect. Washington has decided, rather than expanding intelligence services in a permanent sense, to farm out the increased workload to private contractors. This has advantages, as for any business seeking to outsource operations. When the need for the work reduces, it is easier to allow a contract to expire than it is to lay off excess federal employees or to keep them on, under-employed.

But there are drawbacks as well. Rather than paying solely for the service, the price also reflects the contractor's own costs – their HR, finance, management and marketing departments' wages, office costs, even redundancies. It is questionable if contracting saves anything at all. One doesn't have to look hard to find stories of wasted costs, foul-ups or even corruption in security contracting.

Just last week, USIS, the contractor responsible for performing background checks for security clearances – including Edward Snowden's – was accused of misleading the government as to the thoroughness of their investigations. Lawmakers have been told that thousands of background checks may have been improperly conducted. (The firm has made no comment, but issued a statement saying it was co-operating with an investigation.)

Private intelligence contractors look to hire employees who already have active clearances and intelligence experience – requirements not easily obtained elsewhere than an intelligence agency. The problem becomes circular. Farming out intelligence work to contractors creates its own demand.

Agency employees leave for jobs with higher pay because the government itself has created a market for them, creating the very "brain drain" it seeks to stem. This is because bean-counters are worried about the effect on the budget – namely, of salary, benefits and pension obligations to employees. In the CIA's defence, these decisions are made in Washington, DC, not Langley, Virginia.
http://www.guardian.co.uk/commentisfree/2013/jul/02/contracting-out-us-national-security