There has been a great deal of discussion about classified information in the US lately, and it seemed worth taking some time to explain the system to those who don't normally have much cause to deal with it.1 I occasionally do, although not on a particularly high level, and most of what follows is based on research, not personal experience.
The US has two parallel systems for defense information that the government would like to keep secret. For nuclear matters, mostly relating to the design and production of nuclear weapons, the system is based on the Atomic Energy Act of 1954, and run by the Department of Energy. Everything else uses a common set of clearances, governed not by law but by a series of Executive Orders dating back to the 1950s, the current authority being EO 13526, issued in 2009. We'll primarily focus on this system, as it covers the vast majority of classified documents.
In theory, there are three possible levels of classification: Confidential, Secret and Top Secret. In practice, Confidential is rarely used, and the majority of work is done at the Secret level, with Top Secret being used for information that requires more care. Exactly what that might be, we'll come back to later. In theory, anyone with the relevant clearance is automatically allowed access to information at a given level. If Program A does a Top Secret test where Widget B fails, and the expert on Widget B has a TS clearance but isn't involved in Project A, he can come right in and help sort out the problem. Restricting information based on need-to-know is still encouraged, so you can't just ring up and ask about information for fun, but it's still not that locked-down.
But these broad categories pose a serious problem for particularly sensitive information, and as a result there are also what are known as Special Access Programs, the most famous of which is the Sensitive Compartmented Intelligence (SCI) designation, used for information that would reveal exactly how intelligence was gathered. Special Access Programs create compartments that are restricted to people who have been specially cleared for them on the basis of need to know, sort of like miniature security clearances. This doesn't generally involve a separate background check, but it does mean that the most sensitive information is restricted to select groups. Each compartment has a codeword, and while in theory information at any level can be SAP/SCI, in practice it's always top secret, and TS/SCI is a de facto fourth clearance level at the top of the pyramid. Even dealing with normal classified information is a major administrative pain, and adding to the pain means that something which is TS/SCI is generally quite serious.
So how does this work in practice? Let's take a look at a hypothetical example of the US gathering information on a Russian radar. The project to get the intelligence is going to be TS/SCI. Maybe someone with access sold us the manuals, or a disgruntled engineer passed us a bunch of technical documentation. Maybe we got data through a cyberattack, or a submarine snuck in and covertly recorded a test. Exposure of any of these methods would be a serious problem, so these compartments are kept small, with only those directly involved in the operation read in. The engineers who will analyze it definitely aren't read in on the details, and may be working at the TS level, or may be in a different compartment, as even the raw data could give away quite a bit. In any case, these engineers write up a report that details what we know about this radar system. As lots of people will need this information, it's probably classified TS because a close reading could give away at least some information about what data we have on the radar, and thus how we got it. So for actual operational use, someone else is going to take that report and use it as the basis for a Secret document that gives the pilots the information they need, like the range of the radar.
So what is getting a security clearance like? In practice, it's just a bunch of paperwork you fill out listing everything you've done for the past 10 years: houses, jobs, friends, foreign contacts, shoe sizes and so on. For a Secret clearance, they'll probably just get your fingerprints, check public records to make sure you aren't lying and maybe contact a person or two. For a TS/SCI, they'll do a lot more digging and an in-person interview, and might even ask you to take a polygraph. The most common reason for failure is financial irregularities (other powers are quite willing to pay for American secrets) followed by lying on your paperwork, foreign ties, drug use and a criminal record, or some combination of these, like being a black-market Pokemon dealer.
Dealing with classified information is more paperwork. There are specific standards for every part of the process, from the briefings and annual training you have to do to make sure you know not to disclose it improperly, to the security measures in place for storage of physical documents to the rules around accounts on classified information systems to the way that documents have to be marked. It's all rather a pain, with the pain increasing as the document gets more sensitive, but it does generally work. It's also much less exciting than you'd expect. The vast majority of classified information is less "here's the latest CIA report on Putin's strategy in Ukraine" and more "because we carried out the radio test with encryption on, there are keys in the data, so all data from said test is now Secret". The actual decision about if something is actually classified comes from one of a few thousand people with Original Classification Authority, and every document which contains that information is automatically classified derivatively, in accordance with the program security guide. Any declassification has to be done at the OCA level, and would affect all documents which contain that information.
There are a few other quirks of the system that are worth mentioning. One is that anyone with a clearance is officially prohibited from viewing classified documents they aren't cleared for, even if they're on the front page of the Washington Post. This is primarily to keep people who know more than has leaked from giving things away. Another is the nuclear secrecy system mentioned earlier. This gives out two clearances, the Q clearance that is pretty much equivalent to Top Secret and L, equivalent to Secret. In the past, these required separate investigations, but these days, I believe they share background checks. Third is the concept of Controlled Unclassified Information. This is information that isn't classified, but is also sensitive and shouldn't be widely distributed. This was an attempt to consolidate dozens of previous designations, most prominently FOUO (For Official Use Only), but it seems to have made the problem worse. Third is the use of various special markings to designate what kind of information is contained within. The most common of these is probably NOFORN, which means that it's information which isn't to be shared with a foreign government. Others designate things like "this is communications intercepts, so be careful how it gets transmitted" or "this document talks about sources and methods".
Of course, this is only a small fraction of the complexity that comes with attempting to secure America's secrets. While most nations have broadly similar systems, and even interact with the US in certain ways (which means that some information can have other handling requirements based on the agreement under which it was provided) the sheer scale of the US defense complex means that our system is probably unsurpassed, and while the suggestion that it was set up to drive people trying to penetrate it mad isn't correct, this is primarily because the US government isn't competent enough to pull something like that off intentionally.
1 Please note that Naval Gazing's policy on culture war remains in effect. I've crossposted this to DSL, and you should bring up any object-level discussions of recent political events there. ⇑
Comments
What's DSL?
What's DSL?
Do you think that, overall, the US information classification system is good? If you had the benevolent social planner wand, would you leave the system as it? Change it? End it?
Speaking from a tech perspective, your post above has more-or-less reinforced my existing beliefs on the subject, which is "information wants to be free". I hear you elaborate on the differences in classification level and the way it's used in practice, and all I can think is that the information compartmentalization must be an absolutely massive obstacle to the efficient processing of information, on a sort of capitalistic-style distributed information process kind of argument.
Like you give as an example:
I imagine, in practice, this kind of thing doesn't happen because of various rules and conventions regarding compartmentalization and need to know. Further, I can imagine that the prevention of things like this is a massive blind spot that cripples the effectiveness of our federal military organizations. What are your thoughts in this regard?
@Kitplane
Data Secrets Lox, a forum where I also hang out. I'll edit in a link to the version there.
@Techanon
There's always going to be a tradeoff between ease of access and ease of leakage, and I think they've found a reasonably good position (or set of positions, given the multiple levels involved.) I might try to simplify some pieces of the system, and I'd end the weirdness around nuclear stuff. Probably codify it all as law instead of EO.
No, I was trying to give that as an example of something that does happen, and the quoted example was based on stuff that happens at work. Obviously, there's some level of need-to-know, for instance I, who work on aircraft, can't just call up and ask for the specs for the SPY-3 radar because I'm interested in it. But if I need to get someone's opinion on a classified thing and they have a clearance, I can just go to the classified conference room and ask them.
It's probably not in my top 5 things that cripples the military. Everything is tradeoffs, and when the downsides of a leak are so great, you want to be careful.
What if you are merely married to "a black-market Pokemon dealer"? Are spouses, relatives and friends likely to cause issues for you when getting a clearance (assuming they are citizens, so don't themselves fall under foreign ties).
Speaking only as an outsider who reads a lot of history, my guess is that in peacetime it seems all downside, friction slowing down or stopping people who need to know stuff.
In wartime, it works. Classic example is the Enigma decrypts in WW2. It was vitally important for the Allies to distribute knowledge of Axis plans and operations to Allied field commanders who could do something about it, AND vitally important to keep secret that the Allies were decrypting the supposedly uncrackable German encryption for communications. Shenanigans ensued.
There's actually some of this in Neal Stephenson's "Cryptonomicon", where one of the two timelines follows WW2 characters (real and imaginary) involved in various ways with Enigma.
A non-fictional book is Vannevar Bush "Modern Arms and Free Men". He was fairly high up in the US military R&D establishment during WW2. IIRC he argued that the US/Allies did do better in WW2 because they were generally more open and shared more information than the dictatorships. Which is not the same as being totally open with everything unclassified. Constant adjustment required.
I'm curious about the reason for the separate nuclear classification. Did people in the early 50s suspect that labeling something "top secret" just told Soviet spies what juicy bits of intel to send first?
Oh, I realized another possible reason right after posting: there are probably lots of provisions for (semi?)-automatically sharing regular classified data with allies, right? And the US had reneged on their nuclear sharing deals as a matter of policy.
@Alexander
It can happen, although I don't think it's as common.
@Echo
I think it's a relic of a fight between Congress and Truman over nuclear stuff in the late 40s. As you point out, there was in particular the withdrawal of cooperation with the British.
@Alexander,
I did work with a guy who was previously a "vague handwave" on a US nuclear sub that did exactly the sort of "sneak in and covertly record a test" type stuff.
One day, in the middle of a voyage, he lost his clearance. Because the information that he had married a lady born in Russia had finally made its way to someone who freaked out about it.
He had formally notified his commanding officer and the security guys at his department about his fiance's background. Everyone was cool with it. But it was put in a report which gradually made its way through the system until several months after the marriage when the alarms went off.
This caused some issues, in the middle of the voyage, when they received a message that their "vague handwave" guy was no longer cleared to be there.
Eventually things were sorted out, but I got the impression that it certainly disrupted that one voyage.
How does this stuff cross over internationally? Is a SECRET document in the US treated roughly the same whether it was produced domestically or passed across by MI6, for instance?
Of course the President does have the power to authorise a game of global thermonuclear 20 questions, as Nixon did with France.
Lambert (all three times):
A big part of NATO is standardizing that kind of stuff and the ability to handle others' secrets is a requirement of NATO membership.
As Anonymous notes, NATO has this sort of thing pretty well standardized. You'll sometimes see documents marked to indicate that they're full of NATO Secrets rather than Pentagon Secrets; that's mostly irrelevant to the end user but does change the rules about e.g. who can declassify things. American spook-bureaucrats are allowed to read James Bond's field reports, they're not allowed to blow his cover (he does that all by himself).
There's also Five Eyes aka FVEY, an intelligence-sharing alliance of the US, UK, Canada, Australia, and New Zealand that is tighter and more standardized than NATO. The classification markings will tell you which set of foreign countries, if any, are approved for distribution.
I used to have DoD SAP clearances as a contractor employee (CPS so I got to get the overwater flight clearance with the dunker test) that required a SSBI (both happened back in the 80s/90s: I gather things are different now with Tier 5) but not the really thorough grilling that comes with CI etc. like the lifestyle investigation, poly, ... (I worked with people who did).
I will say that I worked with people who told me that another SAP program they worked on (same campus, different SCIF) that their program required a commitment never in their lifetime to travel to the PRC, even after being read out of their program (and I tended to believe them, though they certainly could have been apply fecal matter to me).
I also have a friend who worked on engineering side of some reasonably heavy strategic ELINT systems whose wife was a serious hard lefty (they have an interesting marriage, although both have moved centerward on their politics) and whose mother in law had been, pre my friend's marriage, a card carrying CPUSA member. I heard interesting stories from him about his wife's reports of her interviews for his SSBIR.
@Alexander, there is a bit of a data point there for you, although my friend's wife couldn't be blackmailed over things she was quite proud of.
I also worked (early 80s) on DoE enrichment programs with people who had Q clearances (DoE equivalent of Top Secret - if memory serves correctly the DoE used the same SSBI investigation with the same SF86 as DoD but DoE had their own adjudication rules), although I only had the L (like a DoD Secret at the time in terms of the level of investigation).
There are both DoE (Q at least) and DoD (CNWDI) clearances for nuclear weapons. I had neither, but the Qs I knew about were all related to enrichment technology (makes sense in that if you can enrich natural uranium far enough, it's pretty easy to make a weapon - see the South African nuclear program). I don't know what if any additional investigation is or was needed for DoE weapons design or effects work (plus someone working on warhead production tooling might well have a need to know on certain very sensitive bit .
The (DoD clearance, not DoE) CNWDI guy I knew worked on penetrating RVs, so he didn't necessarily know anything more than I did from open sources about weapon design, but he knew a lot about what the as-encapsulated physics packages could and could not tolerate in terms of externally applied insults. I also assume, perhaps inaccurately, that targeting people (both SIOP and tactical) have CNWDIs with clearance into effects SAPs.
Apologies for the quadruple (!) comment post - bean, could you please delete the duplicates? There was no feedback to the post button - is that a Thing with Chrome on Win 10 with Blogit? I can use Edge or Opera if they work better.
For highly sensitive programs (e.g. NSA or CIA employee or contractor): Yes, absolutely. A spouse or close relative that could put you in a compromising situation (e.g. criminal activities that could be the subject of blackmail, serious gambling debts, etc.) would be a major problem if it were apparent to the investigator.
For a more typical "secret" clearance (e.g. a surface warfare officer or contractor working on a plane for the Air Force): Often no, if only because of the lower level of scrutiny applied to your friends and relatives during the investigation.
At the same time, it should be a real concern for you personally even if you are granted the clearance. Imagine a person working for a foreign government threatened to report the black market Pokemon activities of your spouse unless you provided them with classified information. This is really not a situation you want to end up in. (For what it's worth, the right move would always be to report the approach to your program's security team. Don't dig yourself a deeper hole.)
One practical matter: If you think you might be interested in such work in the future, you should start tracking the relevant information you would need for your security clearance questionnaire now. Specifically, that means:
Address, start date, and end date for every place you've lived in the past 10 years, plus someone who can provide a reference for every address in the past 3 years.
Full details, start date, end date, and supervisor contact info for every job you've had in the past 10 years.
Countries and dates of all travel outside the US in the past 7 years.
The full form is publicly available here: https://www.opm.gov/forms/pdf_fill/sf86.pdf
It asks many other questions as well, but those 3 topics are much easier to track "as you go" (e.g. add every trip you take to a spreadsheet) rather than trying to reconstruct things several years later when you're filling in your application.
As an additional wrinkle, there are actually 2 types of polygraph exams: counterintelligence (CI) and lifestyle. This page has a good description of what's covered in the two: https://support.clearancejobs.com/t/the-differences-between-counterintelligence-lifestyle-and-full-scope-polygraphs/46
In practice, most people just get a CI polygraph or get "full scope" (CI + lifestyle). I've never heard of someone getting a lifestyle polygraph without the CI one.
In addition, very sensitive programs will also perform a psychological evaluation.
Alex gives good practical advice, and I'm kind of annoyed with myself for not thinking of it, particularly given how useful my first SF86 was when I was filling out paperwork for job applications and the like.