Note: This post is based exclusively on the material released to the public and available on the DHS Web site. I have not seen the full assessment, which would include supporting facts. Nevertheless, an executive summary is a condensed version of a full report and should offer readers enough information on which to base a decision.
On 1 March 2021, the Office of the Director of National Intelligence (ODNI) released a one-page executive summary of a coordinated intelligence assessment regarding the threat posed by domestic violent extremists (DVEs).
The assessment was not an analysis, but a short-term forecast. The writers predicted past and current socio-political events would spur DVEs into action, thus drive up incidents of domestic terrorism over the next few months. Setting aside the appearance of recency bias, speculation is not analysis. An analyst’s job is not to predict future events, but to explain and contextualize past ones. Analysts pass on their findings to an authority, who uses their analyses to base decisions. But it’s hard to resist pressure, especially when it comes from higher-ups, so sometimes analysts find themselves in this position.
If that is what happened here, I would have strongly suggested analysts widen their data set to all attacks inside the United States by both domestic and foreign actors over a broad timeframe. I prefer 10 years or more. Over that period, which DVEs instigated the most attacks–a matter of quantity; and, separately, which DVEs instigated the most violent attacks–a matter of quality. Report both of these results.
Examine patterns. Over the period, did incidents by a specific DVE category rise, fall, remain steady? Did attacks follow meaningful events, which is what analysts are predicting here? If you can show a pattern, it puts a forecast on firmer ground. You’re still making a prediction, which is dicey, but you can say over the past 10 years attacks have consistently followed this pattern, so we have a strong sense we will see a similar trend during the next few months.
The assessment was also hobbled by the typical language of the IC: “elevated”; “almost certainly”; “more likely” “most likely”; “contribute to.” In analysis, words of estimative probability give the analyst cover because we can never say with absolute certainty we know all of the circumstances that contribute to an incident. In this case, words of estimative probability gave analysts room in case their forecast was wrong.
Analysts called the risk of an attack by DVEs in 2021 “heightened,” and later, “elevated.” What is heightened? What is elevated?
The Department of Homeland Security itself is not clear. Under the post-9/11 color-coded advisory scale, elevated meant “significant risk of terrorist attacks.” Post-2011, the agency replaced the threat advisory scale with the threat advisory system, which updated the word elevated to mean “a credible terrorism threat against the United States.” Yet, even a revised title, “Domestic Violent Extremism Poses Credible Threat in 2021,” does not offer anything in terms of actionable intelligence. To put it colloquially, there is not enough there, there.
Next, is the data set. What data were used to support this assessment? Constructing a data set is tough. If you think otherwise, you probably haven’t been tasked with putting together a strong, defensible collection. Which incidents do you include? Do FBI stings count? Disrupted plots, or only completed acts? Only domestic actors? Domestic actors with ties overseas? Final dispositions or are arrests sufficient? How do you categorize an attack if investigators never determined a motive, i.e, the Las Vegas Harvest Music Festival shooting? Would you even include that as an act of domestic terrorism?
As an added complication, the Transactional Records Access Clearinghouse (TRAC) at Syracuse University brought up a critical point about 2020 domestic terrorism statistics in their report, “Domestic Terrorism Prosecutions Reach All-Time High in FY 2020.”
“U.S. Attorneys’ offices vary greatly in their numbers of domestic terrorism prosecutions. The largest during 2020, a total of 78 prosecutions, were brought in Oregon federal courts…
…
“At the other extreme, many U.S. Attorneys’ offices across the country brought no domestic terrorism suits, or just a single suit in all of FY 2020. This includes the U.S. Attorney in the Western District of Washington (Seattle) who was recorded as bringing only a single domestic terrorism suit, although protests there[,] similar to those in nearby Portland, Oregon, had figured prominently in the news.”
TRAC, Syracuse University
So, now we have another dilemma, which is ensuring we keep politics out of intelligence work.
And one final note: analysts are rarely called to account for their predictions. I am not aware of any significant follow-up. Revisiting forecasts after they expire can offer valuable insight. If only to affirm the futility of predictions.
The Intelligence Shop 