Paper # 7. Intelligence Analytic Insufficiency: Fixing the Cognitive Problem

In response to chronic shortcomings, the President, Congress, and senior leaders of our intelligence agencies and service components demand original, prescient and accurate analyses.

This is not the first time leaders have sounded an alarm.

Since 1947, our Intelligence Community (IC) has routinely been charged with so-called intelligence failures caused in large measure by poor analytical performance. Since 1947, Congress has ordered at least 25 major studies, reports, or committees to press for intelligence reforms. The intensity of leadership frustration has been camouflaged by higher priorities such as responding to attacks, repairing damage to national security, and addressing systemic problems. This paper responds to these concerns from a perspective that views analytic insufficiency as a root cause of poor performance.

Intelligence analysts should be self conscious about reasoning processes. We should think about how to make judgments and reach conclusions, not just about the judgments and conclusions themselves.

The complexity of today’s global environment presents a challenging context for intelligence analysis, one that calls for advanced analysis education and training.

Addressing a Persistent Problem

A perfect storm of events involving intelligence analysis triggered a shift from benign neglect to making analytic improvements. Since 9/11, intelligence and operations personnel have placed emphasis on enhancing thinking skills and analytic processes. Seldom in the history of strategic intelligence and related policy failures has senior-executive demand been so focused on setting conditions to develop better all-source analyses. The need is urgent and clear. Consider the attempted airline bombing on Christmas day 2009, revolts across the Arab world (2010), Russia’s invasion of Georgia (2008) and Ukraine (2014), China’s territorial expansions of its periphery (1947-present), and Iran’s disruptive warfare by proxy (1982-present).

Such surprises demonstrate that programs created to transform analysis after the 11 September 2001 attacks have not sufficiently improved the cognitive performance of US intelligence analysts. Clearly the greater IC must look beyond organizational flaws and invest more seriously in advanced analytic education and training. We must maintain a course to develop the analytical skills needed to win today’s fight.

Since the signing of the National Security Act in 1947, Congress and national intelligence leaders have sought to improve the intelligence enterprise, including analysis. However, not since World War II has there been anything approaching a “golden era” of successes.

Current shortcomings can be traced to a decision to disestablish broad intelligence capabilities following that monumental victory. Particularly since the Cuban Missile Crisis in 1961, the threat of nuclear warfare became the nation’s strategic focus. As a consequence, intelligence generally played an ineffective secondary role in conventional matters. Moreover, despite the experiences of conflicts in Korea and Vietnam, no seemed to expect or plan for such “small” wars to recur. Even after they occurred, military leadership did not accept the historical pattern of insurgencies occurring in difficult operational environments.

So we continued to rely general intelligence capabilities – sensors and computers – that emerged to support Cold War nuclear and conventional missions. Intelligence meekly followed operations. The same whimper rings true today unless we hone analytical excellence in our highly contested environment.

Causes and Circumstances

The IC has focused on the machines and computers of intelligence collection, too often neglecting analytical concepts and tools, and the intellectual heft to employ them. This absence of a vision and strategy for improving the cognitive prowess of intelligence analysts has led us to the position where we find ourselves today – an insufficiency of sophisticated analytic skill sets. Not knowing how to think about critical factors such as “will” and “initiative,” and lack of sound analytic principles and mentoring for mental combat have created a Tower of Babel approach to analysis instead of a unified effort.

We foster future shortfalls when we fixate on well-known actors and mission sets to the exclusion of thinking about emerging flash points. Notably, the National Defense Strategy (NDS) of 2018 emphasizes three lines of effort: (1) rebuilding readiness and lethality; (2) strengthening alliances and attracting new partners; and (3) reforming DoD business practices for performance and affordability. While recognizing intelligence as an element of national power, the NDS implies, but does not explicitly state, that better intelligence analysis is vital to achieving objectives.

We must remember that failures to anticipate invasions and grey zone operations result from inadequate investment in sophisticated analytic education across the IC. This deficit is exacerbated by recrudescent insurgencies and increasingly non-linear operational environments (OE).

In the aftermath of an intelligence failure, national leadership inexorably adds layers of management, calls for reorganization, new collection technologies, and more information and database sharing. Failure persists because we ignore the most important, underlying cause: our analysts do not know “how to think,” especially in the non-linear environment that our warriors face in counterinsurgency, counter-terrorism, and other irregular warfare (IW) operations.

The most persistent finding about analysis that permeates every intelligence failure is the post-mortem conclusion that available intelligence was adequate to support a prudent indications-and-warning effort. These intelligence failures were not failures of collection or systems, but of analysis.

Yet another strong component of the analytic challenge is the ineffective Congressionally-mandated, redundant-yet-isolated way we continue to organize the IC. Instead of adopting flat, collaborative, shifting, changing, learning, and adapting networks (such as those we face), we cling to hierarchical bureaucracies that are the antithesis of organizational agility.

The end result is rather predictable — long periods of stasis, punctuated by infrequent, spasmodic reactions to reform after disasters. Moreover, post-crisis legislation such as those that the 911 Commission promulgated typically propagates organizational change while giving short shrift to root causes.

While attempts at organizational solutions have layered on more money and bureaucracy – e.g., the stand-up of the Office of the Director of National Intelligence (O/DNI), the Department of Homeland Security (DHS), and the National Counter-Terrorism Center (NCTC) as a national response to 9/11 – solutions that address higher order thinking and analysis have seldom been addressed. Nonetheless, the root causes of our shortfalls in analysis lies in our inability to properly develop our analysts’ cognitive capabilities.

Further complicating our analytic challenges are revolutionary changes in information and communications technology. These changes, in concert with the forces of globalization, have ushered in a far more complex security environment. As a result, the political, military, and economic environments in which we operate are profoundly nonlinear and unpredictable.

This unprecedented challenge carries enormous implications for analytic and collection strategies. The greatest potential for intelligence reform is to transform our analytic efforts through innovative education and training that empower analysts to understand and penetrate denied minds, to synthesize a vast collection yield, and to create actionable knowledge to warfighters.

Analytic Entropy

Intelligence analysis is in a state of serious entropy. It is no exaggeration to assert that analysis has reached the point of diminishing returns. Often, judgments provided by professional analysts in such authoritative documents as the National Intelligence Estimate (NIE) are challenged by policymakers and other intelligence consumers as “I know better.” Too often, this occurs in order to serve political agendas instead of an overarching national strategy. The media, news service organizations, and national think tanks also challenge intelligence judgments. From intelligence failures and congressional investigation hearings come calls for reorganizing the IC enterprise to address stridently yet shallowly noted intelligence shortcomings. Costly and extensive reorganization of the IC has done little to improve the cognitive performance and prescience of the IC’s 18,000 intelligence analysts.

The IC is not without direction to transform the enterprise. The National Intelligence Strategy includes a call to “strengthen analytical expertise, methods, and practices; tap expertise wherever it resides; and explore alternative analytical views.” Over a decade ago, the 2008 Defense Intelligence Strategy beckoned to “implement collection and analytical transformations to meet in-depth and time-sensitive requirements against all threats, while forecasting and enabling rapid response to emerging events.”

Transformation requires more than the hortatory language of the National Intelligence Strategy. Significant change requires new ways of thinking. This broad policy prescription must be informed by specific guidance to develop rigorous education and training programs that will boost analytic abilities across age groups, ranks, functional positions, and career stages.

Twenty years ago, the Joint Chiefs of Staff incorporated the term “information dominance” into joint doctrine. Today, the challenge is to move from that idea to the concept of decision advantage. Achieving decision advantage in this context means that the United States brings instruments of national power to bear in ways that resolve challenges, diffuse crisis, and deflect emerging threats.

The challenge that the US Intelligence Community faces today is less to access denied areas with sophisticated collection strategies and means, and more to penetrate the minds of terrorists, extremists, populaces, host nation governments, neutrals, and multi-national corporations. These complex targets are the new analytical centers of gravity.

Learning How to Think

To be successful in the kind of wars we are waging in Afghanistan and Iraq, analysts must be able to understand our enemies’ motivations, thinking, intentions, and patterns of behavior, rather than obtaining the innermost “secrets” of specific plans, technical capabilities, or activities. It is no longer enough to capture state secrets. It is no longer sufficient to mine data. Instead, seriously contesting our new adversaries requires application of sophisticated analytic thinking skills. Regrettably in today’s IC, the transformative investment in analysis education and training is woefully underfunded. And it’s at a point in time and our history when the demand for competitive intelligence analysis has never been greater.

There is a compelling requirement to address significant education and training shortfalls in intelligence analysis. Sixty years of doing the same thing over and over without getting a better result seems long enough to conclude that it is time to do something completely different.

Our national decisions makers have a right to expect that intelligence will be accurate most of the time. Employing a favorite baseball analogy, former CIA Director General Mike Hayden suggested that an analytic performance “batting average” of .300 is quite good. He recalled that when a small group of investment bankers once asked him to rate the quality of US intelligence on a scale of 1 to 10, he told them that “the first thing to understand is that anything above 7 isn’t on our scale. If we’re at 8, 9, or 10, we’re not in the realm of intelligence—no one is asking us the questions that can yield such confidence. We only get the hard sliders on the corner of the plate. Our profession deals with subjects that are inherently ambiguous, and often deliberately hidden. Even when we’re at the top of our game, we can offer policymakers insight, we can provide context, and we can give them a clearer picture of the issue at hand, but we cannot claim certainty for our judgments.”

We can do better than a .300 batting average. In our contested information environment, we must. Until we address root causes with proper commitment, bold leadership, and modest educational and training resources, puny progress can be expected. It’s one thing to identify the vision, but entirely another to commit resources towards it. This is where the hand wringing and intellectual breakdown occurs in the IC.

Analytic Vision

First and foremost, must get envision what we need for the future. Without a clear vision, there can be no future-oriented doctrine. With singularly historical doctrine, our strategy rests on a foundation of sand. The educational experience we have in today’s intelligence community is largely built around on-the-job training (OJT)—what worked in the past. This approach cannot cultivate the depth and sophistication of thought needed to win complex wars in which out-thinking our adversaries is far more important than blowing things up and killing people.

A kinetic solution is not the best or certainly sole option for many fights given the kind of enemy we confront, populations we seek to persuade, and the complex environments in which we operate. In this new battlespace, we must penetrate the minds of our adversaries and influence or modify their behavior. Conflicts will be battles of cunning wit. To do this, we need to own the initiative.

The decision maker should expect every intelligence product to explain and articulate: meaning; how to seize and maintain initiative; how the adversary will attempt to do the same, potential second and third order effects; how activities relate to a bigger picture; and implications for possible actions or further intelligence analysis and collection.

In addition, analysts who brief or write for decision makers should have experienced analytical education and developed skilled techniques. Having both types of learning starts with a change in culture.

A Change of Culture

Analysts exist within a “craft culture” that operates as a guild and apprenticeship system. What intellectual foundation underpins this tradecraft? Jeffery Cooper suggests that such a culture builds pragmatically on an accretion of practices that were successful in the past, lacks the strong formal epistemology of a true discipline, and is reliant on implicit transmission of often tacit expertise and domain knowledge to novices. Analysis is idiosyncratic and anecdotal. The flaw in thinking is that what worked in the past will continue to work with minor tweaks.

Unfortunately, the US IC has few analytic masters left with which to teach new analysts. Over 60 percent of analysts today have been hired since 9/11. The IC conducts precious little systematic self-examination of “lessons-learned” to build on its own rich body of experience so as to focus on the changes essential to address new challenges. Instead we are in a constant state of re-learning what happened last time. After more than 60 years of strategic intelligence analysis, where are the compilers of the analytic lessons for the IC? Where is the documentation, including lessons learned and best practices, that will allow new analysts to avoid the Sisyphean task of their forebears?

We need a disciplined profession of intelligence analysis. The concept of professionalism refers to the exercise of judgment in applying specialized knowledge and experience to the solution of complex, non-linear problem sets. Judgment means the ability to reach conclusions and make probabilistic predictions with incomplete information.

The salient feature of a professional is sustained high levels of thinking and outputs of thought – judgments, anticipation of events and activities, transactions, actions, and behavior. Any profession will also advertise other essential factors: the consistency of its education program; evaluative criteria for membership in the profession; a compilation of lessons learned and responses to problems; a prognosis of future environments; and a coherent strategy for coping with all of the above in a competitive manner.

Intelligence analysis has the problem-solving responsibilities of a profession such as medicine or law. We need a consistent training program with habits and practices that foster excellence. The IC needs to compile insights about analysis. Ask a mid-level intelligence professional, “What are two of the most important lessons the IC learned about analysis in the past 60-plus years?” Then ask for references and sources to substantiate the argument. We ought to get answers. There’s hope. Clear judgment can be nurtured by drawing from rich experiences in national security problem solving, and by introducing new ways of thinking.

Addressing the Analytic Challenge

Here’s a flightpath to improve intelligence analysts’ minds:
1) Build a professional cadre of analysts able to establish developmental and career paths.
2) Create a pipeline of analysts skilled in critical thinking techniques and methodologies.
3) Improve the cognitive performance of intelligence analysts.
4) Capture and document intelligence lessons learned.
5) Implement the analytical improvement effort in education and training programs at entry, mid-career, and senior levels.

The root causes of problems within the intelligence analysis community today are not what is happening inside organizational “boxes,” but rather inadequacies of analytic education, training, and mentorship. Lt Gen David Deptula recognized this and in March 2009 established intelligence analysis as one of the U.S. Air Force intelligence career field’s core competencies. This critical step institutionalized analysis as a key component to address analytic deficiencies.

New technology demands more, not less, analytical prowess. Analytical challenges associated with the Remotely Piloted Aircraft (RPA) technology illustrate these points. Commanders cannot get enough of these systems for both tactical and strategic applications. RPAs offer multiple functionalities across mission sets, from ISR to strike operations, and at a rate that outpaces training rates of operators and analysts. As a result of the increased combat capability delivered such technologies, we need people prepared to analyze and synthesize data feeds, ‘maneuver’ sensors based on sampling rates, and anticipate future activities. All services require quality recommendations for superior decision-making.

Our obsession with technology can undermine this requirement unless we invest in how to create superior effects. Analysts are impeded by inadequate preparation for deep thinking, inadequate technical understanding of platform capabilities, inadequate knowledge of cultural, functional, technical, and operational environment constraints pertaining to the adversary they fight, and inadequate understanding of data to present to operators and policy makers so they can manage risk.

Work-arounds are inherently idiosyncratic and anecdotal, causing institutions to teach the wrong intellectual skills and ignore those which are imperative. This reduces the ability of people to learn at the rapidity that’s required to make sense of complex problems. Avoiding this trap requires investing in the back end of ISR technologies through advanced education and training that prepares warriors to out-think as well as out-execute competitors.

Having properly trained personnel is just as important as multi-million dollar sensors bought for the remotely piloted aircraft. Poorly prepared personnel diminish the quality of intelligence and potential of combat platforms throughout the intelligence cycle of tasking, collection, processing, exploitation, and dissemination.

In terms of cost, congruent education and training for collection managers, targeteers, cyber commandos, ISR, and all-source intelligence analysts can be developed at a fraction of the costs of purchasing additional Sky Warriors, Predators, Reapers, Shadows, Global Hawks, and other unmanned aircraft. Thanks to the inherent high quality of our education system and citizenry, we are capable of high-quality analysis that can save time, money, resources, ordnance, hardware, and lives.

Leadership must come from the four-star level in the person of a champion of the cause. The current Chief of Staff of the Air Force thankfully has been pushing a concept of achieving advantage rather than status quo thinking. Multi-domain operations, combined with technologies of command and control, artificial intelligence, virtual constructive reality simulations, only reinforce the need for human intelligence analytical skills. If we want to orient our technological prowess (which also is hotly contested) for competitive advantage, we are going to need to get serious about the investment in warrior scholars and balance that requirement against being a trained operator.

In the final analysis the intelligence analysts ought to be educated and trained with the same rigor as the personnel engaged in operations, if the team is to be integrated and dependably excellent.

Solution Set

Answers need be neither complex nor risky. They simply require bold vision and commitment to building human intellect with programs that are defined and championed by persevering leadership. A comprehensive approach to educating and training analysts means investing in recruits career professionals beyond the basic and intermediate intelligence curricula that now exist. Why? We have to compete, and win the global competition.

As follow-on to entry-level introductions, the IC must pursue advanced education and training that provides the cognitive tools and knowledge required to win 21st century wars. These are wars of wits in analysis and intelligence. To prevail, we need an institution tailored to generate high-quality, reliable, and dependable intelligence analysis in complex operating environments. We need a highly networked center for information environment strategies (CIES) that functions as a weapons school for intelligence analysts.

To meet the analytical transformation objectives for the joint force, a CIES can serve three primary functions:

First, provide advanced intelligence analysis education for analysts at critical stages of their careers. This educational effort will create a new corps of analysts with higher-order cognitive skills and analytical techniques to handle asymmetric threat environments. A hybrid educational program (resident and distance learning), offered to several distinct audiences, could provide certified and transferable coursework at the undergraduate and graduate levels, in support of services’ and joint education programs. In the long term, the Center may eventually conduct these programs with capable academic partners nationwide so as to ensure broader coverage, increased capacity, and long-term sustainability.

Second, serve as a research hub for developing new processes and technologies, capturing and applying lessons learned, sharing best practices, and igniting meaningful dialogue among all participants in the intelligence analysis profession. This part of the program would directly support, strengthen, and professionalize ISR, command and control, and strategic-thinking capabilities. At the same time, it will serve to advance the intelligence analysis profession throughout the IC.

Finally, and perhaps most significantly, the CIES would serves to foster a broader community of other government, academic, and private sector partners, with the mission of advancing the intelligence analysis profession.

Conclusion

America’s adversaries are in for the long haul. To meet this adaptive range of threats, we need leaders to structure an intellectual and practical center to strengthen human capabilities. In an era of unprecedented technological change to include artificial intelligence, humans need to drive collection, anticipate adversary and the populace’s activities, and retain the ultimate prerogative of strategic decision-making.

We, too, must be in this effort over the long haul to improve ‘how to think,’ and bolster our capability to develop and use intelligence analysis to improve the performance of the human-machine team. A commitment to large-scale student throughput that will meet the needs of the joint force is vital to blending academic excellence with practical results. This vibrant combination has been, and needs to continue to be, our national strength.