Showing posts with label Metrics. Show all posts
Showing posts with label Metrics. Show all posts

Thursday, February 9, 2012

Understanding the kind of war on which we are embarked, etc.

The enemy in Afghanistan is resorting to IED attacks and other tactics that endanger civilians with increasing frequency, says the UN. That means we're winning, says LTG Scaparotti.

Spencer wrote about this yesterday; he was the one who asked the general how to square increasing civilian casualties with claims of coalition pogress. Here was Scaparotti's curious reply:
Their – you know, what little bit [civilian casualty numbers are] up – I would tell you that across the battlefield, when you look at Afghanistan, for instance, there is a freedom of action that [enemy forces] have, and it's particular places. But the freedom of action they show today is increasingly in IEDs and suicide bombing.  They don't have the capability to take us on directly.  In fact, they've changed their TTP because they're unable to do that with either us or the ANSF.  So I don't know that I would say it's an increase freedom of action.  I think it's actually reduced, and it's pushed them into a certain TTP, which isn't ideal. 
Now, in doing that, they've increased the use of suicide bombers, for instance.  And if I recall right – correctly – it was about – it went up about 80 percent. 
And those caused civilian casualties.  One in three of their IEDs -- or one in three of the civilians injured, Afghan civilians, were caused by, you know, enemy IEDs.   
So, you know, what I would focus on is the fact that you've got an enemy who has stated that he is concerned about the people, that he doesn't want to harm the people in his actions, and yet over these years you've seen a steady increase in that happening.  On the other hand, we work very, very hard to drive down the – you know, the civilian casualties.  It did go down 4 percent this year.  And we'll continue to try and drive that down as well.
So far as I can tell, this is an answer that is based wholly in an effort to support information operations objectives—that is, to communicate to the Afghan people that the enemy is a liar and does not care about their welfare. (I find it difficult to believe, though, that any significant portion of the Afghan populace is basing its judgment about whether to support the government or the insurgency on such pronouncements from ISAF.) I can't detect any operational logic in this statement.

Proponents of the counterinsurgency campaign in Afghanistan have consistently indicated that the level of civilian casualties inflicted by the enemy could serve as a useful metric for the performance of ISAF's mission. By their logic, the more effective coalition forces proved to be in keeping the enemy from killing civilians, the more likely those civilians would be to support the coalition's and the government's efforts to stamp out the insurgency. Even if protection of civilians is not an operational priority, there is an inescapable truth at the heart of this: if the enemy cannot operate, that constitutes some form of success.

LTG Scaparotti has turned this logic on its head by suggesting that the enemy's refusal to operate in ways that make him more prone to destruction by coalition forces constitutes progress for ISAF. This is exactly backwards. If coalition forces and the ANSF were proving capable of stanching IED attacks and suicide bombings, restricting the Taliban's opportunity and degrading his capability to terrorize individuals and plausibly challenge the government's claim to provide security to its people, then the enemy would be faced with a choice between two paths to continue his resistance: forego violence and enter the political process, or escalate violence and confront security forces directly. If the enemy cannot get at the people, he has to fight the army. That he chooses not to do so clearly demonstrates the very opposite of what LTG Scaparotti contends: the enemy is not desperate, he is not running out of options, he does not feel as though his freedom of action is reduced.

The general plainly recognizes this when he says that the enemy has been "pushed ... into a certain TTP, which isn't ideal." The enemy has adopted methods that render him less vulnerable to destruction by massed firepower, which isn't ideal. I completely agree.

Reliance on irregular tactics and terrorism allows the enemy to sustain his campaign against the Afghan government and its impermanent international enablers while reducing his exposure to their most effective weapons. It is inconceivable to me that there are people who still believe, at this late stage in the war, that the enemy's "refusal to come out and fight" constitutes success for the counterinsurgent.

The enemy doesn't need to fight us—he just needs to keep blowing people up.

Friday, March 5, 2010

Hot quant on qual action

Over at Abu Muqawama last week, Ex posted a quantitative analysis manifesto that he thinks quantitative analysts in security studies should follow - a post that has generated a moderate amount of buzz around the interwebs. I'm not going to through each of his six points individually, but I encourage you to read the whole post. Suffice it to say and not surprisingly, this has really pissed off a lot of the IR community, with interesting commentary here, here, here, here, and here. I didn't look too much, but here is a supportive post on Ex's position and I'm sure there are others.

I certainly appreciate Ex's position on this, there are few things more irritating than some social scientist who creates models completely at odds with reality. So my interpretation of Ex's manifesto is that social scientists who employ quantitative measures should use humility in describing their systems and recognize and acknowledge the downsides of their models. On the other hand, I understand why these IR folks are upset - specifically about the last three points (which strike me as unnecessary swipes at social scientists, but hey, flippant statements are a hallmark of blogging).

I've written on a related topic before (metrics specifically), and having an undergraduate in math, I firmly believe there are uses for quantitative analysis in security studies. As long as that analysis framed with qualitative analysis to better describe these complex human endeavors. In fact, I would argue most social scientists would agree with this. But I think the same goes for qualitative analysts - while logic and common sense can often win a debate, those two things could often simply be conventional wisdom. Qualitative analysis is much more persuasive when backed by quantitative analysis, otherwise it's pretty much somebody's opinion.

This whole argument strikes me as silly. Both factions need to (and many do) understand and state the limitations of their chosen methods of analysis. I firmly believe bad analysis is and will be identified for what it is, no matter if the analysis uses words or graphs. As for me, I like both.

Monday, August 31, 2009

More on McChrystal's Metrics

The Washington Post yesterday reported some vague details of the metrics to be included in GEN McChrystal's Afghanistan assessment, which should hit the White House some time in the very near future. Here's Karen DeYoung:

The White House has assembled a list of about 50 measurements to gauge progress in Afghanistan and Pakistan as it tries to calm rising public and congressional anxiety about its war strategy.

Administration officials are conducting what one called a "test run" of the metrics, comparing current numbers in a range of categories -- including newly trained Afghan army recruits, Pakistani counterinsurgency missions and on-time delivery of promised U.S. resources -- with baselines set earlier in the year. The results will be used to fine-tune the list before it is presented to Congress by Sept. 24.

Lawmakers set that deadline in the spring as a condition for approving additional war funding, holding President Obama to his promise of "clear benchmarks" and no "blank check."

As anyone who's been paying attention will know by now, the military is big on "metrics." I've yet to see a really detailed explanation of the way this works inside DoD, though, probably because it would be too boring. But here's a quick primer.

When doing an assessment of any activity, the military looks at two things: Measures of Performance (MOPs) and Measures of Effectiveness (MOEs). The former category usually gives you raw numbers: how many personnel trained, how many pieces of gear supplied, how many patrols run, how many enemy KIA. If you think about it, the term is pretty self-explanatory: you're measuring performance in whatever activity you're engaged in, but measuring that performance in a quanititative way. MOEs are concerned with how much the thing you're doing has helped to accomplish your objectives. In the case of Afghanistan, that would be something like whether or not the ANA units you've trained meet a certain capability level, or whether the small arms you've supplied have increased marksmanship scores, or whether the increased ANP presence in a certain village has positively impacted voter turnout.

Military and development aid to Pakistan provides perhaps an even simpler way to demonstrate the distinction: a MOP would be "military grant aid provided via Foreign Military Finance, Pakistan Counterinsurgency Capability Funds, Coalition Support Funds, etc., in dollars," and the associated MOE would be something along the lines of "Pakistani counterinsurgency missions conducted in the NWFP."

The whole concept is sort of confusing, because the distinction between a MOP and a MOE is only meaningful if we understand what it is we're trying to do. Just tallying up numbers, obviously, is meaningless. So if the mission is "kill the enemy," and we want to understand the utility of firing weapons at the enemy to accomplish this mission, then we can use rounds fired by coalition forces as an MOP, and number of enemy dead as an MOE. But if we're looking at the bigger picture (and generally, that's what we want to do), then something like a body count is always going to be a MOP. The MOE, then, would be related to a next-order effect, like "number of districts in which coalition forces can operate without taking fire or being engaged by IEDs."

All of which is to say that without understanding the context -- what the mission is, and which "metrics" are MOPs versus MOEs -- a list of a whole bunch of things like this is kind of meaningless. I don't suppose I should say "meaningless," because it does illustrate areas of focus and emphasis to some extent. The three examples listed in the linked article -- "newly trained Afghan army recruits, Pakistani counterinsurgency missions and on-time delivery of promised U.S. resources" -- shows a focus on building partner capacity, on training and equipping our allies and the host nation government, rather than on U.S. combat operations.

Understanding HN capabilities is a good thing, as is a focus on improving them. I'm going to express the same reservation here that I have in the past, though: without a sufficient level of security and governmental reform, efforts to empower the Afghan National Security Forces look more than a little Sisyphian. And there's precedent: in the pre-Surge days in Iraq, the days of the Together Forward operations, everything coming out of MNF-I and CENTCOM indicated a focus on training up the Iraqis so we could get out. The war was hugely unpopular, and as the saying went about Iraqi security forces, "these guys are our exit strategy." The Surge was not just about a troop increase or a change in tactical and operational approach to emphasize counterinsurgency, but the institutional realization that trying to get the Iraqis ready to do the job themselves, at that point in the war, was a near-complete waste of time and effort. So the options were to accept massive risk of an unacceptable reverse once U.S. forces departed, or commit materially to trying to change those circumstances. There's a reason Ricks' book was called "The Gamble": in early 2007, the idea of throwing more U.S. troops into the fight seemed like a Hail Mary, and perhaps it only could've been considered by a President who was so wildly unpopular.

And so it's starting to look like we're in a similar spot in Afghanistan, that 2010 there may look a lot like 2007 in Iraq: everyone seems to understand by now that we're going to need more U.S. troops in country before the security situation approaches anything near acceptability, never mind an environment where shifting the burden to Afghan forces is a plausibly achievable "exit strategy." So now we're back to the politics. Recent polls show that somewhere on the order of 50 percent of Americans now oppose the war. Casualties have gone up this year, and no one wants to see that trend line continue in the direction it's going. The President has an ambitious domestic agenda, and spending a ton of money in South Asia isn't going to help with that.

So what's the answer? Well, Kilcullen was probably right: "We'll fight for two years and then a successful transition, or we'll fight for two years and we'll lose and go home." Notable for its absence is any mention of "winning."

Does a "successful transition" look possible in the next two years? I'm not optimistic. But if we look at the metrics that GEN McChrystal's team has identified, then it certainly seems like "transition" has become the priority. And while that's probably best for America, I'm not sure it bodes well for Afghanistan.

Monday, July 6, 2009

Metrics: Telling the Story, Not Driving Policy

Policy wonks everywhere are mourning the passing of Robert McNamara. Whatever one's opinion of him, he was certainly one of the most interesting Cabinet officers ever. The Post has an excellent piece on his legacy. There are many take-aways for today's conflicts from this article, but these two paragraphs strike me as extremely relevant:

"On his first visit to South Vietnam in 1962, before most Americans had heard of the place and before the involvement of American combat forces, McNamara said that "every quantitative measurement we have shows we're winning this war."

It was a statement often quoted by his critics in later years, because it seemed to encapsulate the fallacy of his approach. American troops did prevail in many of the big battles, and the United States did win the war by every statistical measurement on the Pentagon charts that McNamara so admired. But the numbers -- even the few that were accurate -- had little to do with the political reality on the ground."

Even by ignoring the myth that the only metric used in Vietnam was "body counts," McNamara's story exemplifies how relying too much on metrics distorts the reality of the conflict in which you are engaged. Further, most metrics will be defined in a manner that permits political exploitation - defined to ensure success or defined so vaguely as to be meaningless. As an example, the Iraq "benchmarks" fit the bill for both of these circumstances.

I am a big fan of metrics. The quantitative measurement of conflict is extraordinarily intriguing and useful in helping paint the picture on the success of operations. They do not, however, provide exact realities - especially with regard to gauging the sympathies of the population. GEN McChrystal's command review will no doubt devise metrics that will demonstrate ISAF's success, not dishonestly but due to the nature of the subject. We should track and measure these metrics to gain a sense on how things are going, but remember: they have little to do with the political reality on the ground.