Monday, November 25, 2013

Wicked Problems


Ajit Chaudhuri

‘The specific problems we face cannot be solved using the same patterns of thought that were used to create them.’ Einstein

Most of us know that the Indian education system is in a mess! At the higher end, my experience at the London School of Economics in 2001 was that, while British higher education needed Indians (we are among the few willing to pay for an education), it has to factor in the fact that the only real skill we have is that of rote learning. A comment I heard during my more recent visit to a tech hub in the US, was that, if a computer game required a character to jump two summersaults instead of three, they would call in the Indians to make the change. But the decision to change would be taken by an American. And, from my stints on interview panels gauging entry into management school, it was clear that many young engineers had fewer skills than my neighbours’ servants and drivers, and earned correspondingly lower salaries.

And yet, it is the problems at the other end that I have grappled with over the years – the difficulties that most Indian children face in getting access to a primary school, staying in them, and gaining an education – reflected in the pathetic state of many schools in the public education system, the absconding teachers, the large student to teacher ratios, and the multiple age groups in the same room being taught simultaneously by a single teacher. I don’t need to throw statistics around to support my case, our literacy rates are well known, as is the fact that many government teachers utilize their time to run shops and money-lending businesses. The various NGOs that I have worked in and supported have tried many things – all sorts of alternatives to formal schools, as well as bridge courses, remedial centres and placing additional teachers to get formal schools to work better. I used to think; if only we could get schools up and running in every village and locality, get teachers to actually teach, and get parents to send their children to school instead of into carpet factories or off to rear goats, we could address this problem. Why did I grapple? Because it is a problem worth addressing – no country without vast oil reserves has transformed itself from poor to rich without decentralized governance and universal elementary education, and we don’t have oil. And if we do not address it, we will not derive the advantages of demographic dividend that a young population is supposed to bring to an economy.

It was therefore a depressing experience to attend a recent development conference at Glocal University in Saharanpur (UP, on 25th and 26th October 2013) that focussed upon health and education in UP, and to sit in on two excellent presentations on the state of education in the country. The first was by Prof. Willima Wadhwa of the University of California and of the Indian Statistical Institute, who explored myths and realities regarding the right to education. The second was by Prof. Pankaj Jain of Gyanshala, who talked about education policy options. Prof. Jain, may I add, had been a professor at IRMA in the 1980s and I had been among his less illustrious students. Prof. Wadhwa pointed to the fact that children in grade 5 did not have the competencies of those required in grade 2 (she is the head of Pratham’s ASER programme that conducts the tests that make these observations). She also suggested that there is no evidence for the common presumption that private schools are better than government schools in terms of learning outcomes – the differences are attributable to selection biases (private schools attract those from better-off homes with better-educated parents, and do not give admission to those they deem unsuitable) and non-school factors.

Prof. Jain exploded some more myths – he said that everything that can possibly be tried has already been tried. Teacher education norms in India match world standards, teacher-training and support norms are considerably higher than world standards, and India is already a world leader in providing incentives (free meals, books, etc.) for children to attend schools.

He said that a major problem in India is that teachers are heavily overpaid, and I am going to expound on this because the most quarrelsome of my siblings is a teacher, as is my best friend (and both are on this reading list), and I need to protect myself. In an OECD country, a teacher with 15 years of experience earns at the per capita income of the country, whereas in India s/he earns at seven times the per capita income. Great, you would say, teachers are critical elements of society and nurture future generations, what is wrong in respecting them by paying them well? Well, most of the cost of education is teachers’ salaries, and if these are high the natural outcome is fewer teachers or else state bankruptcy. The public education system cannot therefore implement any civilized norm on student to teacher ratios simply because the number of teachers required to maintain such a norm would drive the country bankrupt. The right to education act, which requires public, private and informal schooling institutions to maintain acceptable student to teacher ratios and teachers’ salaries at par with heavily overpaid and underworked government teachers is an example of wishy-washy policy making. It is simply not possible to meet both these requirements simultaneously. It is also not possible for government teachers’ salaries to be reigned in, or allowed to inflate away; they have powerful public sector unions to protect their interests, and are also a huge source of bribes into the political system – if anything, the problem will only get worse with the new pay commission for public servants.

This brings me to my topic for the month – problems that defy solution, problems wherein scientific-rational approaches cannot be applied because of lack of a clear problem definition, incomplete, contradictory and changing requirements, and differing perspectives of stakeholders, problems such as ‘how to withdraw from Afghanistan’, ‘how to sort out Indian education’ and ‘how to complete a PhD in three years and remain sane and solvent’. 

A 1973 paper by Rittel and Webber termed these as ‘wicked problems’, with ‘wicked’ meaning resistant to solution rather than anything evil, and contrasted them with relatively tame ‘solvable’ problems in mathematics, chess and puzzle solving. They said that complex interdependencies within wicked problems result in efforts to solve one aspect of it revealing or creating other problems. They also said that ‘the search for scientific bases for confronting such problems is bound to fail because they cannot be definitively described, and as the policies that respond to these problems cannot be meaningfully correct or false it makes no sense to talk about optimal solutions’.

The issue of global warming brought about a new superlative category, that of the ‘super wicked problem’ (yes, I know, it sounds like the buses on Indian roads, with categories like deluxe, super deluxe, super luxury, etc., applied liberally to old clangers), with the additional characteristics of time running out, no central authority, and those seeking to solve the problem also causing it.

And finally, we have a situation where every problem interacts with other problems and is therefore part of a set of inter-related problems, a system of problems, or what is termed as a ‘mess’. How do we know that we are in a mess? According to Robert Horn, a mess has the following characteristics –

1.    There is no unique, ‘correct’ view of the problem – there are differing views and contradictory solutions.

2.    Most of the problems identified are connected to other problems.

3.    Data is uncertain and missing.

4.    There are multiple value conflicts at play, as also ideological, political, economic and cultural constraints.

5.    There is often a-logical, illogical or multi-valued thinking around the problems, with uncertainty, ambiguity, and great resistance to change.

6.    There are numerous possible intervention points.

7.    The problem solvers are out of contact with the problems and their potential solutions.

I would like to conclude with some thoughts for the policy advocacy industry within the Indian development sector, which often complains that Indian policy makers are impervious to their pleas and suggestions. It may be that they see you as just another donor-funded special interest group with no accountability and a limited mandate, and treat your views accordingly. Alternatively, it may be that your clear cut and articulate solutions to well defined problems do not recognize the chain of complex interdependencies between problems, and therefore do not make sense to those who have to live with the outcomes of policy suggestions. Because, after all, the reality of development is that most issues fall within the categories of wicked problems, super-wicked problems and messes, whether in India or anywhere else in the world.

Some additional reading for those with an interest in wicked problems:

Horn, Robert; 2001; Knowledge Mapping for Complex Social Messes; presentation to “Foundations in the Knowledge Economy” at the David and Lucille Packard Foundation on 16th July

Levin, K., B. Cashore, S. Bernstein, G. Auld; 2012; Overcoming the Tragedy of Super Wicked Problems: Constraining Our Future Selves to Ameliorate Global Climate Change; Policy Sciences 45(2), 123-152

Rittel, Horst W.J and Melvin M. Webber; 1973; Dilemmas in a General Theory of Planning; Policy Sciences 4(2), 155-169


Ajit Chaudhuri said...

Super reading... Thank you

I always knew that I was doing the right thing but not seemingly getting where I wanted to... now I have a reason.

I have often referred to development problems as complex and multivariate comparing them to a marketing situation where essentially one deals with the 4 P's. And used that argument to state that we actually needed the best brains in development since the problems were complex. And lamented that we often get only the left-overs while the super brains go and play with the 4 P's.

But yes these are wicked problems. And we are constantly only trying to solve the most recent layering to the original problem. So our solutions might address the top layer but rarely lend themselves to address the core. Recall he old development sector discussion on the difference between the symptom and the disease.

Some of us have to attack the core disease - the multiple drug therapy to cure TB and therefore relieve the cough. And most of our problems have become like TB require a combination of efforts and approaches and sadly are also taking the shape of drug resistant TB and therefore require more potent drugs and more potent approaches. A unidimensional approach that specifically addresses one issue is not going to work anymore.

The education issue is somewhat like that. While Pankaj Jain may say that everything has been tried I believe he is also referring to trying to solve the top layers of the problems. And while I would like to complement him for creative thinking I am not too impressed with his solution. His approach also addresses the top layer and provides a solution that I call 'poor solutions for poor people'. The core still does not get touched. And the core needs to. The issue with teachers when described as an economic argument makes sense but I still feel that the solution cannot be without them. The core is there. And resolving this wicked problem does require us to attack there. And perhaps evolve a really hard hitting attack.

I still have faith in the development sector in finding solutions. For if they don't I can only see a disastrous outcome. In the case of education, of first the private sector taking over and select segments of society gaining education to strengthen themselves and the rest developing as the 'service' sector catering to the needs of the affluent. And then eventually the large number of people left behind revolting to the system that's taken over. Of course there is the possibility that our democratic politics will not allow that revolt to happen as there will always be the NREGA and free food that will keep them quiet... like IRDP, and SGSY, and other schemes of the government have always done. But I do hope that the revolt when it has to happen will.

Decentralization in essence is perhaps the one way to address the core. To change the way we look at the problem. To use that oft used training activity - making a line smaller without actually rubbing it. I think the reason we are not letting effective decentralization happen is because the politics of this country is happy 'acting' out solutions that address the top layer of the problem. And seem to be very busy and concerned doing so.

Being able to define the problem is an important element of problem solving. Being able to understand them as 'wicked' problems will mean approaching them differently. And not letting them remain wicked problems for much longer.
I don't know if I sound overoptimistic - but I feel the struggle is far from over.

Thanks Ajit for sharing


Sachin Sachdeva

Ajit Chaudhuri said...

Hi Ajit,

Thank you for this – it’s a great, provocative read.

You make some excellent points - and I, for one, had no idea that our teachers are one of the most highly paid in the world (and the inherent problems that this involves)!

But what I found really interesting – depressing, though it was – was your explaining so lucidly the notion of “wicked” problems and the “interdependecies” factor.

But then, does one just have to accept that there are no solutions, so why try to fix the problem? Or does one chip away at different aspects of the problem and in the belief that even a small difference is worth the effort?

Oh well...

Arundhati Gupta

Ajit Chaudhuri said...

Nice one -well written even though no travel or food involved-just one thing though-assuming that india per capita income is about 800 usd that's abt 48 k inr...are you saying govt teachers earn 3.5 lacs a year?

Hi Viji and assuming a per capita income of USD 800, then yes, a government teacher with 15 years in the system makes 3.36 lakhs a year. I think this is understated somewhat, India's per capita GDP is at USD 1,500 today.


Wow man that is an eye opener. ...

Ajit Chaudhuri said...

Loved the last para. In my case, the example that best fits your conclusion is our advocacy on vaccines. A simple clear cut solution to child mortality, but the government doesn't buy......super wicked problem!

Dr. A. Khan

Ajit Chaudhuri said...

Thoroughly enjoyed reading this one!!!! For a little while your articles had become a little heavy with literary references (except of course, the ones pertaining to food!!!!)

Radhika Roy

Ajit Chaudhuri said...

Very interesting.

I hope you will find a way to influence matters for the good.

JL Pasricha

Ajit Chaudhuri said...

Dear Ajit,

Thanks for introducing me to the concept of 'wicked problems'. But I expected some way out (to
the 'education standards' problem) in your article, just as I had expected a unique converging
solution for the Case in my first class in MAC course at IRMA (later Prof. Kanitkar
enlightened me by saying that is not how the Cases are concluded in a class)!!

best wishes,

Ajit Chaudhuri said...

Dear Ajit,

First of all, please consider yourself lucky if you were able to meet and interact with Prof. Jain. It has been one of the high points of my life in IRMA to listen and interact with him, he has more intuitive understanding of challenges of education in India than any education 'experts' I have met. As an aside, he was a frequent visitor to IRMA during the tenure of previous Director, I think the present regime does not consider him necessary - I'm not surprised given the current regime's grasp of problems.

Second, regarding the wicked problem that you have posed. I had first hand experience of this problem when we were building up the informal primary school in Baddi slum of Himachal. The greatest challenge was to get teachers for this school. We tried out two alternatives. The first failed. The second seems to be succeeding till now.

The first was to get 'qualified' teachers, by this we were looking out for teachers who had at least Master's qualifications but still were prepared to work in our primary school because they were stuck up in Baddi because of their husband's profession. Over time, we realised that the qualified teacher's were over qualified to teach children of age group under 10 - they were too methodical and formal. I realised painfully (over time) that we need not have Master or Bachelor qualification to teach primary school children.

So, we failed first and then tried out our second option. In this round of experimentation, we were only looking for Matriculate pass as teacher's. Now the challenge in teacher recruitment was different, earlier the challenge was to find someone with Master's or Bachelor's willing to work for Rs 3500 per month. Now the challenge was to interview the numerous Matriculate pass's and find out who liked interacting with children. Through experimentation, we have realised that attaching the second criteria (liking interaction with children, I call them Natural teacher's) along with Matriculate pass allows us to have good enough teacher's. These teacher's on their own find mechanisms to deal with children under age group 10 and at the same time are content with Rs 3500 because they know that they are only Matriculate pass.

Basically, I have come to the tentative conclusion that teacher's at any level ONLY requires teacher's 1)of the next higher level 2)with real interest in mixing with the just below level. This may be a thumb's rule to slice through problem of overpaid teacher's who have only naam-ke-vaste higher qualification because in the first place they are products of the same education system like you and me.

I know this is a long narration, but I believe wicked problem's require complicated answer's rather that one sentence repartee's.

Hope you are doing good.

Joseph Kalasserry

Ajit Chaudhuri said...

Sorry, there was an error in previous message. The repaired sentence reads as -

'Basically, I have come to the tentative conclusion that STUDENT's at any level ONLY requires teacher's 1) of the next higher level 2) with real interest in mixing with the just below level.'

Joseph Kalasseri

Ajit Chaudhuri said...

Dear Ajit,

Nice topic. Nice reading. Thanks.

PL Liandinga

Ajit Chaudhuri said...


Not as happy as I normally am after reading your two pages!!

Don't tell me you are still in a age where you think "What Teacher says must be true"!

Just becoz the presenter says there is mess, doesn't mean there is a mess! Actually -being head of Pratham, did the presenter have any other choice to say? And by the by, one of the radical answer to the "Pratham" problem is change the syllabus itself!! Keep that (syllabus) that the millions can manage and not that where only the fistful can manage.

Your Robert Born is actually talking of Democracy that too democracy plural society ( and not mess as he calls it), ​ where there are problems with multiple solutions ( see Argumentative Indians by A. Sen for detailed argument on this) which is real and not "Mess".

More later.


Ajit Chaudhuri said...

This makes so much sense... especially now, when the clamour for change is getting noisier

When you are young and driven, you believe that every problem has a rational solution. And that all it takes is the right perspective, the right priorities, the right policies, the right institutions, the right people and of course, political will. And you always believe that you are 'right.' So if I had read about wicked problems twenty years ago, I would have simply sniggered. Now that I am older, have seen more, have read more, and have thought more, there is nothing to snigger about any more. Every major problem seems wicked, and we are surrounded by messes. Which leads to both sadness and paralysis.

Do you see any way out?

Daman Singh

Anirban Mukerji said...

very interesting