The Aga Khan University-Examination Board: Does Pakistan Need One?

Like every other major policy debate in this country, the one about whether or not the Aga Khan University (or Foundation) should be allowed to establish an independent private sector examination board in Pakistan has become the fodder for a new political thunderstorm—or at least one in the making. We have seen it before. Whether it is the Kalabagh Dam, the NFC award, the Model University Ordinance, or the most recent curriculum controversy---entrenched positions arising from religion, politics, ethnicity, provincialism rather than objective analysis of facts end up driving the outcomes of these policy discussions. With several heavy weights already firing up their intellectual and rhetorical cannons, the Aga Khan University-Examination Board (AKU-EB) controversy has all the ingredients of a “perfect storm” that will keep the religio-political pundits all across the country busy and occupied for a while.

Before I delve deeper into this important subject, a brief chronology of accusations and counter accusations is in order. On June 19, 2004, Dr.
Pervez Hoodbhoy—a well-known and respected educationist and activist—wrote a piece for Dawn-Encounter that questioned Qazi Hussain Ahmed’s criticism of the Aga Khan Education Board experiment. Dr. Hoodbhoy’s article was in response to Qazi Sahib’s article on June 10, 2004 in the same newspaper that argued that AKU-EB should not be allowed since it would transfer the control of the country’s educational curriculum into the hands of an entity run by Ismailis (the Aga Khan Foundation) whom Qazi Sahib described as a “religious minority”. Dr. Hoodbhoy—while lamenting the fact that Qazi Sahib chose to paint this debate in religio-political (“us vs. them”, “American conspiracy”, and “state capture by religious minority”) terms—nonetheless defended AKU-EB and downplayed it by stating that AKU-EB is merely designed as an examination body that would supplement the country’s other 24 examination boards. Moreover, he noted that AKU-EB would not have the authority to change the country’s curriculum thus making it relevant to the ongoing curriculum controversy. In a letter-to-the-editor in Dawn (June 25,
2004) again, the Central Information Secretary of Jamaat-e-Islami questioned the policy of letting a “minority group” control the education system of an “ideological” country like Pakistan. With heavily loaded words like these, the picket lines on this debate are clearly drawn. Like the famous curriculum controversy, this “battle” is bound to be fought along religious vs. securlar lines with an utter disregard to the actual merit or demerits of the original proposal itself.

I have been meaning to write on the subject for a while now, primarily because I felt that the secrecy with which AKU-EB’s blueprints and plans were being developed (and implemented) was not in the best interest of this country’s citizens especially our younger generations. Important issues like that of creating a private sector examination board must be objectively analyzed, carefully debated, and executed in a transparent manner. What are the merits and demerits of the Aga Khan University-Examination Board experiment? What is it that a private sector examination board would be able to do that our 24 public sector examination boards cannot? Does Aga Khan University or the Foundation has the credibility of delivering quality educational testing services nationwide? One may note here the experiences of IBA getting into the testing services business and making a mockery of itself. Why would USAID or the Government of Pakistan like to create a new institution where 24 already exist? These and other questions need to be objectively analyzed and examined before a decision can be reached on the issue. As intellectuals, educationists, policymakers and political leaders of this nation, we owe at least that much to the future generations of Pakistan.
The starting point of this debate, to my mind, is to try to objectively evaluate the claims of the AKU-EB itself and assess whether they seem realistic and credible enough and what would they mean for the overall educational establishment as well as our children and future generations. I have been fortunate enough to be privy to a presentation by officials of the proposed AKU-EB that was designed to encourage a national school system to seek affiliation with AKU-EB. This opportunity provided me with some first-hand information on a subject that has otherwise been carried out in quite a bit of secrecy so far. What follows below draws heavily from the AKU-EB presentation made only a couple of months back. I have tried to objectively assess the credibility of AKU-EB’s plans and identify issues that need further clarification from AKU-EB. What is needed perhaps is a transparent debate within the academic community that is aided with objective facts on the issue rather than religious and foreign policy stereotypes.

The presentation describes AKU-Examination Board as having three-fold objectives, namely, promoting improvement in quality of education in schools through reformed examination of SSC and HSSC, training of teachers and development of supplementary learning materials, developing and disseminating expertise in educational assessment and testing, and using advances in computer software to ensure fair, transparent and trustworthy examination processes. As obvious from the above mentioned broad description of goals, AKU-EB, on the one hand clearly aspires for a much greater role in the educational system of Pakistan than merely that of an examination board, and on the other hand aspires to takes the practice of educational testing to levels unseen even in the advanced countries of the world.

But how does AKU-EB propose to achieve its three-fold objectives? The AKU-EB presentation attempts to answer this question in some detail. On the question of improving the quality of education in Pakistani schools, AKU-EB officials start with stating that AKU-EB would begin with making teachers the “managers of learning”. Several statements are made in support of the above—some fairly obvious, even non-trivial, and others not-so-obvious. For example, the presentation claims that AKU-EB would empower the teachers by putting the relevant syllabus in the hands of every teacher, recommend multiple texts and provide teachers with reprographic capabilities, develop subject materials by subject-specialists and distribute them “free of cost”
to every registered student, conduct examinations that favor open-ended answers rather than “right” answers, and legitimize newspaper, TV and internet as tools of knowledge and learning etc.

The benefits of some of the above-mentioned approaches are obvious as they constitute well-established good pedagogical practice elsewhere (e.g.
assessing students based on their understanding of the subject rather than looking for a “right” answer, allowing the use of multiple texts and learning tools etc.) and AKU-EB would add value if it adopts these. Others are not so obvious, especially vis-à-vis the additional value that AKU-EB plans to bring to the table. Do other examination boards or systems not encourage teachers to become managers of learning? Do they not put syllabus in the hands of the teachers or ensure that subject materials are accessible to all registered students? Would AKU-EB undertake the tremendous responsibility of teacher training without which the teachers would not even be able to mark the “open-ended” questions set by the board, what to talk of teaching the students to answer them in the first place. If AKU-EB plans to enter the realm of teacher training as well, how is that part of AKU-EB’s mission going to be funded?

On the question of developing expertise in educational assessment, AKU-EB aims to develop educational “standards” in terms of typical performance expectations at each grade level. Here AKU-EB hints at using a clever trick to smoothen the transition of this system. Each affiliated school would be required to submit a longitudinal history of their SSC grades to AKU-EB.
AKU-EB would work out the proportion of A1, A, B and C grades each affiliated school would get if it did not change its performance standards.
The AKU-EB would then continue to award that proportion of grades to each affiliated school. Examiners would then describe typical performance standards at each grade after which the grade proportions can vary within schools based on improvement in their performance on the set standards.
This is a clever trick that would make it easier for schools and their students to shift from one examination board (e.g. Federal Board) to AKU-EB without having to go through a worsening or artificial improvement in test scores.
However, the devil is in the details and how is this proposal actually implemented remains to be seen. The educational community and the participating schools should insist on detailed documentation of the underlying formulas that would be used to implement this trick—the details of the proposal, as it stands right now, are rather sketchy. Another potential issue of concern for parents and students is the acceptance of AKU-EB within the wider tertiary academic community. The suffering of students graduating from Federal Board and not getting admissions in provincially chartered universities (e.g. University of Karachi) that discriminate in the favor of local examination boards is all too common.

The document also introduces certain other terms and methodologies whose details are so sketchy and ideas so vague that any meaningful analysis of the idea itself or its practical impact on the outcomes of educational assessment exercise cannot be undertaken at this stage. For example, AKU-EB would classify each cognitive subject as knowledge, understanding, or application of skill or idea and would assess student achievement with reference to each of these constructs. The idea sounds like a pedagogical nirvana but would merely classifying everything that students learn as knowledge, understanding, or application actually change something about what they actually learn? Unless, of course, AKU-EB also designs the curriculum and the entire learning experience in a manner that emphasizes a balance between knowledge, understanding, and application, a mere reclassification of already existing curriculum or even adopting new test formats (without changes in curriculum and teaching) would make little difference to the actual learning experience of students.

Another proposal put forth in the document is that of involving subject teachers of affiliated schools to develop teaching and learning support materials that would be collated centrally and distributed to all schools.
Teachers would also be involved in providing detailed feedback to AKU-EB on several issues to ensure improvement of the latter’s assessment methodologies. This is really a beaten horse. The problems of the teaching community, of which ability and motivation are only the tip of the iceberg, run deep. Who doesn’t believe that teachers and teachers alone are key to quality of education in any system, yet the situation on the ground represents a complete lack of motivation and initiative on the part of teachers. In all likelihood, as in the past, the teachers’ community would not appreciate and welcome this additional “burden” and the proposal would end up gathering dust on somebody’s table.

The third and final objective of AKU-EB relates to the development of a fair and trustworthy assessment process leveraging the use of information technology. It is here that the situation gets a little too tricky. AKU-EB is proposing to use Computer Assisted Marking (CAM) to assess “open-end”
“hand written” questions! Presumably, each examiner sees the candidate’s answer to one question in his/her own handwriting and assigns marks to it.
The description here gets too vague and the questions that arise in one’s mind, too many. Would AKU-EB provide laptops to all teachers assigned for marking the exams. In a fully developed and implemented system, hundreds of thousands of students appear for an SSC or HSSC exam and hundreds if not thousands of teachers are called upon to grade their exams. Would students be required to type in their answers? How would the logistical nightmare of taking every student’s answer sheet apart—one question at a time—and putting it into several different computers be accomplished? How would such a complex process be made error-proof? While AKU-EB documents makes it appear as if computer-assisted marking would be a panacea of all errors and biases, it is not quite clear how that would be the case.

This gives rise to an even more fundamental question. Why should such an automated system be put into place in an under-developed country like Pakistan when such systems for assessing “open-ended” questions are not even used in some of the most advanced countries of the world—not even in America! Is this another one of the examples of technology oversell or overkill? One wonders if this is a precursor to another sinister procurement contract designed to make Pakistan a junkyard of a foreign firm’s outdated computer technology that would only waste more money and deliver little additional value? One of the benefits claimed by AKU-EB, apart from hypothesized decrease in error rates, for installing this computer assisted marking technology is the ability of AKU-EB to provide feedback to each school on its strengths and weaknesses of each school’s performance, question by question. If this means providing simple statistics (mean, median, standard deviation etc.) on a particular school’s position vis-à-vis all the schools, that should be do-able without the use of such elaborate system of computer assisted marking. However, if this means feedback of more qualitative nature than mere statistics, the question arises as to why should AKU-EB do this in the first place. Aren’t all good competitive schools do that kind of self-assessment on an on-going basis for their students anyway?

None of the analysis done so far suggests any definitive answers or value judgments. What I have tried to do is to truthfully relate what AKU-EB claims to be planning to do as well as raise questions and identify weaknesses in AKU-EB’s own logic of how they intend to achieve that. Any observer can use the discussion so far and reach his or her own objective conclusion about whether AKU-EB seems like a good idea or not. I’ll leave it up to a competent authority within AKU-EB to answer these questions and objections, one by one, in a convincing manner and for the educationists, policymakers, political leaders—and above all, the parents and students themselves—to assess the validity of these answers.

However, having done an objective analysis of AKU-EB, I would like to leave the readers with a thought. The key question, from a national policy standpoint, that arises from above analysis is as follows: What is it that AKU-EB is offering to do that any other properly endowed and empowered public sector examination board cannot? Why do we need to re-invent the wheel over and over again by creating new institutions and letting the old ones languish away into oblivion? AKU-EB proposes to provide its full-package of services at a cost that would be about five to ten-times higher than the current examination boards on a per student basis. And given the above analysis, it is more than likely that AKU-EB would not be able to deliver good on several of the promised services anyway. Are we, despite our low and lagging levels of societal literacy, ready to put this additional financial burden on the marginal student? Why can’t we work within and improve upon the existing institutions, the 24-odd examination boards, and make them deliver higher quality educational testing services? If USAID must spend money on strengthening the education sector of Pakistan, why shouldn’t that money be spent on the current institutions instead of creating an entirely new one and putting the parents and students in considerable transition turmoil?

The basic and primary education system is the foundation on which a nation is built. It is the earliest and perhaps the most crucial stage of a long process through which a country’s doctors, lawyers, judges, engineers, military, artists, politicians, and scientists—its citizens—are produced.
Not only is basic and primary education important in its own right—for creating an aware and useful citizenry—but also because it provides inputs to subsequent tiers of education and scholarship. Owing to our resource constraints and lack of commitment, the state has failed to make good on the “social contract” in the area of education delivery to all citizens. Several parallel systems of education have sprung up to meet the vacuum left behind by the state’s failure. These range from the notorious madrassas on the one hand to the elitist O/A levels, IB etc. on the other. In between are the public school system and the local private schools that offer Matriculation and Intermediate systems of educational assessment at affordable prices.
AKU-EB would represent yet another tier in this already multi-tiered and fragmented education system. Some have called it “Educational Apartheid”. I believe it is plain bad policy that would only make us a nation more and more fragmented from within.