AL AIN—A small band of dedicated university administrators is trying to do what many have said could not be done—create a culture of data collection at Arab universities.
There’s been plenty of hand wringing and grand speeches about improving Arab higher education. Many policymakers agree that Arab universities need to get better to give Arab economies and Arab minds and souls a lift.
But that will never happen unless Arab universities get a clear understanding of where they are now and fix precise goals for where they want to go.
Doing that takes data. Like someone climbing a cliff, Arab universities need to drive a piton into one clearly defined place in the rock, eye a place above that position, hammer a piton into another clearly-defined place and then pull themselves up.
Those who are best known as “institutional researchers” can help out in this slow climb. They create the definitions for data, collect data, interpret them and distribute them. The researchers talk faculty members into supplying data and cajole students into taking extra tests. They interpret data for administrators who sometimes have short attention spans.
The institutional researchers operate by persuasion and not by power, but their importance is rising in the world of Arab higher education.
“No one can argue about the importance of institutional research,” said Ali Rashid Al Noaimi, the vice chancellor of the United Arab Emirates University last week at a meeting of MENA-AIR, the Association of Institutional Research. “Our day-to-day decisions depend on you.”
Some examples of what institutional research can do were on display at the association’s annual meeting, held this year here at the United Arab Emirates University campus. The meeting drew about 225 attendees, some of whom are just beginning to do such basic tasks as counting students while others are veterans who do deep data dives.
Researchers from Sultan Qaboos University, in Oman, spoke about how they had tried to measure whether the university was teaching students how to think more critically. Like many other countries, Oman is trying to emphasize entrepreneurship and innovation in higher education and teaching critical thinking is crucial to that effort. (Whether it’s possible to measure critical thinking is, of course, controversial in and of itself.)
The researchers tested 1,725 students upon admission in 2010 using the “California Critical Thinking Skills Test,” one of the more popular measures of core reasoning skills. Students could choose if they wanted to take the test in English or Arabic. The researchers followed up in 2013 by asking some professors to administer the test in their classes and got usable results for 206 students, who got $50 gift cards for participating. The researchers also tried to motivate the students by telling them that the tests were “their chance to give us a grade on how well we have done.”
The researchers found improvement in critical thinking that was “significant but not huge.”
The researchers speculated that some students might not be taking the tests seriously and others looking at the results wondered whether the faculty members themselves were critical thinkers or effective at teaching it.
In another effort at measuring whether or not universities are making a difference, two researchers from Abu Dhabi spoke about a “graduate destination survey” they were in the middle of. They are trying to reach about 5,800 graduates of higher-education institutions in Abu Dhabi to inquire about their employment situation, their educational experience and if they were going on to further study.
The researchers were chasing graduates down by phone calls and texts, trying to motivate them to participate with a random drawing for iPad Airs. The survey began on February 6 and the researchers reported at the meeting last week that they had already contacted 1,713 graduates and only 41 had said they would not participate. While the results are not in, the survey already seems to show that, with adequate resources, such surveys are possible.
Those attending the conference also listened to a sometimes rancorous discussion between three companies that rank universities: U.S. News and World Report, QS and Times Higher Education. U.S. News recently announced that it was going to do some MENA-focused rankings, but clearly its competitors are not going to easily cede it the territory. A QS representative told attendees to “watch this space”—hinting at a product to come—and editor Phil Baty said via an online video call that he was eager “to have a meaningful consultation with universities in the Middle East.”
All of the rankings researchers agreed that measuring the quality of teaching is difficult—in the MENA region or elsewhere. Faculty-to-student ratios are sometimes used as a proxy. But, as Baty said, “The number of waiters in a restaurant doesn’t tell you how good the food is.” (See related article “Are Rankings Relevant to the Arab World?”)
Rankings could be interpreted as attempts at boiling down institutional research into a single number: They are appealing to students and parents who are puzzled about the wide variety of choices they sometimes have.
The next step forward from data collection is data sharing—among institutions and with the public. The American University of Beirut sets an example by publishing data from what is known as the “common data set” on its website.
In five or ten years, Arab students and their parents may have more numbers to base their decisions on and perhaps the data will help some Arab universities keep climbing up the quality cliff.
David Wheeler is the editor of Al-Fanar Media.