AMMAN—Mohammed wants to build a rectangular patio five meters in length and three meters wide. He needs 81 bricks per square meter. How many bricks does Mohammed need?
Since the turn of the millennium, teenage students around the world have been asked questions like this as part of the Programme for International Student Assessment—better known as the PISA test, which is administered every three years.
The exam seeks to gauge student performance in mathematics, science and reading. One of the exam’s goals is to make it possible to compare the results of the countries that take part. The last round of testing took place in 2012 with over half a million students quizzed throughout 65 countries and territories. Critics of the test doubt its worth and question its accuracy, but advocates say the results help politicians to execute data-driven education policies. (See accompanying story: Why Aren’t Jordanian Kids In School?)
Controversy over the PISA test, of course, is only part of a broader global debate about the value of educational tests. Some countries are swinging toward widespread testing in the belief that it holds educators accountable. Others are backpedaling from testing in the belief that “teaching to the test” sucks creativity out of the classroom.
When it comes to PISA, enthusiasts say it’s more than a stick to beat educators about the head: The tests can be used as a lever to lift up education. “All tests have issues, but I think international benchmarking is a good thing and this is the only real test that does it,” says Isobel Coleman, a senior fellow at the Council on Foreign Relations.
But she also recommends that policy-makers should dig into the data and not just focus on topline results. (For this article, Al-Fanar Media conducted its own analysis of PISA data to look for examples of statistically significant variation that might be useful to policy-makers.)
Most education professionals, of course, say PISA results should not be used in isolation. “PISA measures results along tangible metrics, but that only tells you part of the story,” says Safwan M. Masri, executive vice president for Global Centers and Global Development at Columbia University and the director of the Columbia Global Centers | Middle East. Politicians should instead pay more attention to how students are or aren’t prepared to be “agents of change” to engage with democratic transitions, says Masri.
More Arab countries are enrolling in the PISA tests. Tunisia was the first in 2003. Jordan, the United Arab Emirates and Qatar had all signed up by 2012. Lebanese students will take the exam for the first time in 2015 and Algeria also is set to take part in the future. All of them, of course, are extraordinarily different countries in education spending, culture and language.
Predictably, perhaps, Arab countries perform poorly compared to richer nations like Japan and Finland. “The results we do have are deeply disturbing,” says Andreas Schleicher, PISA’s chief statistician.
The results should act as a wake up call and a road map for education policy makers across the region, adds Schleicher. “Many of these countries want bright shining universities, yet they don’t seem too worried about how they produce the students to fill them.”
Some signs of improvements exist and the Arab world’s first participant in the tests has used them to leapfrog forward. Tunisia came in last for reading in 2003. In 2012, it placed ahead of nine other countries.
This early bounce happens often. “Some of the countries with the lowest results in the initial PISA assessments have seen the fastest improvements,” says Schleicher.
Policy makers in Tunisia say the test has been useful. The country has not made sweeping changes based on results, an official says, but has made some adjustments. “We added an extra hour per week of mathematics and introduced physics lessons to primary schools,” says the Ministry of Education’s director of evaluation, Mhirsi Chadia.
PISA’s accompanying surveys make it different from other standardized tests. Students are asked about their background and home life—what their parents do for a living, how many books are in the home and whether they are happy in school. Educators also fill out their own survey, which among other things tells PISA about the location of the school—rural or urban—and where the funding comes from—public or private.
That allows PISA to match up student performance with socio-economic factors. “The survey is very important,” says Schleicher. “It looks at the influence of the students themselves, parents and teachers on performance. This triangulation is critical.”
The survey data is all made public, which means that just about anyone with an Internet connection can see, for instance, how well working-class rural students are doing compared to a country’s urban elite and fellow students elsewhere in the world.
For the four Arab countries that took the test in 2012, the statistics highlight a few areas where investments could yield improvements.
Students report being happy in school, but they are out of school a lot. Typically, students in the Arab countries tested (excluding Qatar) were more likely to say they’re happy at school than students in Western countries. This may be good news, but unfortunately it also coincides with higher rates of students missing classes.
The problem is especially chronic in Jordan where an impressive 84 percent of students say they’re content at school, yet a staggering 57 percent confessed missing either a whole day of school or some classes at least once during the last two weeks. This rate is more than double the average OECD country.
“There are a number of factors at play here,” says Coleman. She adds that it’s unlikely to be a question of child labor—especially not in Qatar, one of the world’s richest countries. It’s more likely a knock-on effect of teachers not showing up. “They’re called phantom teachers. They’re on the payroll but are paid very little and so don’t feel like they have to show up. It’s especially an issue in rural areas.”
Tunisia also has this problem. “Teachers are often on strike,” says Chadia, “There is of course a link between teachers not showing up and children not doing so either.”
The autonomy of schools may play a role in test results. If teachers and principals have more control of the curriculum they may have more success with students, or so goes one theory. In Jordan, less than seven percent of students go to schools where the principals and teachers report having control over choosing course content and textbooks. In contrast, nearly 65 percent of Western students attend schools where educators can choose the textbooks and nearly 40 percent where staff members have control over course content.
But school autonomy turns out to be controversial in Arab countries.
Masri, of Columbia’s global center, doesn’t think Arab schools should decentralize from the government. He worries that awarding greater independence to principals could backfire. “If abused, it could lead to the promotion of a specific agenda that is antithetical to the higher quality of education that we are striving for,” he says.
Not all share this view, including some of Jordan’s politicians. “I am a strong believer in empowering our schools and their leaders,” says Haifa Najjar, a senator in the upper house of the Jordan Parliament and the superintendent of two private schools. She doesn’t think autonomy should be rushed but harbors no concerns that schools could become dominated by an Islamist agenda. “I’m not worried about that at all,” she says. “Jordan is a country of moderates.”
Arab education ministers may be tempted to cite their budgets as a limiting factor in educational performance. But it’s worth noting that the Jordanian government announced a 27 percent hike in education spending last year. And World Bank data shows that education spending is relatively respectable in the region. Tunisia spent 24.4 percent of its GDP per capita on secondary education in 2008. The world average at the same time was 14.8 percent.
The fact that more Arab countries are signing up for PISA tests is an encouraging sign, says Schleicher, since it means they’re becoming less afraid of performing poorly on the international stage and more concerned with improving education. “In the past we’ve had a number of countries ask if they can do the test but not publish their results,” he says, “As a policy maker you can’t really win. If the results are good then your predecessor takes credit, if you get bad results then you’re in trouble.”
Data analysis was conducted for this article by Beirut-based consultant Tarek Oueidat.
