Heads will be scratching at the Gauteng Department of Education today, as the results of the 2015 matric exams show that two high profile projects to introduce new technologies into public school classrooms have failed to impress with their results.

Indeed, in the majority of schools that formed the “Classrooms of the Future” pilot that we’ve been tracking, the percentage of learners taking their exams last year who passed have suffered a fall in results that’s greater than the regional average.

Gauteng’s overall pass rate dropped 3% from 87.2% in 2014 to 84.2% in 2015, while the average pass rate in the nine schools we’ve looked at fell by 5.4%

Only two schools did better than the regional drop: Sunward Park, which was the first public school in South Africa to go paper-free, saw a small drop of 0.7% with a much larger cohort than last year (244 pupils compared to 197 in 2014). J Kekana in Soweto, meanwhile, was the only school which saw a rise in its pass rate, going from a lowly 65.7% last year to a very impressive 81% this time round.

The classrooms of the future project was introduced in two stages last year. At the start of the academic year, a phase one trial made up of seven schools included five senior schools. They were:

• Tandi Eleanor Secondary
• Ponelopele Oracle Secondary
• Bioitumelong
• Phomolong Secondary

Phase two of the project saw schools in Soweto equipped with new tablets and interactive whiteboards for matric classes in July. Some 4 000 classrooms were marked to be upgraded, but the Gauteng Department of Education hasn’t confirmed how many were completed by the end of the academic year. We don’t have a list of the schools which were upgraded, so have focussed on four schools in the area.

• Protea Glen Secondary (where phase two was launched)
• Phafogang Secondary – which made headlines when thieves broke in and stole the new equipment
• J Kekana Secondary School
• Progress Comprehensive

We’ve been able to compare all the results from 2014 to 2015 for all nine schools.

The biggest relative drops were in Tandi Eleanor, which saw its overall pass rate fall from 67% to 52.5%, and Protea Glen, which fell from 93.6% to 81.4%. Boitumelong, the flagship of phase one where all pupils have tablets, saw its pass rate fall almost 10% from 80.3% to 70.9%. Phomolong Secondary failed to achieve a 100% pass rate for the first time in three years.

While it’s tempting to see these results as a condemnation of the project, which has cost millions of rand so far, you must be careful with interpretation. Most of these schools also saw vastly increased numbers of pupils sitting exams this year – in the case of Phomolong Secondary 101 more pupils wrote papers than in 2014.

So it might be the case that technology encourages more learners to stay on for matric than previously, and makes classes more engaging.

Similarly, we haven’t seen a breakdown of what levels of pass were achieved. Did a lower proportion pass but with better overall grades, for example? A lot more analysis needs to be done before we can judge that, and we’ll be trying to get in touch with individual schools over the next months.

We have approached the Gauteng Department of Education for comment, but have not received a reply at the time of publication. All data comes from the annual reports by the Ministry of Education for 2014 (here) and 2015 (here).

It’s unlikely to mean an end for the project overall, but it can’t be the news MEC Panyaza Lesufi was hoping for.