Benchmark comparison of CM9 to CM11 (Quadrant, CF-Bench)

[Guest HomeEnd]

Hi fellas,

 

since I got back my trusty old Orange San Francisco yesterday, I thought it would be interesting to do some ROM comparisons, investigating if there is such thing as a "performance build".

 

The hardware:

Gen1 TFT Blade

512MB RAM

Gen1-to-Gen2 TPT v10f (180MB /system)

Stock CPU clock

 

The software:

- Eco-CM9-20130723-stable-blade by jventura (http://www.modaco.com/topic/355540-23072013ics-eco-cm9-aiming-for-stability-and-battery-efficiency/)

- cm-10-20140613-NIGHTLY-blade-DAEMOND by daemand/KonstaT (http://www.modaco.com/topic/360395-devrom2014-08-16-cyanogenmod-10-continuation-of-konstats-work/)

- cm-10.1-20130724-KonstaKANG-blade by KonstaT (http://www.modaco.com/topic/359832-devrom247-cyanogenmod-101-android-422/)

- cm-10.2-20131220-KonstaKANG-blade by KonstaT (http://www.modaco.com/topic/365330-devrom2012-cyanogenmod-102-android-431/)

- cm-11-20140608-ZeeLogKANG-blade by zeelog (http://www.modaco.com/topic/368013-devrom-86-cyanogenmod-110-android-443/)

 

Procedure every time was: Complete wipe (including /system), flash ROM, first boot, one-minute-wait, reboot and then start the benchmark. No gapps or other stuff, internet connection off (apart from some seconds to submit the quadrant score, since it won't display the values otherwise).

 

So, without further ado, I present: STATISTICS!

 

 

To visualize the results (and, to make it look a little bit more like science than nerd-spending-hours-doing-useless-nerd-stuff), I also created some charts:

 

As you can see, the best score is achieved by the good ol' CM9 build, followed not by CM10 but (surprise!) 10.1. The more recent versions both perform worse.

 

The detailed Quadrant scores:

Processor and memory are more or less equal in the first three versions, but especially the CPU performance really degrades in Android 4.3 and 4.4. Maybe that's why KitKat feels kinda sluggish even just scolling through the menus.

Interestingly, the I/O was significantly better in ICS than in the later versions.

 

And we get to the CF-Bench Native and Java results:

ICS scores first place, followed by the second JB release. Java performance is kinda bad in KK (almost half the score of CM9)

 

Last, but not least: The graphics performace!

From 4.0 to 4.2, 3D got a bit better while 2D gets just a tad less frames. Notice the huge drop at CM10.2, for both dimensional layers. KitKat performs average.

 

 

So, what do we learn?

 

CM9 is still the performance king, because of the +30% I/O throughput compared to the newer versions. Apart from that, the ROM of your choice should be KonstaT's CM10.1 build, which even offers a gain in 3D framerates (but not that big).

 

Kitkat and the latest JellyBean should be avoided, since there is a big gap in CPU respectively memory / 3D performance.

 

What this small excursion into the land of modern communication devices didn't cover: Stability, functionality, battery consumption, and, ironically, communication. I don't even know if the phone speaker works (it didn't in some CM10 build for my LG2x. Took me weeks to notice.)

 

I personally will give CM10.1 a shot as my daily driver. A little bit more modern than ICS, but not much slover. Also needs only 160MB /system TPT ;)

 

Have a nice day!

 

Cheers,

HomeEnd

[Guest szpilasty]

Thank You for your work!

As you said there is so much other things that are more important than just performance.

If someone needs all functions (camcorder, radio etc) then only CM7 will provide that.

I've tried all roms and almost all builds since February 2013. I think less problems I had was with cm10.2. (longest usage)

[Guest Lucas Dracul]

for gaming cm7 does more for me

[Guest q2dm1]

Very nice info, thank you. It would be interesting to see how CM7.2 as well as stock gingerbread-based ROMs (eg swedish snow) compare to android 4.x on this phone.

[Guest la72pt]

... I thought it would be interesting to do some ROM comparisons, investigating if there is such thing as a "performance build"...

 

.. I personally will give CM10.1 a shot as my daily driver. A little bit more modern than ICS, but not much slover....

 

thank you for you report

[Guest sharix]

Thanks for doing this!

[Guest ddesign]

Thanks nice guide!!! :)

 (y)

[Guest elrond_bs]

Unlike the other ROMs, the KitKat ROM has integrated int2ext script which will affect the performance and the results.

[Guest silverchair]

Can you share how the int2ext script would affect the performance? Also I see that latest KitKat builds do not have int2ext - can you run quadrant and post score without it?

[Guest silverchair]

Just wanted to chimn in that later KitKat release versions have degraded performance. The test KitKat version here (8 JUNE 14) scored 524 total on Quadrant, the one from 10 JULY 14 scored 506 total, while the one from 5 AUGUST 2014 (latest fitting on 180 MB system) scored 498. Haven't tested with the 2 latest ones from January since they require system partition to be bigger than 180 MB and I do not see any significant improvements that would make it worth messing with TPT once again.

[Guest Smiles]

Great review! 

I am wondering i'm having now a gen1 can i straigt away install Eco-CM9-20130723-stable-blade?