We here at MoDaCo do plenty of device reviews. We don't tend to bring too much science into it, preferring to give an impression of our views which are always informed by the other devices each individual reviewer has used.
Whilst other publications try to offer a slightly colder and more clinical analysis of a device, we want to bring our own personalities and feelings about products into the mix.
The way we do things here at MoDaCo leads to a more impassioned end result but can have a lower level of accuracy in a technical sense. For instance, we give an impression of how we like the screen on a new device without bringing out a colorimeter to test its accuracy. You probably have a preference for the type of detail and accuracy you want to read about in product reviews, but hopefully see the benefit of all the different approaches taken across the various sources online.
Nonetheless, we have decided to bring a small amount of science to our otherwise largely subjective reviews. This is not going to be easy, none of us here at MoDaCo are scientists, nor do we have the vast resources required to take proper measurements of all aspects of the devices we review.
The first component of devices we are looking to measure is battery. Our intention is to run a battery test on each device that gives an indication of how that device compares to all the others we have tested. We are not looking to make this test perfect - that is an impossible task - but instead to give a reasonably accurate prediction of the differences between devices.
The current plan is for our battery test to have two components which we will execute on all devices. Firstly we will test the device for video playback efficiency. We will loop a standard video on the device until its previously fully charged battery reaches 10% and record the time it lasted.
Each of these two tests will be run at two different screen brightness levels. Firstly we will get the screen calibrated to as close to 200 lux brightness as possible. Secondly, we will set the brightness to maximum and run the whole test again. Once completed, we will take an average of the two runs to produce an endurance number.
It is most important to note that the results we get will not stand up on their own. If we suggest that an HTC One M8 can play video for 10 hours, that does not mean you will get the same result. However, if we suggest that number for the M8 and then say that a Sony Xperia Z2 can play video for 12 hours, you too should expect approximately 20% longer video playback time between the devices.
Within the next few months we will have our new battery testing tools prepared and in use. In the meantime, we want to know your thoughts on this. Please let us know what you think in the comments below.
Are we choosing a reasonable methodology and if not, what would you prefer to see?
Are tests like this useful to you?