33 replies to this topic

[tcpaulh]

http://is.gd/SGx70Y




;-)

[unaszplodrmann]

Dazzozo, on 19 September 2012 - 12:50 PM, said:

Does it even pass CTS data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAMAAAC6V+0/AAAABGdBTUEAANbY1E9YMgAAAh9QTFRFV0AI////V0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIV0AIW0IIV0AIW0IIW0QJW0IIV0AIV0AIV0AIQzQQa0oJbEwJbVELbk4Kb0wJb1QMcVILc1cMlF8Ll2oNmXEQnn8Vo2gLpGgLpGoMpW8NpXEOp3YQq4MUrH0Rro0Xr4QUsI4XsZAYuXQNwIkSwI0VyIAOyocQyqcdy4QQzosS1Kod1rIf3ZMS3ZUT3a8c3o4Q3rcf4ZER5JUR5ZYS5ZgS5aIW55wT554U6KkZ6qQV6qUW6qYW6qcX66gW7Lsf7a0X7a0Y7a8Z7bEb7rAY7rEY7rMa7rMb77EY77Ma77QZ77cd77oc8bgb8bkb8bob8b4f8cUh8skh87wb870c874d88If88kk89Im9Mwl9cMd9cch9sUe9sYf9scf9sgg9skh9swi9swk98kf98og988k99Mm+c8h+dMk+dYm+9Mi+9Uj+9Yk+9cl+9gl+9kn+9on+9oo+9op+9so/NUj/Ncl/Ngl/Non/Nso/N0o/N0p/dol/d0n/eAp/eEp/twm/t0k/t4l/uAn/uIq/uMp/uMq/uQq/wIA/21n/98m/+Am/+An/+En/+Eo/+Io/+Ip/+Mo/+Mp/+Mq/+Qp/+Qq/+Qr/+Uq/+UrZYWE0gAAACd0Uk5TAAABCQoLHCYnMDY4OjuCg4WHiImKm56fx8jJys3P1Ofp6enq7/DyAdnyKAAAAUJJREFUGNNjYMQCGBgZgICZi09CRUWCj4sZxIMIsgnr2niWlXhYaQmzQQWZOOWMS/unT+trq48xkuNkAguyK7n2z168ZdOiWT1NuU7y7CBBVjH7CQvU1dU3A3FPXYalKAtQkEe/eZb65s0bN27csEa9tThMmxsoKGTXuxAksm7durXzuyoSLASAgjJ+U9eDRVavXr1qcm2KuyRQUCV/yhqwyOqVK1dMqkn2VgAKquZ1q6uDRFauWD5PXT3eCyQo69M6c4U6FLQURblJAwVFbGs75yxbrr506VL19qr0UDNBoCCvXmZjx4y56kuWLFFvyI701+EBOV7cOqe6pX2i+hL1yqzoQFMxVpA3OZQd0wrLKyvVC1LDAx0U2WEBYhgcl5QYGxHiawALEFDQaZq7BAU4m2jAgw4cyPxSampS/PBAxgIAZN1ka/x53bAAAAAASUVORK5CYII=

No seriously I'm going to check when I'm back

Mmm. Are you taking bets?

[tcpaulh]

Dazzozo, on 19 September 2012 - 12:50 PM, said:

Does it even pass CTS

No seriously I'm going to check when I'm back

Well I'd finished downloading CTS and the media pack before Huawei pulled the release so I'm giving it a go from Windows. Phone goes crazy. It's currently doing all sorts of graphics tests. Scratch that, it's now taking pictures. Shutter going like the clappers. Fun. Oh god, now it's playing all sorts of cheesy music.

Edit:

Anyone who wants a play:-

Follow http://source.androi.../cts-intro.html

Tips

Make sure adb and java are in your path environment variable

I installed to d:\cts and made a .bat file in the android-cts/tools directory to launch it containing

java -cp ddmlib-prebuilt.jar;tradefed-prebuilt.jar;hosttestlib.jar;cts-tradefed.jar -DCTS_ROOT=D:\cts com.android.cts.tradefed.command.CtsConsole

Then from the console that pops up you can list tests with "list plans"

Then "run cts --plan xxx" where x is the test to run eg Android

Edited by tcpaulh, 19 September 2012 - 10:06 PM.

[Davidoff59]

Did it pass tcpaul?

[tcpaulh]

Davidoff59, on 19 September 2012 - 10:14 PM, said:

Did it pass tcpaul?

Still running. Currently enjoying an animated owl singing like a sparrow :-O

There have been a couple of fails in the console. Hope I can find the log at the end.

Edited by tcpaulh, 19 September 2012 - 10:25 PM.

[Dazzozo]

tcpaulh, on 19 September 2012 - 10:19 PM, said:

Still running. Currently enjoying an animated owl singing like a sparrow :-O

There have been a couple of fails in the console. Hope I can find the log at the end.

I believe it opens a report page that is intended to be emailed to Google.

[tcpaulh]

Dazzozo, on 19 September 2012 - 11:01 PM, said:

I believe it opens a report page that is intended to be emailed to Google.

Cheers

I only ran the Android plan. Not sure if this is all we need (when vofafone release ics). Also, think I overdid it on the media. I copied everything to the SD card. Probably should have just copied 800x480 stuff.

[tcpaulh]

Davidoff59, on 19 September 2012 - 10:14 PM, said:

Did it pass tcpaul?

Here are the test failures + a little additional info :- https://docs.google....3LcikuhOgLTtQHQ

[bladebuddy]

How many fails. That's unbelievable?

[tcpaulh]

I guess those modes are unsupported. Not sure though.

[unaszplodrmann]

Webkit and textureview issues notwithstanding, not so bad, afterall. The problem with posting the results is that most will just look at the summary and be all OMGWTFBBQ!!111

Edited by unaszplodrmann, 20 September 2012 - 10:51 AM.

[tcpaulh]

Was only a subset of tests. Will prob run more when official comes out.

Available test plans are :-

1. CTS- contains all tests and will run ~17,000 tests on your device. These tests are required for compatibility. At this point performance tests are not part of this plan (this will change for future CTS releases).
2. Signature- contains the signature verification of all public APIs
3. Android- contains tests for the android APIs
4. Java- contains tests for the Java core library
5. VM- contains tests for the Dalvik virtual machine
6. RefApp- contains reference application tests (more coming in future CTS release)
7. Performance- contains performance tests for your implementation (more coming in future CTS releases)

I ran Android which took 3 hours

Any views on which of the others are most important or is it just 'all of them'?

Edit: Might as well run the java test ;-) ... too much spare time at the moment. Should be updating my CV

Java test completed in 1h 50m with 11 fails. Previous Google doc updated.

Edited by tcpaulh, 20 September 2012 - 03:08 PM.

[tcpaulh]

...and we're off

Running full CTS tests against B952

I forgot the media folders bbb_full and bbb_short needed to be in sdcard/test directory.

Results in about 3 or 4 hours

[tcpaulh]

Well that took longer than expected.

There were 91 fails. I've logged 89 at https://docs.google....Wy25fUUBzk/edit

I took three out that I caused by hanging up during telephony/sms tests

That's not the full report, just an overview of fails. Full report is on it's way to google.