Partition layout change

[Guest Arr Too]

the best disassembler in the world:

Ahem... maybe one that doesn't more than cost four times what the phone cost? :(

[Guest isambard]

Ahem... maybe one that doesn't more than cost four times what the phone cost? :(

well, it would also disassemble arm code running on a htc desire HD ;)

[Guest IronDoc]

Moving apps to /system/app not pratical. /system not a r/w partition, so these apps not upgreadable from market. It's simply wasted space, so we need the modified layout badly. :unsure:

They are upgradeable; it must store the updates in data I guess. You can integrate updates into system with titanium.

[Guest isambard]

are the system.img and boot.img meant to be in yaffs2 format? what tool do you use to extract/make these? i have a yaffs2 tool but haven't used it before and it doesn't seem to be able to extract these images as yaffs2 and i'm wondering if they images are really yaffs2 format or whether i'm doing something wrong with the imaging tool.

EDIT: i just noticed that Sebastian's romdump tool produces the same .img files. what do people use to manipulate these? thanks.

Edited December 31, 2010 by isambard

[Guest Arr Too]

The system partition is a YAFFS2 filesystem, so system.img ought to be a dump of that -- you use unyaffs to extract the files. The boot.img is different and you need to use these tools to extract and repack the boot and ramdisk parts.

[Guest isambard]

The system partition is a YAFFS2 filesystem, so system.img ought to be a dump of that -- you use unyaffs to extract the files. The boot.img is different and you need to use these tools to extract and repack the boot and ramdisk parts.

thanks for the link. it's good to see some information is out there. since i'm completely new to android, i feel like i'm constantly re-inventing the wheel and having to re-discover things that are already known. this will hopefully help to reduce some of that. thanks.

[Guest fonix232]

Basically then, AMSS.MBN is the RADIO part of the firmware? Then that "little" bastard changes the phone to see the +256MB RAM :unsure:

Also, CustomMTD may even work on our phone, but needs a bit of a re-engineering, as our partition table's order is a bit different.

Edited January 1, 2011 by fonix232

[Guest isambard]

Basically then, AMSS.MBN is the RADIO part of the firmware? Then that "little" bastard changes the phone to see the +256MB RAM :unsure:

Also, CustomMTD may even work on our phone, but needs a bit of a re-engineering, as our partition table's order is a bit different.

yes, amss is the Advanced Mobile Subscriber Software in qualcomm lingo. i imagine it would control the ram amount as i guess all the ram is on the same chip, but i'm trying to get hold of an msm7227 datasheet to get more info on the qualcomm chipset.

i'm hoping it is as easy as just changing the partition tables and the onboard phone firmware making the necessary changes. but first i want to try to understand the boot process to make sure that the OS can still boot from a changed partition table (i.e. make sure there's no need to rebuild for altered partition table).

technically, the phone's boot-strapping routine should somehow locate and load the first stage bootloader. since i'm not familiar with android i don't know what this is, but i'm guessing this should sit in the boot partition. how the phone locates this could be hard-wired or dynamically discovered, but as we wont change the boot partition, we don't care and this should work.

this then leads me onto the boot.img and where i got stuck unpacking the image, because we should examine the bootloader. this should set up the ram etc. and the bootloader should also locate and load the kernel. this is the interesting bit for us when it comes to re-partitioning. because if the system partition location is hardwired into the bootloader, then we'd need to change this to enable boot (assuming kernel is located in system partition). but if it is discovered in some way (e.g. it can read the partition table etc.) then we can leave the boot loader alone.

i'm also not sure where the kernel is located and whether it is loaded into ram first before executing or whether it is executing in place. again similar issues with understanding where the kernel is located and how it is found and loaded.

Edited January 1, 2011 by isambard

[Guest fonix232]

yes, amss is the Advanced Mobile Subscriber Software in qualcomm lingo. i imagine it would control the ram amount as i guess all the ram is on the same chip, but i'm trying to get hold of an msm7227 datasheet to get more info on the qualcomm chipset.

i'm hoping it is as easy as just changing the partition tables and the onboard phone firmware making the necessary changes. but first i want to try to understand the boot process to make sure that the OS can still boot from a changed partition table (i.e. make sure there's no need to rebuild for altered partition table).

technically, the phone's boot-strapping routine should somehow locate and load the first stage bootloader. since i'm not familiar with android i don't know what this is, but i'm guessing this should sit in the boot partition. how the phone locates this could be hard-wired or dynamically discovered, but as we wont change the boot partition, we don't care and this should work.

this then leads me onto the boot.img and where i got stuck unpacking the image, because we should examine the bootloader. this should set up the ram etc. and the bootloader should also locate and load the kernel. this is the interesting bit for us when it comes to re-partitioning. because if the system partition location is hardwired into the bootloader, then we'd need to change this to enable boot (assuming kernel is located in system partition). but if it is discovered in some way (e.g. it can read the partition table etc.) then we can leave the boot loader alone.

i'm also not sure where the kernel is located and whether it is loaded into ram first before executing or whether it is executing in place. again similar issues with understanding where the kernel is located and how it is found and loaded.

The CustomMTD method passes a special command-line parameter to the bootloader, and this parameter overwrites the built-in MTD partition layout. So yes, it is kinda hardwired, but can be easily modified.

Boot.img is a package of the kernel, ramdisk, and the root filesystem (/.), with initial boot procedures, etc.

The boot process is the following:

- Phone is switched on

- Primary bootloader loaded

- Secondary bootloader loaded by primary (this secondary sets up the MTD partition layout for the system)

- Kernel launched by SPL (secondary bootloader)

- System launched by kernel

[Guest isambard]

The CustomMTD method passes a special command-line parameter to the bootloader, and this parameter overwrites the built-in MTD partition layout. So yes, it is kinda hardwired, but can be easily modified.

Boot.img is a package of the kernel, ramdisk, and the root filesystem (/.), with initial boot procedures, etc.

The boot process is the following:

- Phone is switched on

- Primary bootloader loaded

- Secondary bootloader loaded by primary (this secondary sets up the MTD partition layout for the system)

- Kernel launched by SPL (secondary bootloader)

- System launched by kernel

you're right. i'm an idiot. the boot.img was not a yaffs2 image but a custom format of a few things mashed together. the format can be determined from the public android source code:

http://android.git.kernel.org/?p=platform/...otimg/bootimg.h

Edited January 1, 2011 by isambard

[Guest isambard]

ok. i tried it, but the partition sizes did not change :unsure:

on the plus side, i can confirm this recovery mechanism works for the swiss version of the blade and it's a very effective way of restoring the phone - including the recovery partition.

[Guest StevenHarperUK]

ok. i tried it, but the partition sizes did not change :unsure:

on the plus side, i can confirm this recovery mechanism works for the swiss version of the blade and it's a very effective way of restoring the phone - including the recovery partition.

Just to be certain does this confirm that : http://android.modaco.com/content/zte-blad...vices-to-512mb/

Is good for the swiss device?

thanks

[Guest isambard]

i can't confirm exactly that since, i never applied a non-modified version of that zip file.

what i did was this:

create the image file, but then replace the partition_zte.mbn with my hacked version. the hacked version reduced the size of the system partition to around 130mb (i did not try to increase the data partition at this stage).

when i applied the update, i was left with a 'bricked' phone. the system.img failed at 98%. at first i was hopeful that this was due to the system partition being too small to fit the image.

i then retried only this time, i replaced the mbn and the boot.img and system.img from a clockwork backup and the recovery.img was replaced with a clockworkmod image.

the low-level update mechanism applied these nicely.

i can also confirm that the update mechanism also applies when a file is missing e.g. i tried to update without a system.img and this worked.

the question is now: does the update mechanism totally ignore the partition_zte.mbn file? or did i make some sort of mistake in it which means it is rejected (or rejected due to the gap between system and data in my hacked version - it possibly just reads the partition start info and ignores the length - maybe i need to try again!). does it not touch the partition at all, or does it simply overwrite with the default layout. i'm going to try again but decrease system and increase data to see if i can rule out one of the possibilities above.

[Guest StevenHarperUK]

I am very interested in this.

It's a great way to get back to a stock device - also to apply custom ROMS, recovery etc..

Also its nice to see that it's recoverable from when it goes wrong.

If you get time would you be able to try the clean Hungarian update.zip - so I can confirm it works

Ta for your brave testing :unsure:

[Guest rjm2k]

i can't confirm exactly that since, i never applied a non-modified version of that zip file.

what i did was this:

create the image file, but then replace the partition_zte.mbn with my hacked version. the hacked version reduced the size of the system partition to around 130mb (i did not try to increase the data partition at this stage).

when i applied the update, i was left with a 'bricked' phone. the system.img failed at 98%. at first i was hopeful that this was due to the system partition being too small to fit the image.

i then retried only this time, i replaced the mbn and the boot.img and system.img from a clockwork backup and the recovery.img was replaced with a clockworkmod image.

the low-level update mechanism applied these nicely.

i can also confirm that the update mechanism also applies when a file is missing e.g. i tried to update without a system.img and this worked.

the question is now: does the update mechanism totally ignore the partition_zte.mbn file? or did i make some sort of mistake in it which means it is rejected (or rejected due to the gap between system and data in my hacked version - it possibly just reads the partition start info and ignores the length - maybe i need to try again!). does it not touch the partition at all, or does it simply overwrite with the default layout. i'm going to try again but decrease system and increase data to see if i can rule out one of the possibilities above.

Have you tried my hack?

[Guest isambard]

Have you tried my hack?

no i did not try yours. it has a chance of working better than mine, as i did not fully 'fix' the mbn file.

[Guest Tom G]

The boot process is the following:

- Phone is switched on

- Primary bootloader loaded

- Secondary bootloader loaded by primary (this secondary sets up the MTD partition layout for the system)

- Kernel launched by SPL (secondary bootloader)

- System launched by kernel

appsboot.mbn is at least part of the SBL. It is the bit that runs directly before linux starts and loads the kernel, ramdisk etc. This is the bit we can see output from on rs232. It is probably were the partition info is loaded and passed to the linux kernel.

[Guest Tom G]

I've had a play with partition_zte.mbn and I don't think it is what you all hoped. I think it may only be used by the flash program so it knows where to write the partitions. isambard had corruption when attempting to modify it I assume because the new images would have been written to the wrong addresses, but if you change the partition_zte.mbn file and don't write the partitions nothing breaks (and it should break if the partitions had changed). The partition layout from the bootloader does not change after flashing with a modified partition_zte.mbn.

The below output is after flashing with the attached partition_zte.mbn. It is exactly the same as before flashing.

ptn 0 name='recovery' start=00000156 len=00000024 flags=00000000 type=Apps Writable=Yes

ptn 1 name='boot' start=0000017a len=00000024 flags=00000000 type=Apps Writable=Yes

ptn 2 name='splash' start=0000019e len=0000000c flags=00000000 type=Apps Writable=Yes

ptn 3 name='misc' start=000001aa len=00000003 flags=00000000 type=Apps Writable=Yes

ptn 4 name='cache' start=000001ad len=0000014a flags=00000000 type=Apps Writable=Yes

ptn 5 name='system' start=000002f7 len=0000067c flags=00000000 type=Apps Writable=Yes

ptn 6 name='userdata' start=00000973 len=00000681 flags=00000000 type=Apps Writable=Yes

ptn 7 name='persist' start=00000ff4 len=0000000c flags=00000000 type=Apps Writable=Yes

ptn 8 name='MIBIB' start=00000000 len=0000000a flags=00000000 type=Modem Writable=Yes

ptn 9 name='QCSBL' start=0000000a len=00000002 flags=00000000 type=Modem Writable=Yes

ptn 10 name='OEMSBL1' start=0000000c len=00000005 flags=00000000 type=Modem Writable=Yes

ptn 11 name='OEMSBL2' start=00000011 len=00000005 flags=00000000 type=Modem Writable=Yes

ptn 12 name='AMSS' start=00000016 len=000000d4 flags=00000000 type=Modem Writable=Yes

ptn 13 name='APPSBL' start=000000ea len=00000003 flags=00000000 type=Modem Writable=Yes

ptn 14 name='FOTA' start=000000ed len=00000002 flags=00000000 type=Modem Writable=Yes

ptn 15 name='EFS2' start=000000ef len=00000060 flags=00000000 type=Modem Writable=Yes

ptn 16 name='APPS' start=0000014f len=00000002 flags=00000000 type=Modem Writable=Yes

ptn 17 name='FTL' start=00000151 len=00000005 flags=00000000 type=Modem Writable=Yes

ptn 18 name='EFS2APPS' start=00000156 len=ffffffff flags=00000000 type=Modem Writable=Yes

partition_zte.zip

Edited January 2, 2011 by Tom G

[Guest Tom G]

OK. I think i've Found it.

It looks like the partition layout starts at 0xAA70 of appsboot.mbn.

The partition_zte.mbn probably needs to match what is there for the flash process to work correctly.

[Guest Tom G]

New partition layout :unsure:

ptn 0 name='recovery' start=00000156 len=00000024 flags=00000000 type=Apps Writable=Yes

ptn 1 name='boot' start=0000017a len=00000024 flags=00000000 type=Apps Writable=Yes

ptn 2 name='splash' start=0000019e len=0000000c flags=00000000 type=Apps Writable=Yes

ptn 3 name='misc' start=000001aa len=00000003 flags=00000000 type=Apps Writable=Yes

ptn 4 name='cache' start=000001ad len=0000014a flags=00000000 type=Apps Writable=Yes

ptn 5 name='system' start=000002f7 len=0000047c flags=00000000 type=Apps Writable=Yes

ptn 6 name='userdata' start=00000773 len=00000881 flags=00000000 type=Apps Writable=Yes

ptn 7 name='persist' start=00000ff4 len=0000000c flags=00000000 type=Apps Writable=Yes

ptn 8 name='MIBIB' start=00000000 len=0000000a flags=00000000 type=Modem Writable=Yes

ptn 9 name='QCSBL' start=0000000a len=00000002 flags=00000000 type=Modem Writable=Yes

ptn 10 name='OEMSBL1' start=0000000c len=00000005 flags=00000000 type=Modem Writable=Yes

ptn 11 name='OEMSBL2' start=00000011 len=00000005 flags=00000000 type=Modem Writable=Yes

ptn 12 name='AMSS' start=00000016 len=000000d4 flags=00000000 type=Modem Writable=Yes

ptn 13 name='APPSBL' start=000000ea len=00000003 flags=00000000 type=Modem Writable=Yes

ptn 14 name='FOTA' start=000000ed len=00000002 flags=00000000 type=Modem Writable=Yes

ptn 15 name='EFS2' start=000000ef len=00000060 flags=00000000 type=Modem Writable=Yes

ptn 16 name='APPS' start=0000014f len=00000002 flags=00000000 type=Modem Writable=Yes

ptn 17 name='FTL' start=00000151 len=00000005 flags=00000000 type=Modem Writable=Yes

ptn 18 name='EFS2APPS' start=00000156 len=ffffffff flags=00000000 type=Modem Writable=Yes

~ # cat /proc/mtd

dev:	size   erasesize  name

mtd0: 00480000 00020000 "recovery"

mtd1: 00480000 00020000 "boot"

mtd2: 00180000 00020000 "splash"

mtd3: 00060000 00020000 "misc"

mtd4: 02940000 00020000 "cache"

mtd5: 08f80000 00020000 "system"

mtd6: 11020000 00020000 "userdata"

mtd7: 00180000 00020000 "persist"

~ # df -h

Filesystem				Size	  Used Available Use% Mounted on

tmpfs				   207.5M		 0	207.5M   0% /dev

/dev/block/mtdblock4	 41.3M	  1.2M	 40.1M   3% /cache

/dev/block/mtdblock6	272.1M	 38.0M	234.1M  14% /data

/dev/block/mtdblock5	143.5M	 92.8M	 50.7M  65% /system

Use the attached modified appsboot.mbn and partition_zte.mbn if you don't want to do it yourself.

It looks like both partitions still successfully mount after the change without rewritting them, but I would expect some corruption. It would be a good idea to rewrite the system and userdata partitions.

PS. I take no responsibility for any bricked devices using the attachment. It worked for me, but that doesn't mean it'll work for everyone. That said it does appear fairly safe, I'm just not sure if appsboot.mbn is actually the bit that is doing the flashing so playing with it potentially could brick the phone.

resize.zip

Edited January 2, 2011 by Tom G

[Guest joyfun]

lol the data partition increased

It seems like change the SPL in google G1(Dream)

New partition layout :unsure:

~ # cat /proc/mtd

dev:	size   erasesize  name

mtd0: 00480000 00020000 "recovery"

mtd1: 00480000 00020000 "boot"

mtd2: 00180000 00020000 "splash"

mtd3: 00060000 00020000 "misc"

mtd4: 02940000 00020000 "cache"

mtd5: 08f80000 00020000 "system"

mtd6: 11020000 00020000 "userdata"

mtd7: 00180000 00020000 "persist"

~ # df -h

Filesystem				Size	  Used Available Use% Mounted on

tmpfs				   207.5M		 0	207.5M   0% /dev

/dev/block/mtdblock4	 41.3M	  1.2M	 40.1M   3% /cache

/dev/block/mtdblock6	272.1M	 38.0M	234.1M  14% /data

/dev/block/mtdblock5	143.5M	 92.8M	 50.7M  65% /system

Use the attached modified appsboot.mbn and partition_zte.mbn if you don't want to do it yourself.

It looks like both partitions still successfully mount after the change without rewritting them, but I would expect some corruption. It would be a good idea to rewrite the system and userdata partitions.

PS. I take no responsibility for any bricked devices using the attachment. It worked for me, but that doesn't mean it'll work for everyone. That said it does appear fairly safe, I'm just not sure if appsboot.mbn is actually the bit that is doing the flashing so playing with it potentially could brick the phone.

Edited January 2, 2011 by joyfun

[Guest Tom G]

lol the data partition increased

It seems like change the SPL in google G1(Dream)

Wasn't an increase in the data partition (and reduction in system) what we were trying to achieve?

Previously it was

~ # df -h

Filesystem				Size	  Used Available Use% Mounted on

tmpfs				   207.5M		 0	207.5M   0% /dev

/dev/block/mtdblock4	 41.3M	  1.1M	 40.1M   3% /cache

/dev/block/mtdblock5	207.5M	118.7M	 88.8M  57% /system

/dev/block/mtdblock6	208.1M	 17.3M	190.8M   8% /data

I'm not sure 143MB will be enough for system. Its probably close to what we want, but may be a little too small. No thought went into the new sizes, I just reduced system by 0x200 blocks and added them to userdata (0x200 blocks * 0x20000 byte blocksize = 64MB moved from system to data). I know some roms don't make use of the cache partition so that space could also be reallocated.

Edited January 2, 2011 by Tom G

[Guest Arr Too]

New partition layout :unsure:

Top work! Was this helped by your RS232 debug port?

[Guest Tom G]

Top work! Was this helped by your RS232 debug port?

A little because I could see when the bootloader output changed (and could see that partition_zte.mbn didn't effect it), but it could all be done without the rs232. The changes can be seen from linux.

[Guest Krinyo]

Tom, You are incredible. And brave! :unsure: This is what I need! But I'm not so brave, so waiting a little for another brave mates... and for a good 2.2 port with same layout. B)