Jump to content


Photo

Running android in ram?

- - - - -

  • Please log in to reply
54 replies to this topic

#21
kallt_kaffe

kallt_kaffe

    Hardcore

  • Developer Team
  • PipPipPipPipPipPip
  • 1,185 posts
  • Gender:Male
  • Devices:Nexus 4 + 10 + Asus Fonepad

As far as can understand you have it running FROM ram (a /system ramdisk) rather than from nand.

Yes

And so its only any 'on the fly' loading of system modules that would be speeded up.
Does much of that go on (with our big ram)? And for what areas of the device's operation?

I'm sure some things will go quicker, but it'll cost ~120Mb RAM (with JJ) so is it worth it? Dunno? Installed apps, dalvik-cache, app-data are still on NAND.

However as I believe someone mentioned you could take this another step and put the entire system on the sd-card and load it into RAM at boot time and free up most of the NAND for data.

  • 0
Blog - App

#22
rjm2k

rjm2k

    Hardcore

  • Members
  • PipPipPipPipPipPip
  • 1,096 posts
Seems a bit OTT when there is cache in RAM anyway, some of the files loaded will not be used very frequently so loading everything into RAM would be wasteful. What might be better (if it's not already done) is firstly a large cache and secondly the ability to force certain files (like the framework) to be pre-loaded and always remain in cache (which they probably do anyway).

As pointed out earlier in the thread the WM guys are normally loading from SDcard which is already slower than NAND so the benefit they see is not likely to follow on the Blade which loads from NAND anyway.

  • 0

#23
kallt_kaffe

kallt_kaffe

    Hardcore

  • Developer Team
  • PipPipPipPipPipPip
  • 1,185 posts
  • Gender:Male
  • Devices:Nexus 4 + 10 + Asus Fonepad

Seems a bit OTT when there is cache in RAM anyway, some of the files loaded will not be used very frequently so loading everything into RAM would be wasteful. What might be better (if it's not already done) is firstly a large cache and secondly the ability to force certain files (like the framework) to be pre-loaded and always remain in cache (which they probably do anyway).

As pointed out earlier in the thread the WM guys are normally loading from SDcard which is already slower than NAND so the benefit they see is not likely to follow on the Blade which loads from NAND anyway.

Yes, we propably loose more than we gain from this hack, unless we take it further and load the entire system from SD-card. We propably could also put /cache in RAM and get over 450Mb for /data. However I'm not willing to go that far, but a simple hack like not mounting /cache in init.rc would make it end up in RAM which would be much simpler than linking it into /data. Unless of course /cache contains stuff that need to survive a reboot?

  • 0
Blog - App

#24
HMB

HMB

    Newbie

  • MoDaCo Silver
  • Pip
  • 9 posts
  • Devices:ZTE Blade

Yes, we propably loose more than we gain from this hack, unless we take it further and load the entire system from SD-card. We propably could also put /cache in RAM and get over 450Mb for /data. However I'm not willing to go that far, but a simple hack like not mounting /cache in init.rc would make it end up in RAM which would be much simpler than linking it into /data. Unless of course /cache contains stuff that need to survive a reboot?

Thank you for your work kallt_kaffe! I will try out the rom on wednesday. Do I lose clockwork recovery if I flash the modified boot.img?

Does anyone know how fast the NAND-memory on the phone is? I'm not convinced that it is faster than say a class10 SD-card.

  • 0

#25
zerosignull

zerosignull

    Diehard

  • MoDaCo Silver
  • PipPipPipPip
  • 382 posts

Thank you for your work kallt_kaffe! I will try out the rom on wednesday. Do I lose clockwork recovery if I flash the modified boot.img?

Does anyone know how fast the NAND-memory on the phone is? I'm not convinced that it is faster than say a class10 SD-card.


its probably not.

The system will cache what it needs into RAM when used. As thee system does _not_ use any virtual memory you will only really see benefits in application start up.

edit: best benchmark I found after a quick search for internal disk speed is RL Benchmark SQLite Performance. Blade spanks the Legend, Tattoo and Galaxy S.

Edited by zerosignull, 07 February 2011 - 08:24 PM.

  • 0

#26
wbaw

wbaw

    account closed

  • Banned
  • PipPipPipPipPipPip
  • 1,885 posts
  • Gender:Not Telling

but a simple hack like not mounting /cache in init.rc would make it end up in RAM which would be much simpler than linking it into /data. Unless of course /cache contains stuff that need to survive a reboot?


In android /cache is only used for downloaded apks from the market & they don't always get removed once the app is installed & it's no longer needed. If we can fix that issue, so that the apks are removed from /cache as soon as they've been installed, then it'd work well, otherwise it risks wasting ram if someone downloads or updates a few large apps between reboots. not mounting cache, linking /cache to data & clearing it on reboot is simple enough.

It is also used for storing recovery logs, but a minimum size of 1.5mb that only gets mounted by the recovery program is fine for that.

Edited by wbaw, 07 February 2011 - 09:08 PM.

  • 0

#27
wbaw

wbaw

    account closed

  • Banned
  • PipPipPipPipPipPip
  • 1,885 posts
  • Gender:Not Telling

Is there much in /system that gets changed? I thought things like the settings get stored somewhere else, otherwise they would have had problems with this too on the build for the HD2 . If there are no changes while Android is running it would be no problem that the RAM isn't permanent. Only problem would be the need of a(ext2 or 3, I'd guess) partition for /system on the SD.


system doesn't normally get changed, unless you use something with root access to change it, uninstall apps with titantium backup, or flash an 'addon' to a rom, or do something like that. it does seem like quite an extreme hack & i'd say the performance benefits are questionable. you'd probably want to use swap & mess about with the android task killer settings too, so that you could get better performance. it quickly gets complicated, for a hack that might not even work well at all.

And what if the device has a spontaneous reboot...?


If dalvik-cache was in ram, it's no big problem to lose it on reboot, it just means that the next boot will take longer when it's regenerated.

Preloading anything into ram shouldn't really provide any performance benefits overall, it'll cost ram & the time it takes to preload it.

It'll get cached in ram if you use it anyway, I think it's better to just let the kernel decide what to do with the ram.

  • 0

#28
FelixL

FelixL

    Enthusiast

  • Members
  • PipPipPip
  • 196 posts
  • Location:Germany
  • Devices:San Francisco, Moto Defy
Okay, thank's kallt for trying it, and thank's for the rest for the detailed explanations.
In unrelated news, this forum needs a "thank's" button :P
Would reduce the spam in most threads.

  • 0

#29
kallt_kaffe

kallt_kaffe

    Hardcore

  • Developer Team
  • PipPipPipPipPipPip
  • 1,185 posts
  • Gender:Male
  • Devices:Nexus 4 + 10 + Asus Fonepad

In android /cache is only used for downloaded apks from the market & they don't always get removed once the app is installed & it's no longer needed. If we can fix that issue, so that the apks are removed from /cache as soon as they've been installed, then it'd work well, otherwise it risks wasting ram if someone downloads or updates a few large apps between reboots. not mounting cache, linking /cache to data & clearing it on reboot is simple enough.

It is also used for storing recovery logs, but a minimum size of 1.5mb that only gets mounted by the recovery program is fine for that.

/cache in RAM didn't work well. It seems Market checks the free space before starting to download so it aborts and logcat shows it thinks there is no free space. The ramdisk seems to be dynamic and you can put stuff there until you run out of ram but i suppose diskfree calls doesn't work the normal way with it.

  • 0
Blog - App

#30
0x90

0x90

    Regular

  • Members
  • PipPip
  • 75 posts
  • Location:Sweden - Skövde
  • Devices:zte blade
I dont think the magic is to run the system partition in ram. Most of the framework files and data gets loaded in to ram and stays there anyway. To get a performance increase i think you have to run /data in ram. One approach would be to use a compination of tmpfs and unionfs that stores the changes into the disk but still runs mostly of the ram.

Btw. Have anyone tried formatting the partitions to ext3/ext4? I know that its a quite common approach the get better IO performance on a lot android devices.

  • 0
Open source system developer

0x90 mod

#31
rjm2k

rjm2k

    Hardcore

  • Members
  • PipPipPipPipPipPip
  • 1,096 posts

/cache in RAM didn't work well. It seems Market checks the free space before starting to download so it aborts and logcat shows it thinks there is no free space. The ramdisk seems to be dynamic and you can put stuff there until you run out of ram but i suppose diskfree calls doesn't work the normal way with it.


Googling suggests RAM disks can be fixed size, how were you creating yours?

  • 0

#32
0x90

0x90

    Regular

  • Members
  • PipPip
  • 75 posts
  • Location:Sweden - Skövde
  • Devices:zte blade

Googling suggests RAM disks can be fixed size, how were you creating yours?


Ramdisks can be a fixed size. Tmpfs mounts cant be. So my guess is that he is using tmpfs..

Edited by 0x90, 08 February 2011 - 11:49 AM.

  • 0
Open source system developer

0x90 mod

#33
ztedd

ztedd

    Regular

  • Members
  • PipPip
  • 85 posts
  • Devices:ZTE OCH_P_P729BV1.0.0B01
In fact, the dir /dev is a ramdisk. has about 200 M free space but only uses as much ram as needed.

  • 0

#34
oh!dougal

oh!dougal

    Hardcore

  • Members
  • PipPipPipPipPipPip
  • 1,022 posts
  • Location:England
  • Devices:DX2 FroYo San Francisco

I dont think the magic is to run the system partition in ram. Most of the framework files and data gets loaded in to ram and stays there anyway.

That was the impression I was under.

To get a performance increase i think you have to run /data in ram.

Mostly its just storage, isn't it? But isn't there one thing in there that COULD make a big difference - putting only the dalvik cache into ram.
Booting would take longer (though shutdown and startup scripts could potentially save and restore it, even from sdcard, to avoid rebuilding it.)
But during runtime, having the dalvik cache in a ramdisk rather than nand ought to lead to a noticeable performance improvement, oughtn't it?


AFAIK our /cache is ONLY used during market downloads.
Wbaw did some good work on partitioning to minimise /cache's permanent nand footprint. Apart from the partitioning, he was using a boot-time script to redirect /cache usage to a directory in /data. This does reduce wasted nand space, but does require that boot-script in any/every rom you use with that partitioning scheme.

Edited by oh!dougal, 08 February 2011 - 01:02 PM.

  • 0

#35
Dr. Mouse

Dr. Mouse

    Newbie

  • Members
  • Pip
  • 4 posts
  • Devices:Orange San Francisco
Hi

I am not an Android expert, but I thought I might share a few experiences I have had with running things from RAM under Linux.

The first time I ever tried this, I was actually exporting a disc from Linux to use to boot a Windows XP machine. All I did for this was run a loop containing 'cat /dev/sdb1 &>/dev/null'. Quite simply, it continually read the HDD I was interested in, thereby keeping it in the cache. It resulted in lightning reads (coming straight from RAM), but normal speed writes (as it was still writing directly to disc). This option I found to be the safest, simplest way.

The next method I have used is to modify the 'init' script, copying the required files to a tmpfs and using them for the system. Gives excellent performance (so long as you have enough spare RAM in the system for it), but it does have a major downside (or upside, depending on your usage): Writes go only to RAM, so all changes are lost on a reboot/shutdown/power loss/crash. In my case, this was for a system I was using as a set-top box, so it didn't matter to me (I had scripts I could run to manually sync files if I needed to save changes), but you would need to sync regularly and/or on shutdown, or find a way to hook into system events to know when to sync.

The next way I tried was for a more mainstream system, so the above approach was not good enough. What I did was to start with the method above, then use unionfs so changes were saved to a HDD partition. This worked quite well, but we are back to writes being the same speed as they would otherwise, and changed data must either be accessed from disc or merged into the ramdisk.

Another way I tried, mainly to fit more into RAM than I had available, was a variant on what I believe is known as Casper. Basically, for the filesystem you wish to load into RAM, you create a "squashfs" image (read-only compressed filesystem), load that into ram, mount it, then use unionfs to disk or RAM, depending on whether you want to keep changes (basically as above, but using squashfs to reduce RAM use). The CPU use from decompression is tiny (at least on a desktop, this was on a single core Atom with 2GB RAM for a Mythbuntu box at the time), and is more than compensated for by the speed of access to RAM. This method also speed boot times, as it lakes less time to load the smaller compressed image than it would the full one.

And here's the final method I've tried (god, I've spent far too much time on this, didn't realise till I started writing this). Use the Linux Software RAID driver. Build a RAID1 (mirror) using the disk storage and a file on tmpfs, with the disk set to be write-mostly (and write-behind if you can stomach data loss in a power cut or crash). This will then start off syncing the disk to RAM, reading "misses" from disk. Once this is synced, all reads and writes will go to RAM, but writes will also go to the disk.

Phew!

I don't know how much can easily be used on a Droid device (as I don't know what modules are available, probably varies between devices), but in this case I would say using SquashFS and UnionFS is probably best (due to limited RAM), but it's something which would need to be experimented with.

Wish you all well with this.

  • 0

#36
ninzor

ninzor

    Newbie

  • Members
  • Pip
  • 24 posts
  • Devices:ZTE Blade
I wonder how much of boost the dalvik cache would give when it was on ram...

Thank you for your another contribution to the ZTE Blade community, kallt_kaffe!

Edited by ninzor, 08 February 2011 - 06:18 PM.

  • 0

#37
richardtkemp

richardtkemp

    Newbie

  • Members
  • Pip
  • 18 posts
  • Devices:Dream
That essay was very informative, thanks Mouse! Also kallt for your quick test.

  • 0

#38
Phoenix Silver

Phoenix Silver

    Hardcore

  • MoDaCo Silver
  • PipPipPipPipPipPip
  • 1,839 posts
  • Gender:Female
  • Location:Strasbourg.
  • Devices:ZTE Blade Orange France
  • Twitter:@phoenixbjp

Hi

I am not an Android expert, but I thought I might share a few experiences I have had with running things from RAM under Linux.

The first time I ever tried this, I was actually exporting a disc from Linux to use to boot a Windows XP machine. All I did for this was run a loop containing 'cat /dev/sdb1 &>/dev/null'. Quite simply, it continually read the HDD I was interested in, thereby keeping it in the cache. It resulted in lightning reads (coming straight from RAM), but normal speed writes (as it was still writing directly to disc). This option I found to be the safest, simplest way.

The next method I have used is to modify the 'init' script, copying the required files to a tmpfs and using them for the system. Gives excellent performance (so long as you have enough spare RAM in the system for it), but it does have a major downside (or upside, depending on your usage): Writes go only to RAM, so all changes are lost on a reboot/shutdown/power loss/crash. In my case, this was for a system I was using as a set-top box, so it didn't matter to me (I had scripts I could run to manually sync files if I needed to save changes), but you would need to sync regularly and/or on shutdown, or find a way to hook into system events to know when to sync.

The next way I tried was for a more mainstream system, so the above approach was not good enough. What I did was to start with the method above, then use unionfs so changes were saved to a HDD partition. This worked quite well, but we are back to writes being the same speed as they would otherwise, and changed data must either be accessed from disc or merged into the ramdisk.

Another way I tried, mainly to fit more into RAM than I had available, was a variant on what I believe is known as Casper. Basically, for the filesystem you wish to load into RAM, you create a "squashfs" image (read-only compressed filesystem), load that into ram, mount it, then use unionfs to disk or RAM, depending on whether you want to keep changes (basically as above, but using squashfs to reduce RAM use). The CPU use from decompression is tiny (at least on a desktop, this was on a single core Atom with 2GB RAM for a Mythbuntu box at the time), and is more than compensated for by the speed of access to RAM. This method also speed boot times, as it lakes less time to load the smaller compressed image than it would the full one.

And here's the final method I've tried (god, I've spent far too much time on this, didn't realise till I started writing this). Use the Linux Software RAID driver. Build a RAID1 (mirror) using the disk storage and a file on tmpfs, with the disk set to be write-mostly (and write-behind if you can stomach data loss in a power cut or crash). This will then start off syncing the disk to RAM, reading "misses" from disk. Once this is synced, all reads and writes will go to RAM, but writes will also go to the disk.

Phew!

I don't know how much can easily be used on a Droid device (as I don't know what modules are available, probably varies between devices), but in this case I would say using SquashFS and UnionFS is probably best (due to limited RAM), but it's something which would need to be experimented with.

Wish you all well with this.

i used of casper method some times too
people often use it to run things from a usb stick

  • 0
Si le corps est mortel, l’âme elle est éternelle.

#39
Dr. Mouse

Dr. Mouse

    Newbie

  • Members
  • Pip
  • 4 posts
  • Devices:Orange San Francisco

i used of casper method some times too
people often use it to run things from a usb stick


Yep, I got the idea from Ubuntu, then modified it for my own use (this one was a boot-from-network media PC). Involved some dirty hacking about of the initrd and init scripts, but it was very fast (once it had loaded the image into RAM)

  • 0

#40
Phoenix Silver

Phoenix Silver

    Hardcore

  • MoDaCo Silver
  • PipPipPipPipPipPip
  • 1,839 posts
  • Gender:Female
  • Location:Strasbourg.
  • Devices:ZTE Blade Orange France
  • Twitter:@phoenixbjp

Yep, I got the idea from Ubuntu, then modified it for my own use (this one was a boot-from-network media PC). Involved some dirty hacking about of the initrd and init scripts, but it was very fast (once it had loaded the image into RAM)

yes i made the same thing :P
i have 2 or 3 ready systems in a 8 gigs stick (unetbootin is my friend hehe)
anyway with actual computers not a problem 2 or 4 gigs memory is standart

for me i prefer archlinux it's not like ubuntu a take you by the hand system

Edited by Phoenix Silver, 09 February 2011 - 02:31 PM.

  • 0
Si le corps est mortel, l’âme elle est éternelle.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users