215 replies to this topic

[freeasabeer]

(for reference, topic previous name was: screen coming to MCR and have also merge the "mc coming to MCR" topic)

8-feb: Major update in the way these additional toys are handled (now with ipkg feed).

I will now publish the linux tools packaged as ipk files and one will have to use the opkg tool to install / uninstall / udate these ipk packages.
For reference, opkg is a lightweight package management system based on Ipkg. Ok, but what is ipkg ? Easy: ipkg re-implements for embedded systems the functionality of Debian tools dpkg, apt, and apt-get. For example, it uses much less disk space for the package metadata.
Believe me, this will make your life as end-user quiet easier and as a side effect it will make my life as developer & packager quite easier as well

So, having said that; the first action is to have opkg installed on your phone. To achieve this, download this specific opkg bootstrap  opkg_bootstrap_v2.tar.bz2.zip   262.9K   1285 downloads.
Unzip it to your SD card, open a shell with adb and run:

cd /
tar xjf /sdcard/opkg-bootstrap-v2.tar.bz2
/data/xbin/opkg install /sdcard/opkg_0.1.7+svn519-2_arm.ipk

You now have installed opkg as an ipk package. This means:
1) you are now able to easily install other ipk
2) opkg can now also be easily updated if required !

One of the benefitis of opkg is the ability to set-up an internet feed so as to be able to update or install packages while being online. Edit opkg configuration file /data/local/opkg.conf in this way:

src/gz common http://freeasabeer.byethost33.com/android/feed/common
src/gz mcr http://freeasabeer.byethost33.com/android/feed/mcr
#src/gz cyanogen http://freeasabeer.byethost33.com/android/feed/cyanogen

dest root /

#option verbose-wget
#
# Proxy support:
#
#option http_proxy http://localhost:5865
#option ftp_proxy http://proxy.tld:3128
#option proxy_username <username>
#option proxy_password <password>

Note:
If you're using Paul's MCR, then those settings are OK.
If you're using one of the 2.1 cyanogen based ROM, then comment out the second line and uncomment the third one.


You are now ready to update the package list and to install from the feed:

 opkg update

Note 2:
If you already had opkg installed from the previous bootstrap, then you will have to do the following (as there is a bug in former opkg package that prevent it to actually find the configuration file):

opkg -f /data/local/opkg.conf update
opkg install opkg

Some quick instructions on how to use opkg:

Always run "opkg update" before to be sure to get the latest updates

Install the rsync package:

opkg install rsync

Want to remove the package, easy:

opkg remove rsync

Want to list installed packages:

opkg list_installed

Want to list files belonging to a given package (e.g. rsync):

opkg files rsync

You don't need to reboot your phone in recovery mode, neither to just reboot it. Just install things  live.


PS: you don't need to uninstall or remove any of the previously installed tools: they will (should) be nicely overwritten.

Forewords:

these linux tools are mainly intended for geeks that like top play with the internals of their Hero from a shell (either adb shell, ssh connection, or a terminal application). Regular users won't find any benefit by installing these tools: they won't make their hero faster or work better, they will just increase the ROM memory consumption.



1. mc (Midnight Commander)

If you don't know what mc is then you most probably don't need it. But you may be interested to discover what it is.

For the geeks that do want it to easily browse the phone file system from a ssh or adb shell connection, I've done a uclibc statically linked binary of mc that you can install with opkg.

From what I've seen, it mostly works ok from a linux xterm connected with adb shell: browsing files is ok and I've not tested any advanced features of mc. I have no idea if it is usable from a windows cmd terminal or a cygwin one.
Once installed, open a shell on your phone and start mc this way: mc



2. GNU screen

As a midnight commander companion, we can now enjoy screen !

Don't know what screen is ? Check Wikipedia.

Basically screen allow to open several shell connections in parallel from one physical connection to the phone (either adb shell or ssh) and to switch from one to the other one at will. It also allow to close the physical connection without losing the ongoing shell connections so that you can resume them once you have restarted the physical connection.

An interesting side effect I've noticed is that vi works great within a screen shell where as is was unusable from a direct shell !

I have however to raise a warning here: in order to get screen to work I had to setup a system/etc/passwd file with the solely purpose to declare the root user and I've done it with an empty password. So I'm not sure yet of the security implication that this could have on the phone... On a regular unix system the security breach is obvious but here on an anyway rooted phone I'm not so sure. So, you've been warned.

Once installed, you can start screen with: screen and resume an ongoing session with screen -r



3. tree

Tree is a recursive directory listing program that produces a depth indented listing of files, which is colorized ala dircolors if the LS_COLORS environment variable is set and output is to tty.



4. rsync

Due to popular demand, here is also rsync...
Note: installing rsync will also install a new version of dropbear.
Some hints on how to use rsync with the Hero:

1) From the Hero to a remote host
In this scenario, you run rsync on the hero itself and connect to a remote host.
There are no oddities here, just use rsync as you normally would.

2) From a linux/mac host to a remote Hero
In this scenario, you rsync on linux/mac host and connect to a remote hero.
In this case, the rsync invocation is a bit tricky, for example:
rsync -av -e 'ssh -p 2222' --rsync-path=/data/xbin/rsync <some local files/dir> root@<YOU_HERO_IP>:/data/tmp



5. busybox

This is an updated busybox as compared to what Paul currently ships with MCR (even current betas). This one do include my ip resolv tweak !



6. wget

Don't ask for wget, it is already provided by busybox  



7. htop

First release for htop... However I don't really see how to use the Fx keys... So pretty useless ?



8.  zip

The famous zip binary is now available !



Enjoy !

Edited by freeasabeer, 11 March 2010 - 09:09 AM.

[freeasabeer]

Changelog:

5-feb-2010: Major update
- Make use of opkg tool and of the .ipk package format to handle the tools

3-feb-2010: screen 0.3
- Regression fix in screen wrapper script

3-feb-2010: busybox  0.3
- Bug fix for the update script of busybox 1.16.0

29-jan-2010: busybox  0.2
- busybox 1.16.0 packged for modaco with the ip name resol fix.

29-jan-2010: zip 0.1
- first release for the famous zip binary

25-jan-2010: htop 0.1
- first release for htop. (Within ConnectBot, the Fn key is emulated with: Trackball + n)

21-jan-2010: screen 0.2
- made it more MCR friendly: recompiled to use /data/screen as its "data" folder where the global screenrc file can be found
- addition of the uft8encodings in /data/screen
- uses now /data/local/tmp as the temporary folder instead of /data/tmp

18-jan-2010: busybox 0.1
Modifications to Paul's busybox
- crond: spool folder is set to /data/cron by default
- ls color enabled by default
- fixed the ip name resolve bug. Requires a valid /etc/resolv.conf file.
The update package also make /etc/resolv.conf a symlink to /data/local/resolv.conf and provides a valid /data/local/resolv.conf file (Google DNS).
A simple script that reads net.dns1 property and updates /data/local/resolv.conf accordingly is also provided in /data/xbin/set_resolconf.

15-jan-2010: rsync 0.3
- fixed issue in the update-script that prevented installation in the case rsync was already installed
- lesson to learn: don't publish an update too late in the night

15-jan-2010: rsync 0.2
- dropbear rebuild to fix non working scp
- /system/bin/dropbear.sh script updated to allow use of public keys in /data/dropbear/.ssh/authorized_keys

Edited by freeasabeer, 05 February 2010 - 10:27 PM.

[kendon]

this is so f***ing awesome! thanks!

do you take requests? i'd be interested in tree aswell...
and i am sure i can find more, but i don't have the skills to make them work...

edit: needed to get a superlong hyperlink on my phone, abused the forum for this  

Edited by kendon, 13 January 2010 - 01:49 PM.

[freeasabeer]

Done !
Now thinking of the best way to provide it without running off topic from the 4th post on  
Not sure opening a new topic is the best one ? I'll attach it to the first post of this topic. 

Edited by freeasabeer, 13 January 2010 - 10:05 PM.

[PaulOBrien]

Is this a static compilation?

I can include these in future MCR builds (along with nano, busybox) etc.

I have a request... rsync...

P

[freeasabeer]

Yes they are all static binaries against uclibc.
rsync, that's a pretty good suggestion ! I'll have a look at it and keep you informed.
And yes, you can include then in MCR!

[kendon]

freeasabeer, on Jan 13 2010, 16:00, said:

Yes they are all static binaries against uclibc.
rsync, that's a pretty good suggestion ! I'll have a look at it and keep you informed.
And yes, you can include then in MCR!

woohoo this is getting fancy want rsync too

Edited by kendon, 13 January 2010 - 04:07 PM.

[pulser]

kendon, on Jan 13 2010, 15:59, said:

woohoo this is getting fancy want rsync to

This is getting pretty interesting. So our phones are becoming feasibly linux terminals right now (well at least via. adb/ssh)

I take it none of these would work through an on-device emulator like Better Terminal or ConnectBot (the latter obviously being an SSH client)?

[kendon]

freeasabeer, on Jan 13 2010, 15:34, said:

Now thinking of the best way to provide it without running off topic from the 4th post on  
Not sure opening a new topic is the best one ? I'll attach it to this post for this time but definitively not the best way though.

i'd suggest renaming this thread or creating a new one named "unix tools for android"...

[kendon]

pulser, on Jan 13 2010, 17:08, said:

This is getting pretty interesting. So our phones are becoming feasibly linux terminals right now (well at least via. adb/ssh)

I take it none of these would work through an on-device emulator like Better Terminal or ConnectBot (the latter obviously being an SSH client)?

just tested tree in connectbot, works great.

[kendon]

next one: wget. not necessary, but could come in handy...
sorry for hijacking this thread, but this is way to cool to let it die

[intronauta]

kendon, on Jan 13 2010, 17:28, said:

i'd suggest renaming this thread or creating a new one named "unix tools for android"...

+1!

Awesome  

Love these tools!

[freeasabeer]

if we add too many linux tools then at some point we may ask ourself:

- why not go the "debian on android" route ?

- as currently all these binaries are statically linked to avoid any dependencies to other libs, wouldn't it be preferable to follow a similar way to the "debian on android" route and set up a chroot with all our beloved linux tools which can then be dynamically linked and hence save some space ?

I do not have the answer *yet* and do not want to push one particular option at this stage. But we need to think about it.

If we keep with a reasonable number of tools (let say not more than 10) then I think we can keep the static binaries approach.

[PaulOBrien]

I'm not expert enough to understand the implications of this... elaborate?

P

[pulser]

Paul, on Jan 13 2010, 17:28, said:

I'm not expert enough to understand the implications of this... elaborate?

P

+1.

I appreciate these are good things, but other than fooling about with on a quiet afternoon, I can't really figure out a use of this.

Not that I'm criticising this, in fact I'm actually about to start downloading and giving these a shot So, don't take this as negative. I'm pretty impressed we're getting these kind of things running on the Hero.
Just that I don't quite know why these things will be useful. I'm sure someone can enlighten us

[steveh2]

The point he's making is that with no shared library use each one can be large, both on disk and in memory.

However, even with a large list available to the cooker, most people will just use a few of them in a rom I'd guess.

[fyremoon]

freeasabeer, on Jan 13 2010, 12:29, said:

An interesting side effect I've noticed is that vi works great within a screen shell where as is was unusable from a direct shell !

vi doesn't work on non visual sessions. Visual mode gives you the ability to mark text for cut, copy (yank) and paste.

ed and ex are both non visual editors. ed spawned sed (which are both part of busybox) but ex isn't included in busybox for some reason (its usually part of vi) so I've emailed the busybox team to ask why.

Edited by fyremoon, 13 January 2010 - 07:45 PM.

[fyremoon]

kendon, on Jan 13 2010, 16:36, said:

next one: wget. not necessary, but could come in handy...
sorry for hijacking this thread, but this is way to cool to let it die

wget is actually included in busybox, so there won't be any need for a separate binary.

Open up a shell and type busybox and you will be presented by the number of compiled in programs ready for use.

[PaulOBrien]

vi? Pah! Nano ftw!

P

[fyremoon]

Its amazing what is in busybox.

If you want to run an FTP server or a web server, you can from busybox. It will require a change to the init scripts so that inetd starts (also part of busybox) and an entry in /etc/inetd.conf

I know what would be nice would be a PHP binary or even Perl, so I can write a web app that uses sqlite database so I don't have to rely on one running on a distant server.