Jump to content


Photo

linux tools coming to MCR

- - - - -

  • Please log in to reply
215 replies to this topic

#1
freeasabeer

freeasabeer

    Regular

  • Members
  • PipPip
  • 88 posts
  • Location:Paris, France
  • Devices:HTC Hero
(for reference, topic previous name was: screen coming to MCR and have also merge the "mc coming to MCR" topic)

8-feb: Major update in the way these additional toys are handled (now with ipkg feed).

I will now publish the linux tools packaged as ipk files and one will have to use the opkg tool to install / uninstall / udate these ipk packages.
For reference, opkg is a lightweight package management system based on Ipkg. Ok, but what is ipkg ? Easy: ipkg re-implements for embedded systems the functionality of Debian tools dpkg, apt, and apt-get. For example, it uses much less disk space for the package metadata.
Believe me, this will make your life as end-user quiet easier and as a side effect it will make my life as developer & packager quite easier as well B)

So, having said that; the first action is to have opkg installed on your phone. To achieve this, download this specific opkg bootstrap Attached File  opkg_bootstrap_v2.tar.bz2.zip   262.9KB   1288 downloads.
Unzip it to your SD card, open a shell with adb and run:
cd /
tar xjf /sdcard/opkg-bootstrap-v2.tar.bz2
/data/xbin/opkg install /sdcard/opkg_0.1.7+svn519-2_arm.ipk
You now have installed opkg as an ipk package. This means:
1) you are now able to easily install other ipk
2) opkg can now also be easily updated if required !

One of the benefitis of opkg is the ability to set-up an internet feed so as to be able to update or install packages while being online. Edit opkg configuration file /data/local/opkg.conf in this way:
src/gz common http://freeasabeer.byethost33.com/android/feed/common
src/gz mcr http://freeasabeer.byethost33.com/android/feed/mcr
#src/gz cyanogen http://freeasabeer.byethost33.com/android/feed/cyanogen

dest root /

#option verbose-wget
#
# Proxy support:
#
#option http_proxy http://localhost:5865
#option ftp_proxy http://proxy.tld:3128
#option proxy_username <username>
#option proxy_password <password>

Note:
If you're using Paul's MCR, then those settings are OK.
If you're using one of the 2.1 cyanogen based ROM, then comment out the second line and uncomment the third one.


You are now ready to update the package list and to install from the feed:

opkg update

Note 2:
If you already had opkg installed from the previous bootstrap, then you will have to do the following (as there is a bug in former opkg package that prevent it to actually find the configuration file):
opkg -f /data/local/opkg.conf update
opkg install opkg


Some quick instructions on how to use opkg:

Always run "opkg update" before to be sure to get the latest updates ;)

Install the rsync package:
opkg install rsync

Want to remove the package, easy:
opkg remove rsync

Want to list installed packages:
opkg list_installed

Want to list files belonging to a given package (e.g. rsync):
opkg files rsync

You don't need to reboot your phone in recovery mode, neither to just reboot it. Just install things live.


PS: you don't need to uninstall or remove any of the previously installed tools: they will (should) be nicely overwritten.

Forewords:

these linux tools are mainly intended for geeks that like top play with the internals of their Hero from a shell (either adb shell, ssh connection, or a terminal application). Regular users won't find any benefit by installing these tools: they won't make their hero faster or work better, they will just increase the ROM memory consumption.



1. mc (Midnight Commander)

If you don't know what mc is then you most probably don't need it. But you may be interested to discover what it is.

For the geeks that do want it to easily browse the phone file system from a ssh or adb shell connection, I've done a uclibc statically linked binary of mc that you can install with opkg.

From what I've seen, it mostly works ok from a linux xterm connected with adb shell: browsing files is ok and I've not tested any advanced features of mc. I have no idea if it is usable from a windows cmd terminal or a cygwin one.
Once installed, open a shell on your phone and start mc this way: mc



2. GNU screen

As a midnight commander companion, we can now enjoy screen !

Don't know what screen is ? Check Wikipedia.

Basically screen allow to open several shell connections in parallel from one physical connection to the phone (either adb shell or ssh) and to switch from one to the other one at will. It also allow to close the physical connection without losing the ongoing shell connections so that you can resume them once you have restarted the physical connection.

An interesting side effect I've noticed is that vi works great within a screen shell where as is was unusable from a direct shell !

I have however to raise a warning here: in order to get screen to work I had to setup a system/etc/passwd file with the solely purpose to declare the root user and I've done it with an empty password. So I'm not sure yet of the security implication that this could have on the phone... On a regular unix system the security breach is obvious but here on an anyway rooted phone I'm not so sure. So, you've been warned.

Once installed, you can start screen with: screen and resume an ongoing session with screen -r



3. tree

Tree is a recursive directory listing program that produces a depth indented listing of files, which is colorized ala dircolors if the LS_COLORS environment variable is set and output is to tty.



4. rsync

Due to popular demand, here is also rsync...
Note: installing rsync will also install a new version of dropbear.
Some hints on how to use rsync with the Hero:

1) From the Hero to a remote host
In this scenario, you run rsync on the hero itself and connect to a remote host.
There are no oddities here, just use rsync as you normally would.

2) From a linux/mac host to a remote Hero
In this scenario, you rsync on linux/mac host and connect to a remote hero.
In this case, the rsync invocation is a bit tricky, for example:
rsync -av -e 'ssh -p 2222' --rsync-path=/data/xbin/rsync <some local files/dir> root@<YOU_HERO_IP>:/data/tmp



5. busybox

This is an updated busybox as compared to what Paul currently ships with MCR (even current betas). This one do include my ip resolv tweak !



6. wget

Don't ask for wget, it is already provided by busybox :)



7. htop

First release for htop... However I don't really see how to use the Fx keys... So pretty useless ?



8. zip

The famous zip binary is now available !



Enjoy !

Edited by freeasabeer, 11 March 2010 - 09:09 AM.

  • 0

#2
freeasabeer

freeasabeer

    Regular

  • Members
  • PipPip
  • 88 posts
  • Location:Paris, France
  • Devices:HTC Hero
Changelog:

5-feb-2010: Major update
- Make use of opkg tool and of the .ipk package format to handle the tools

3-feb-2010: screen 0.3
- Regression fix in screen wrapper script

3-feb-2010: busybox 0.3
- Bug fix for the update script of busybox 1.16.0

29-jan-2010: busybox 0.2
- busybox 1.16.0 packged for modaco with the ip name resol fix.

29-jan-2010: zip 0.1
- first release for the famous zip binary

25-jan-2010: htop 0.1
- first release for htop. (Within ConnectBot, the Fn key is emulated with: Trackball + n)

21-jan-2010: screen 0.2
- made it more MCR friendly: recompiled to use /data/screen as its "data" folder where the global screenrc file can be found
- addition of the uft8encodings in /data/screen
- uses now /data/local/tmp as the temporary folder instead of /data/tmp

18-jan-2010: busybox 0.1

Modifications to Paul's busybox
- crond: spool folder is set to /data/cron by default
- ls color enabled by default
- fixed the ip name resolve bug. Requires a valid /etc/resolv.conf file.
The update package also make /etc/resolv.conf a symlink to /data/local/resolv.conf and provides a valid /data/local/resolv.conf file (Google DNS).
A simple script that reads net.dns1 property and updates /data/local/resolv.conf accordingly is also provided in /data/xbin/set_resolconf.

15-jan-2010: rsync 0.3
- fixed issue in the update-script that prevented installation in the case rsync was already installed
- lesson to learn: don't publish an update too late in the night :)

15-jan-2010: rsync 0.2
- dropbear rebuild to fix non working scp
- /system/bin/dropbear.sh script updated to allow use of public keys in /data/dropbear/.ssh/authorized_keys

Edited by freeasabeer, 05 February 2010 - 10:27 PM.

  • 0

#3
kendon

kendon

    Addict

  • Members
  • PipPipPipPipPip
  • 923 posts
  • Devices:HTC Hero, ZTE Blade
this is so f***ing awesome! thanks!

do you take requests? i'd be interested in tree aswell... B)
and i am sure i can find more, but i don't have the skills to make them work...

edit: needed to get a superlong hyperlink on my phone, abused the forum for this :)

Edited by kendon, 13 January 2010 - 01:49 PM.

  • 0

#4
freeasabeer

freeasabeer

    Regular

  • Members
  • PipPip
  • 88 posts
  • Location:Paris, France
  • Devices:HTC Hero
Done !
Now thinking of the best way to provide it without running off topic from the 4th post on :)
Not sure opening a new topic is the best one ? I'll attach it to the first post of this topic. 

Edited by freeasabeer, 13 January 2010 - 10:05 PM.

  • 0

#5
PaulOBrien

PaulOBrien

    It's My Party

  • Founder
  • PipPipPipPipPipPip
  • 36,221 posts
  • Gender:Male
  • Location:Norwich, UK
  • Devices:All the Nexus!
  • Twitter:@paulobrien
Is this a static compilation?

I can include these in future MCR builds (along with nano, busybox) etc.

I have a request... rsync... :)

P

  • 0

You can follow me on Twitter - http://twitter.com/paulobrien / Follow MoDaCo on Twitter - http://twitter.com/modaco

Want to donate? MoDaCo is raising money for the Multiple Sclerosis society.

Posted Image


#6
freeasabeer

freeasabeer

    Regular

  • Members
  • PipPip
  • 88 posts
  • Location:Paris, France
  • Devices:HTC Hero
Yes they are all static binaries against uclibc.
rsync, that's a pretty good suggestion ! I'll have a look at it and keep you informed.
And yes, you can include then in MCR!

  • 0

#7
kendon

kendon

    Addict

  • Members
  • PipPipPipPipPip
  • 923 posts
  • Devices:HTC Hero, ZTE Blade

Yes they are all static binaries against uclibc.
rsync, that's a pretty good suggestion ! I'll have a look at it and keep you informed.
And yes, you can include then in MCR!

woohoo this is getting fancy :) want rsync too :)

Edited by kendon, 13 January 2010 - 04:07 PM.

  • 0

#8
pulser

pulser

    Diehard

  • Members
  • PipPipPipPip
  • 432 posts
  • Gender:Male
  • Location:/dev/null

woohoo this is getting fancy :) want rsync to :)


This is getting pretty interesting. So our phones are becoming feasibly linux terminals right now (well at least via. adb/ssh)

I take it none of these would work through an on-device emulator like Better Terminal or ConnectBot (the latter obviously being an SSH client)?

  • 0

#9
kendon

kendon

    Addict

  • Members
  • PipPipPipPipPip
  • 923 posts
  • Devices:HTC Hero, ZTE Blade

Now thinking of the best way to provide it without running off topic from the 4th post on :)
Not sure opening a new topic is the best one ? I'll attach it to this post for this time but definitively not the best way though.

i'd suggest renaming this thread or creating a new one named "unix tools for android"...

  • 0

#10
kendon

kendon

    Addict

  • Members
  • PipPipPipPipPip
  • 923 posts
  • Devices:HTC Hero, ZTE Blade

This is getting pretty interesting. So our phones are becoming feasibly linux terminals right now (well at least via. adb/ssh)

I take it none of these would work through an on-device emulator like Better Terminal or ConnectBot (the latter obviously being an SSH client)?

just tested tree in connectbot, works great.

  • 0

#11
kendon

kendon

    Addict

  • Members
  • PipPipPipPipPip
  • 923 posts
  • Devices:HTC Hero, ZTE Blade
next one: wget. not necessary, but could come in handy...
sorry for hijacking this thread, but this is way to cool to let it die :)

  • 0

#12
intronauta

intronauta

    Newbie

  • Members
  • Pip
  • 17 posts
  • Devices:HTC HERO

i'd suggest renaming this thread or creating a new one named "unix tools for android"...


+1!

Awesome :)

Love these tools!

  • 0

#13
freeasabeer

freeasabeer

    Regular

  • Members
  • PipPip
  • 88 posts
  • Location:Paris, France
  • Devices:HTC Hero
if we add too many linux tools then at some point we may ask ourself:

- why not go the "debian on android" route ?

- as currently all these binaries are statically linked to avoid any dependencies to other libs, wouldn't it be preferable to follow a similar way to the "debian on android" route and set up a chroot with all our beloved linux tools which can then be dynamically linked and hence save some space ?

I do not have the answer *yet* and do not want to push one particular option at this stage. But we need to think about it.

If we keep with a reasonable number of tools (let say not more than 10) then I think we can keep the static binaries approach.

  • 0

#14
PaulOBrien

PaulOBrien

    It's My Party

  • Founder
  • PipPipPipPipPipPip
  • 36,221 posts
  • Gender:Male
  • Location:Norwich, UK
  • Devices:All the Nexus!
  • Twitter:@paulobrien
I'm not expert enough to understand the implications of this... elaborate?

P

  • 0

You can follow me on Twitter - http://twitter.com/paulobrien / Follow MoDaCo on Twitter - http://twitter.com/modaco

Want to donate? MoDaCo is raising money for the Multiple Sclerosis society.

Posted Image


#15
pulser

pulser

    Diehard

  • Members
  • PipPipPipPip
  • 432 posts
  • Gender:Male
  • Location:/dev/null

I'm not expert enough to understand the implications of this... elaborate?

P


+1.

I appreciate these are good things, but other than fooling about with on a quiet afternoon, I can't really figure out a use of this.

Not that I'm criticising this, in fact I'm actually about to start downloading and giving these a shot :) So, don't take this as negative. I'm pretty impressed we're getting these kind of things running on the Hero.
Just that I don't quite know why these things will be useful. I'm sure someone can enlighten us :)

  • 0

#16
steveh2

steveh2

    Newbie

  • Members
  • Pip
  • 14 posts
  • Devices:HTC Hero
The point he's making is that with no shared library use each one can be large, both on disk and in memory.

However, even with a large list available to the cooker, most people will just use a few of them in a rom I'd guess.

  • 0

#17
fyremoon

fyremoon

    Regular

  • MoDaCo Silver
  • PipPip
  • 97 posts
  • Location:St Leonards, UK
  • Devices:HTC Hero/T-mobile G2
  • Twitter:@fyremoon

An interesting side effect I've noticed is that vi works great within a screen shell where as is was unusable from a direct shell !


vi doesn't work on non visual sessions. Visual mode gives you the ability to mark text for cut, copy (yank) and paste.

ed and ex are both non visual editors. ed spawned sed (which are both part of busybox) but ex isn't included in busybox for some reason (its usually part of vi) so I've emailed the busybox team to ask why. :)

Edited by fyremoon, 13 January 2010 - 07:45 PM.

  • 0

#18
fyremoon

fyremoon

    Regular

  • MoDaCo Silver
  • PipPip
  • 97 posts
  • Location:St Leonards, UK
  • Devices:HTC Hero/T-mobile G2
  • Twitter:@fyremoon

next one: wget. not necessary, but could come in handy...
sorry for hijacking this thread, but this is way to cool to let it die :)


wget is actually included in busybox, so there won't be any need for a separate binary. :)

Open up a shell and type busybox and you will be presented by the number of compiled in programs ready for use.

  • 0

#19
PaulOBrien

PaulOBrien

    It's My Party

  • Founder
  • PipPipPipPipPipPip
  • 36,221 posts
  • Gender:Male
  • Location:Norwich, UK
  • Devices:All the Nexus!
  • Twitter:@paulobrien
vi? Pah! Nano ftw! :)

P

  • 0

You can follow me on Twitter - http://twitter.com/paulobrien / Follow MoDaCo on Twitter - http://twitter.com/modaco

Want to donate? MoDaCo is raising money for the Multiple Sclerosis society.

Posted Image


#20
fyremoon

fyremoon

    Regular

  • MoDaCo Silver
  • PipPip
  • 97 posts
  • Location:St Leonards, UK
  • Devices:HTC Hero/T-mobile G2
  • Twitter:@fyremoon
Its amazing what is in busybox.

If you want to run an FTP server or a web server, you can from busybox. It will require a change to the init scripts so that inetd starts (also part of busybox) and an entry in /etc/inetd.conf

I know what would be nice would be a PHP binary or even Perl, so I can write a web app that uses sqlite database so I don't have to rely on one running on a distant server.

  • 0




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users