Multitouch possible in theory?

[Guest xangma]

OK, try this. APK inside the zip.

I've targeted Android 1.5, but the MotionEvent passed to OnTouch does not include the getPoints() method that 2.x uses for multitouch. So, it could be that we must track the MOTION_DOWN / MOTION_UP events to see if there is a rectangle selected. However, because it's late, I can't see how you'd tell which finger had moved other than a heuristic that says "the one closest to the last event seen".

Anyway, give it a try. Once installed and running, touch the screen, and then touch somewhere else. You'd be looking for two "DOWN" events, with no intermediate "UP" events. You get the idea. Clear the log from the menu "Clear" button.

Let me know if I was supposed to target 1.6 or higher.

Dave

EDIT: Only been able to test in the emulator, since my Pulse is broken, have I mentioned that already?

Testing now, how did you break your pulse?

EDIT: yeah so it doesnt work. It took me a while to get my head around it. Useful program. Only registers one =[

Edited February 24, 2010 by xangma

[Guest david_dawkins]

Testing now, how did you break your pulse?

It looks like the USB socket came loose from the mainboard. It's in for repair.

[Guest david_dawkins]

> EDIT: yeah so it doesnt work

Arse. Never mind. It was fun for me anyway.

[Guest xangma]

It looks like the USB socket came loose from the mainboard. It's in for repair.

not cool.

Your app works =P I mean the multitouch. It will be useful soon when the new rom comes out. Thanks =]

Edited February 24, 2010 by xangma

[Guest david_dawkins]

not cool.

Your app works =P I mean the multitouch. It will be useful soon when the new rom comes out. Thanks =]

I assumed you meant the multitouch, the "arse" was in sympathy for all the effort you'd put in :( I will research

code examples for Android 1.5 multitouch and make sure I didn't miss anything.

When the new ROM is out, my app will need recoding slightly I think, to show the getPoints() stuff too. The UP/DOWN

tracking may still work too.

I have to crash now. later.

[Guest BigBearMDC]

Hi gyus

I read on the blog from the G1 MR project that theres a framework that handles the touch inputs and drops the information of the 2nd touch. I don't remember the file now, but I'll look for it in the afternoon nad inform you then :-)

Greetings,

BigBear

[Guest xangma]

Hi gyus

I read on the blog from the G1 MR project that theres a framework that handles the touch inputs and drops the information of the 2nd touch. I don't remember the file now, but I'll look for it in the afternoon nad inform you then :-)

Greetings,

BigBear

That might be the key. Will have a look later =]

[Guest david_dawkins]

That might be the key. Will have a look later =]

I'll update the app if you find anything, let me know

[Guest BigBearMDC]

Found it :(

Full Working MultiTouch on the T-Mobile G1 Android Phone

10 January 2009 by lukehutch

The real story behind multitouch

(including screenshots, video and working code for functional multitouch on the G1)

Short story: I have full multitouch scaling and panning working in specially-developed apps on a stock T-Mobile G1 Android phone with a change to just one system classfile (i.e. with no modifications to the kernel whatsoever).

MultiTouch running on the G1 without kernel modification (red and green circles are drawn where touch points are detected)

MultiTouch running on the G1 without kernel modification (red and green circles are drawn where touch points are detected)

Long story: read on for full details, including a video of the action, and full source code so that you can run this yourself (assuming you are a developer and understand the risks of doing this — this is NOT yet for end-users).

Shameless plug: if you like or use this multi-touch work for Android, please donate to support continued development of awesome features for Android!

Please donate to support continued development of awesome features for Android!

For those with ADD or that don’t want to read the gory details, you can just watch the video on YouTube (it is also embedded below).

THE GORY DETAILS

Touch screens and tinfoil hats

When the T-Mobile G1 / HTC Dream was released, it only supported single-touch rather than iPhone-style multitouch. Theories as to the lack of multitouch included hardware limitations, software support for it not being ready in the Android stack, and the threat of being devoured by Apple’s patent lawyers. Dan Morrill, a Google developer advocate for Android, made statements that the device was single-touch and the Android stack had no support yet for multitouch, but that Google would be willing to work together with handset manufacturers to develop multitouch software support when the hardware manufacturers were ready to release a multitouch handset. Eventually even one of HTC’s chiefs chimed in that the Dream was only ever designed to be a single-touch device.

Recently though, videos started surfacing on the net that showed various experiments people were performing on ListViews with two fingers that seemed to indicate the screen supported multiple touchpoints — however the results of these tests were still pretty inconclusive. Finally though, after the source of the Android stack was released, a developer Ryan Gardner / RyeBrye posted on his blog that he had managed to locate some lines in the kernel driver that were commented out that indicated that multitouch was indeed possible on these devices — and he hacked together a demo of two-fingered drawing that proved it.

To use RyeBrye’s solution, you have to recompile your phone’s kernel. It works by removing the comments around some debug statements (lines 132-151 of the the Synaptics I2C driver, synaptics_i2c_rmi.c) that dump motion events out to a logfile. He then wrote a user interface to read the logfile and draw dots on the screen.

Google, of course, continued to remain silent on the multitouch issue, and conspiracy theories grew thicker…

Enabling multitouch on the G1, the real way

RyeBrye did a great service to the Android hacker community by demonstrating that the screen is multitouch-capable. However there are some real limitations to his approach (which he fully acknowledged), such as having to recompile your kernel and having to get at the events by parsing a logfile. Also it looks like nobody yet has picked up the ball and turned his work into a working system.

Actually, it turns out that if you read a little further down in the driver code (lines 187-200 of synaptics_i2c_rmi.c), you’ll notice that you don’t need to recompile your kernel at all to get multitouch working on the G1 — the kernel driver in fact already emits multitouch information! The driver emits ABS_X, ABS_Y and BTN_TOUCH values for position and up/down information for the first touchpoint, but also emits ABS_HAT0X, ABS_HAT0Y and BTN_2 events for the second touchpoint. Where are these events getting lost then?

I pulled apart the Android stack and scoured it for the location where these events are passed through to Dalvik through JNI. It turned out to be very difficult pinpoint where input events were getting received MotionEvent objects populated (because they are processed on an event queue, the objects are recycled rather than created, and it happens in non-SDK code — egrep wasn’t much help either). The exact point at which multitouch information is lost though turns out to be $ANDROID_HOME/frameworks/base/services/java/com/android/server/KeyInputQueue.java. This class is the only code running on Dalvik that ever gets to see the raw device events — and it promptly discards ABS_HAT0X, ABS_HAT0Y and BTN_2. (It doesn’t seem to do so intentionally or maliciously, it just ignores anything it doesn’t recognize, and it is not coded to recognize those event symbol types.)

Link

Greetings,

BigBear

Edited February 25, 2010 by BigBearMDC

[Guest xangma]

Found it :(

Link

Greetings,

BigBear

Just got that now : $ANDROID_HOME/frameworks/base/services/java/com/android/server/KeyInputQueue.java .

In there ... http://android.git.kernel.org/?p=platform/...296db0b;hb=HEAD

SO! Who's good at java programming? We need that framework thing redone with that extra code in it. Who's up for it? I'll see what I can do soon ...

[Guest BigBearMDC]

Just got that now : $ANDROID_HOME/frameworks/base/services/java/com/android/server/KeyInputQueue.java .

In there ... http://android.git.kernel.org/?p=platform/...296db0b;hb=HEAD

SO! Who's good at java programming? We need that framework thing redone with that extra code in it. Who's up for it? I'll see what I can do soon ...

Me! :(

Just give me the code and I'll see what I can do.

Edit: I should have looked to the link first :(

Greetings,

BigBear

Edited February 25, 2010 by BigBearMDC

[Guest BigBearMDC]

Woohoo :(

// Process position events from multitouch protocol.

						} else if (ev.type == RawInputEvent.EV_ABS &&

								(classes&RawInputEvent.CLASS_TOUCHSCREEN_MT) != 0) {

							if (ev.scancode == RawInputEvent.ABS_MT_TOUCH_MAJOR) {

								di.mAbs.changed = true;

								di.mAbs.mNextData[di.mAbs.mAddingPointerOffset

										+ MotionEvent.SAMPLE_PRESSURE] = ev.value;

							} else if (ev.scancode == RawInputEvent.ABS_MT_POSITION_X) {

								di.mAbs.changed = true;

								di.mAbs.mNextData[di.mAbs.mAddingPointerOffset

									+ MotionEvent.SAMPLE_X] = ev.value;

								if (DEBUG_POINTERS) Log.v(TAG, "MT @"

										+ di.mAbs.mAddingPointerOffset

										+ " X:" + ev.value);

							} else if (ev.scancode == RawInputEvent.ABS_MT_POSITION_Y) {

								di.mAbs.changed = true;

								di.mAbs.mNextData[di.mAbs.mAddingPointerOffset

									+ MotionEvent.SAMPLE_Y] = ev.value;

								if (DEBUG_POINTERS) Log.v(TAG, "MT @"

										+ di.mAbs.mAddingPointerOffset

										+ " Y:" + ev.value);

							} else if (ev.scancode == RawInputEvent.ABS_MT_WIDTH_MAJOR) {

								di.mAbs.changed = true;

								di.mAbs.mNextData[di.mAbs.mAddingPointerOffset

									+ MotionEvent.SAMPLE_SIZE] = ev.value;

							}


						// Process position events from single touch protocol.

						} else if (ev.type == RawInputEvent.EV_ABS &&

								(classes&RawInputEvent.CLASS_TOUCHSCREEN) != 0) {

							if (ev.scancode == RawInputEvent.ABS_X) {

								di.mAbs.changed = true;

								di.curTouchVals[MotionEvent.SAMPLE_X] = ev.value;

							} else if (ev.scancode == RawInputEvent.ABS_Y) {

								di.mAbs.changed = true;

								di.curTouchVals[MotionEvent.SAMPLE_Y] = ev.value;

							} else if (ev.scancode == RawInputEvent.ABS_PRESSURE) {

								di.mAbs.changed = true;

								di.curTouchVals[MotionEvent.SAMPLE_PRESSURE] = ev.value;

								di.curTouchVals[MotionEvent.NUM_SAMPLE_DATA

												 + MotionEvent.SAMPLE_PRESSURE] = ev.value;

							} else if (ev.scancode == RawInputEvent.ABS_TOOL_WIDTH) {

								di.mAbs.changed = true;

								di.curTouchVals[MotionEvent.SAMPLE_SIZE] = ev.value;

								di.curTouchVals[MotionEvent.NUM_SAMPLE_DATA

												 + MotionEvent.SAMPLE_SIZE] = ev.value;

							}


						// Process movement events from trackball (mouse) protocol.

						} else if (ev.type == RawInputEvent.EV_REL &&

								(classes&RawInputEvent.CLASS_TRACKBALL) != 0) {

							// Add this relative movement into our totals.

							if (ev.scancode == RawInputEvent.REL_X) {

								di.mRel.changed = true;

								di.mRel.mNextData[MotionEvent.SAMPLE_X] += ev.value;

							} else if (ev.scancode == RawInputEvent.REL_Y) {

								di.mRel.changed = true;

								di.mRel.mNextData[MotionEvent.SAMPLE_Y] += ev.value;

							}

						}


						// Handle multitouch protocol sync: tells us that the

						// driver has returned all data for -one- of the pointers

						// that is currently down.

						if (ev.type == RawInputEvent.EV_SYN

								&& ev.scancode == RawInputEvent.SYN_MT_REPORT

								&& di.mAbs != null) {

							di.mAbs.changed = true;

							if (di.mAbs.mNextData[MotionEvent.SAMPLE_PRESSURE] > 0) {

								// If the value is <= 0, the pointer is not

								// down, so keep it in the count.


								if (di.mAbs.mNextData[di.mAbs.mAddingPointerOffset

													  + MotionEvent.SAMPLE_PRESSURE] != 0) {

									final int num = di.mAbs.mNextNumPointers+1;

									di.mAbs.mNextNumPointers = num;

									if (DEBUG_POINTERS) Log.v(TAG,

											"MT_REPORT: now have " + num + " pointers");

									final int newOffset = (num <= InputDevice.MAX_POINTERS)

											? (num * MotionEvent.NUM_SAMPLE_DATA)

											: (InputDevice.MAX_POINTERS *

													MotionEvent.NUM_SAMPLE_DATA);

									di.mAbs.mAddingPointerOffset = newOffset;

									di.mAbs.mNextData[newOffset

											+ MotionEvent.SAMPLE_PRESSURE] = 0;

								} else {

									if (DEBUG_POINTERS) Log.v(TAG, "MT_REPORT: no pointer");

								}

							}[/code]

[Guest robot1000]

That link posted above has been online for 13 months!

[Guest BigBearMDC]

That link posted above has been online for 13 months!

Which one?

Greetings,

BigBear

[Guest xangma]

That link posted above has been online for 13 months!

Well lets get on it then! haha

[Guest BigBearMDC]

Well lets get on it then! haha

Well, the G1 has multitouch since then :(

Greetings,

BigBear

[Guest xangma]

Looks like we can't use this code straight off the bat. The things are defined differently and it imports completely different files. We need to know how it was actually implemented in the G1 instead of just pinching code from 2.1 or anything.

[Guest BigBearMDC]

Looks like we can't use this code straight off the bat. The things are defined differently and it imports completely different files. We need to know how it was actually implemented in the G1 instead of just pinching code from 2.1 or anything.

Yap I didn't think that either. It would be great if we could get the file from the Pulse ROM... or maybe compare it with the file from android 1.5 source?

Hmm anyway think positive! WE WILL GET MULTITOUCH WORKING!

:(

Greetings,

BigBear

[Guest xangma]

We are only a step away.

Huawei customised the synaptics driver cause they're numptys. If we want this to work ... we need to read this: http://lukehutch.wordpress.com/2009/01/25/...obile-g1-today/

And then look at this:

http://go2.wordpress.com/?id=725X1342&...KernelPatch.zip

Which is a kernel patch for the ORIGINAL synaptics driver.

We need to implement this code in the MODIFIED synaptics driver, and we're laughing. I however, am a dumbass and cannot do this ...

EDIT: To prove my dumbass theory, I only recently found out htat I was using mkbootfs wrong and that's why my kernels weren't booting. They all still don't boot, but the ones that actually should boot (2.6.27) now do.

I now use

 mkbootimg --cmdline "mem=128M console=ttyMSM2,115200n8 no_console_suspend=1" --kernel KERNEL1/zImage --ramdisk newramdisk.cpio.gz -o mynewimage.img

Edited February 25, 2010 by xangma

[Guest BigBearMDC]

We are only a step away.

Huawei customised the synaptics driver cause they're numptys. If we want this to work ... we need to read this: http://lukehutch.wordpress.com/2009/01/25/...obile-g1-today/

And then look at this:

http://go2.wordpress.com/?id=725X1342&...KernelPatch.zip

Which is a kernel patch for the ORIGINAL synaptics driver.

We need to implement this code in the MODIFIED synaptics driver, and we're laughing. I however, am a dumbass and cannot do this ...

EDIT: To prove my dumbass theory, I only recently found out htat I was using mkbootfs wrong and that's why my kernels weren't booting. They all still don't boot, but the ones that actually should boot (2.6.27) now do.

I now use

 mkbootimg --cmdline "mem=128M console=ttyMSM2,115200n8 no_console_suspend=1" --kernel KERNEL1/zImage --ramdisk newramdisk.cpio.gz -o mynewimage.img

Well done!

But I think we will need the patched KeyInputQueue too.

I'm trying to get the already patched one - no success yet :(

If your boot.img is readys, let me know :(

Edit:

If you are a dumbass, we all are as intelligent as a piece of bread :(

Greetings,

BigBear

Edited February 25, 2010 by BigBearMDC

[Guest xangma]

Well done!

But I think we will need the patched KeyInputQueue too.

I'm trying to get the already patched one - no success yet :(

If your boot.img is readys, let me know :(

Edit:

If you are a dumpass, we all are as intelligent as a piece of bread :(

Greetings,

BigBear

My boot.img can be ready whenever you want. I just don't have the skill to put that extra code into the 1319 driver or whatever it is we use. I need a modified driver =P

[Guest BigBearMDC]

My boot.img can be ready whenever you want. I just don't have the skill to put that extra code into the 1319 driver or whatever it is we use. I need a modified driver =P

Hold on a sec :(

[Guest BigBearMDC]

Here you are :(

I hope it works!

Greetings,

BigBear

synaptics_i2c_rmi.c.zip

Edited February 25, 2010 by BigBearMDC

[Guest xangma]

Here you are :(

I hope it works!

Greetings,

BigBear

No, that's the unmodified one. Sorry =[

The one we use is in /drivers/input/touchscreen/synap...

I'll attach it.

synaptics_i2c_rmi_tm1319.zip

[Guest BigBearMDC]

No, that's the unmodified one. Sorry =[

The one we use is in /drivers/input/touchscreen/synap...

I'll attach it.

Sorry, I missed that.

Hold on a further sec :(