Jump to content

Multitouch possible in theory?


Guest lemmyc

Recommended Posts

OK, try this. APK inside the zip.

I've targeted Android 1.5, but the MotionEvent passed to OnTouch does not include the getPoints() method that 2.x uses for multitouch. So, it could be that we must track the MOTION_DOWN / MOTION_UP events to see if there is a rectangle selected. However, because it's late, I can't see how you'd tell which finger had moved other than a heuristic that says "the one closest to the last event seen".

Anyway, give it a try. Once installed and running, touch the screen, and then touch somewhere else. You'd be looking for two "DOWN" events, with no intermediate "UP" events. You get the idea. Clear the log from the menu "Clear" button.

Let me know if I was supposed to target 1.6 or higher.

Dave

EDIT: Only been able to test in the emulator, since my Pulse is broken, have I mentioned that already?

Testing now, how did you break your pulse?

EDIT: yeah so it doesnt work. It took me a while to get my head around it. Useful program. Only registers one =[

Edited by xangma
Link to comment
Share on other sites

It looks like the USB socket came loose from the mainboard. It's in for repair.

not cool.

Your app works =P I mean the multitouch. It will be useful soon when the new rom comes out. Thanks =]

Edited by xangma
Link to comment
Share on other sites

Guest david_dawkins
not cool.

Your app works =P I mean the multitouch. It will be useful soon when the new rom comes out. Thanks =]

I assumed you meant the multitouch, the "arse" was in sympathy for all the effort you'd put in :( I will research

code examples for Android 1.5 multitouch and make sure I didn't miss anything.

When the new ROM is out, my app will need recoding slightly I think, to show the getPoints() stuff too. The UP/DOWN

tracking may still work too.

I have to crash now. later.

Link to comment
Share on other sites

Guest BigBearMDC

Hi gyus

I read on the blog from the G1 MR project that theres a framework that handles the touch inputs and drops the information of the 2nd touch. I don't remember the file now, but I'll look for it in the afternoon nad inform you then :-)

Greetings,

BigBear

Link to comment
Share on other sites

Hi gyus

I read on the blog from the G1 MR project that theres a framework that handles the touch inputs and drops the information of the 2nd touch. I don't remember the file now, but I'll look for it in the afternoon nad inform you then :-)

Greetings,

BigBear

That might be the key. Will have a look later =]

Link to comment
Share on other sites

Guest BigBearMDC

Found it :(

Full Working MultiTouch on the T-Mobile G1 Android Phone

10 January 2009 by lukehutch

The real story behind multitouch

(including screenshots, video and working code for functional multitouch on the G1)

Short story: I have full multitouch scaling and panning working in specially-developed apps on a stock T-Mobile G1 Android phone with a change to just one system classfile (i.e. with no modifications to the kernel whatsoever).

MultiTouch running on the G1 without kernel modification (red and green circles are drawn where touch points are detected)

MultiTouch running on the G1 without kernel modification (red and green circles are drawn where touch points are detected)

Long story: read on for full details, including a video of the action, and full source code so that you can run this yourself (assuming you are a developer and understand the risks of doing this — this is NOT yet for end-users).

Shameless plug: if you like or use this multi-touch work for Android, please donate to support continued development of awesome features for Android!

Please donate to support continued development of awesome features for Android!

For those with ADD or that don’t want to read the gory details, you can just watch the video on YouTube (it is also embedded below).

THE GORY DETAILS

Touch screens and tinfoil hats

When the T-Mobile G1 / HTC Dream was released, it only supported single-touch rather than iPhone-style multitouch. Theories as to the lack of multitouch included hardware limitations, software support for it not being ready in the Android stack, and the threat of being devoured by Apple’s patent lawyers. Dan Morrill, a Google developer advocate for Android, made statements that the device was single-touch and the Android stack had no support yet for multitouch, but that Google would be willing to work together with handset manufacturers to develop multitouch software support when the hardware manufacturers were ready to release a multitouch handset. Eventually even one of HTC’s chiefs chimed in that the Dream was only ever designed to be a single-touch device.

Recently though, videos started surfacing on the net that showed various experiments people were performing on ListViews with two fingers that seemed to indicate the screen supported multiple touchpoints — however the results of these tests were still pretty inconclusive. Finally though, after the source of the Android stack was released, a developer Ryan Gardner / RyeBrye posted on his blog that he had managed to locate some lines in the kernel driver that were commented out that indicated that multitouch was indeed possible on these devices — and he hacked together a demo of two-fingered drawing that proved it.

To use RyeBrye’s solution, you have to recompile your phone’s kernel. It works by removing the comments around some debug statements (lines 132-151 of the the Synaptics I2C driver, synaptics_i2c_rmi.c) that dump motion events out to a logfile. He then wrote a user interface to read the logfile and draw dots on the screen.

Google, of course, continued to remain silent on the multitouch issue, and conspiracy theories grew thicker…

Enabling multitouch on the G1, the real way

RyeBrye did a great service to the Android hacker community by demonstrating that the screen is multitouch-capable. However there are some real limitations to his approach (which he fully acknowledged), such as having to recompile your kernel and having to get at the events by parsing a logfile. Also it looks like nobody yet has picked up the ball and turned his work into a working system.

Actually, it turns out that if you read a little further down in the driver code (lines 187-200 of synaptics_i2c_rmi.c), you’ll notice that you don’t need to recompile your kernel at all to get multitouch working on the G1 — the kernel driver in fact already emits multitouch information! The driver emits ABS_X, ABS_Y and BTN_TOUCH values for position and up/down information for the first touchpoint, but also emits ABS_HAT0X, ABS_HAT0Y and BTN_2 events for the second touchpoint. Where are these events getting lost then?

I pulled apart the Android stack and scoured it for the location where these events are passed through to Dalvik through JNI. It turned out to be very difficult pinpoint where input events were getting received MotionEvent objects populated (because they are processed on an event queue, the objects are recycled rather than created, and it happens in non-SDK code — egrep wasn’t much help either). The exact point at which multitouch information is lost though turns out to be $ANDROID_HOME/frameworks/base/services/java/com/android/server/KeyInputQueue.java. This class is the only code running on Dalvik that ever gets to see the raw device events — and it promptly discards ABS_HAT0X, ABS_HAT0Y and BTN_2. (It doesn’t seem to do so intentionally or maliciously, it just ignores anything it doesn’t recognize, and it is not coded to recognize those event symbol types.)

Link

Greetings,

BigBear

Edited by BigBearMDC
Link to comment
Share on other sites

Found it :(

Link

Greetings,

BigBear

Just got that now : $ANDROID_HOME/frameworks/base/services/java/com/android/server/KeyInputQueue.java .

In there ... http://android.git.kernel.org/?p=platform/...296db0b;hb=HEAD

SO! Who's good at java programming? We need that framework thing redone with that extra code in it. Who's up for it? I'll see what I can do soon ...

Link to comment
Share on other sites

Guest BigBearMDC
Just got that now : $ANDROID_HOME/frameworks/base/services/java/com/android/server/KeyInputQueue.java .

In there ... http://android.git.kernel.org/?p=platform/...296db0b;hb=HEAD

SO! Who's good at java programming? We need that framework thing redone with that extra code in it. Who's up for it? I'll see what I can do soon ...

Me! :(

Just give me the code and I'll see what I can do.

Edit: I should have looked to the link first :(

Greetings,

BigBear

Edited by BigBearMDC
Link to comment
Share on other sites

Guest BigBearMDC

Woohoo :(

// Process position events from multitouch protocol.
} else if (ev.type == RawInputEvent.EV_ABS &&
(classes&RawInputEvent.CLASS_TOUCHSCREEN_MT) != 0) {
if (ev.scancode == RawInputEvent.ABS_MT_TOUCH_MAJOR) {
di.mAbs.changed = true;
di.mAbs.mNextData[di.mAbs.mAddingPointerOffset
+ MotionEvent.SAMPLE_PRESSURE] = ev.value;
} else if (ev.scancode == RawInputEvent.ABS_MT_POSITION_X) {
di.mAbs.changed = true;
di.mAbs.mNextData[di.mAbs.mAddingPointerOffset
+ MotionEvent.SAMPLE_X] = ev.value;
if (DEBUG_POINTERS) Log.v(TAG, "MT @"
+ di.mAbs.mAddingPointerOffset
+ " X:" + ev.value);
} else if (ev.scancode == RawInputEvent.ABS_MT_POSITION_Y) {
di.mAbs.changed = true;
di.mAbs.mNextData[di.mAbs.mAddingPointerOffset
+ MotionEvent.SAMPLE_Y] = ev.value;
if (DEBUG_POINTERS) Log.v(TAG, "MT @"
+ di.mAbs.mAddingPointerOffset
+ " Y:" + ev.value);
} else if (ev.scancode == RawInputEvent.ABS_MT_WIDTH_MAJOR) {
di.mAbs.changed = true;
di.mAbs.mNextData[di.mAbs.mAddingPointerOffset
+ MotionEvent.SAMPLE_SIZE] = ev.value;
}

// Process position events from single touch protocol.
} else if (ev.type == RawInputEvent.EV_ABS &&
(classes&RawInputEvent.CLASS_TOUCHSCREEN) != 0) {
if (ev.scancode == RawInputEvent.ABS_X) {
di.mAbs.changed = true;
di.curTouchVals[MotionEvent.SAMPLE_X] = ev.value;
} else if (ev.scancode == RawInputEvent.ABS_Y) {
di.mAbs.changed = true;
di.curTouchVals[MotionEvent.SAMPLE_Y] = ev.value;
} else if (ev.scancode == RawInputEvent.ABS_PRESSURE) {
di.mAbs.changed = true;
di.curTouchVals[MotionEvent.SAMPLE_PRESSURE] = ev.value;
di.curTouchVals[MotionEvent.NUM_SAMPLE_DATA
+ MotionEvent.SAMPLE_PRESSURE] = ev.value;
} else if (ev.scancode == RawInputEvent.ABS_TOOL_WIDTH) {
di.mAbs.changed = true;
di.curTouchVals[MotionEvent.SAMPLE_SIZE] = ev.value;
di.curTouchVals[MotionEvent.NUM_SAMPLE_DATA
+ MotionEvent.SAMPLE_SIZE] = ev.value;
}

// Process movement events from trackball (mouse) protocol.
} else if (ev.type == RawInputEvent.EV_REL &&
(classes&RawInputEvent.CLASS_TRACKBALL) != 0) {
// Add this relative movement into our totals.
if (ev.scancode == RawInputEvent.REL_X) {
di.mRel.changed = true;
di.mRel.mNextData[MotionEvent.SAMPLE_X] += ev.value;
} else if (ev.scancode == RawInputEvent.REL_Y) {
di.mRel.changed = true;
di.mRel.mNextData[MotionEvent.SAMPLE_Y] += ev.value;
}
}

// Handle multitouch protocol sync: tells us that the
// driver has returned all data for -one- of the pointers
// that is currently down.
if (ev.type == RawInputEvent.EV_SYN
&& ev.scancode == RawInputEvent.SYN_MT_REPORT
&& di.mAbs != null) {
di.mAbs.changed = true;
if (di.mAbs.mNextData[MotionEvent.SAMPLE_PRESSURE] > 0) {
// If the value is <= 0, the pointer is not
// down, so keep it in the count.

if (di.mAbs.mNextData[di.mAbs.mAddingPointerOffset
+ MotionEvent.SAMPLE_PRESSURE] != 0) {
final int num = di.mAbs.mNextNumPointers+1;
di.mAbs.mNextNumPointers = num;
if (DEBUG_POINTERS) Log.v(TAG,
"MT_REPORT: now have " + num + " pointers");
final int newOffset = (num <= InputDevice.MAX_POINTERS)
? (num * MotionEvent.NUM_SAMPLE_DATA)
: (InputDevice.MAX_POINTERS *
MotionEvent.NUM_SAMPLE_DATA);
di.mAbs.mAddingPointerOffset = newOffset;
di.mAbs.mNextData[newOffset
+ MotionEvent.SAMPLE_PRESSURE] = 0;
} else {
if (DEBUG_POINTERS) Log.v(TAG, "MT_REPORT: no pointer");
}
}[/code]

Link to comment
Share on other sites

Looks like we can't use this code straight off the bat. The things are defined differently and it imports completely different files. We need to know how it was actually implemented in the G1 instead of just pinching code from 2.1 or anything.

Link to comment
Share on other sites

Guest BigBearMDC
Looks like we can't use this code straight off the bat. The things are defined differently and it imports completely different files. We need to know how it was actually implemented in the G1 instead of just pinching code from 2.1 or anything.

Yap I didn't think that either. It would be great if we could get the file from the Pulse ROM... or maybe compare it with the file from android 1.5 source?

Hmm anyway think positive! WE WILL GET MULTITOUCH WORKING!

:(

Greetings,

BigBear

Link to comment
Share on other sites

We are only a step away.

Huawei customised the synaptics driver cause they're numptys. If we want this to work ... we need to read this: http://lukehutch.wordpress.com/2009/01/25/...obile-g1-today/

And then look at this:

http://go2.wordpress.com/?id=725X1342&...KernelPatch.zip

Which is a kernel patch for the ORIGINAL synaptics driver.

We need to implement this code in the MODIFIED synaptics driver, and we're laughing. I however, am a dumbass and cannot do this ...

EDIT: To prove my dumbass theory, I only recently found out htat I was using mkbootfs wrong and that's why my kernels weren't booting. They all still don't boot, but the ones that actually should boot (2.6.27) now do.

I now use

 mkbootimg --cmdline "mem=128M console=ttyMSM2,115200n8 no_console_suspend=1" --kernel KERNEL1/zImage --ramdisk newramdisk.cpio.gz -o mynewimage.img

Edited by xangma
Link to comment
Share on other sites

Guest BigBearMDC
We are only a step away.

Huawei customised the synaptics driver cause they're numptys. If we want this to work ... we need to read this: http://lukehutch.wordpress.com/2009/01/25/...obile-g1-today/

And then look at this:

http://go2.wordpress.com/?id=725X1342&...KernelPatch.zip

Which is a kernel patch for the ORIGINAL synaptics driver.

We need to implement this code in the MODIFIED synaptics driver, and we're laughing. I however, am a dumbass and cannot do this ...

EDIT: To prove my dumbass theory, I only recently found out htat I was using mkbootfs wrong and that's why my kernels weren't booting. They all still don't boot, but the ones that actually should boot (2.6.27) now do.

I now use

 mkbootimg --cmdline "mem=128M console=ttyMSM2,115200n8 no_console_suspend=1" --kernel KERNEL1/zImage --ramdisk newramdisk.cpio.gz -o mynewimage.img

Well done!

But I think we will need the patched KeyInputQueue too.

I'm trying to get the already patched one - no success yet :(

If your boot.img is readys, let me know :(

Edit:

If you are a dumbass, we all are as intelligent as a piece of bread :(

Greetings,

BigBear

Edited by BigBearMDC
Link to comment
Share on other sites

Well done!

But I think we will need the patched KeyInputQueue too.

I'm trying to get the already patched one - no success yet :(

If your boot.img is readys, let me know :(

Edit:

If you are a dumpass, we all are as intelligent as a piece of bread :(

Greetings,

BigBear

My boot.img can be ready whenever you want. I just don't have the skill to put that extra code into the 1319 driver or whatever it is we use. I need a modified driver =P

Link to comment
Share on other sites

Guest BigBearMDC
My boot.img can be ready whenever you want. I just don't have the skill to put that extra code into the 1319 driver or whatever it is we use. I need a modified driver =P

Hold on a sec :(

Link to comment
Share on other sites

Guest BigBearMDC
No, that's the unmodified one. Sorry =[

The one we use is in /drivers/input/touchscreen/synap...

I'll attach it.

Sorry, I missed that.

Hold on a further sec :(

Link to comment
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.