MultiTouch

Using Jmol with Multi-Touch Panels

Note: This is a novel feature and experience with it is limited. If you have a multi-touch device, the Jmol developers will be happy to share your experience and suggestions.

...This is a draft yet...

Introduction to multi-touch technology

...

Preliminary demo of Jmol working with multi-touch features.

How to use it

You need at least version 11.9.11 of Jmol. It is recommended that you use the latest 12.1.x

First, you must start the device connection executing JmolMultiTouchDriver, a driver written in C++ and currently set up only for the HP TouchSmart computer using Windows, or any computer with NextWindow technology.

• Get JmolMultiTouchDriver.exe for HP TouchSmart.

Jmol Application

While JmolMultiTouchDriver is running, you start Jmol with the -Msparshui flag. Example:

java -jar Jmol.jar -Msparshui

There's also

java -jar Jmol.jar -Msparshui-simulated

Jmol Applet

While JmolMultiTouchDriver is running, the signed applet must be started with the multitouchSparshUI parameter set to "true".

One way to do this, if you use Jmol.js functionality, is:

 jmolSetCallback("multiTouchSparshUI", "true")

prior to

 jmolApplet(...)

With latest version of Jmol.js, you can also add parameters to the URL line, so you can adapt any page to work with multi-touch. For example, if you have JmolMultiTouchDriver running, the following should work: http://chemapps.stolaf.edu/jmol/docs/examples-11/molecules/molecules.htm?PARAMS=multiTouchSparshUI:true

Jmol embedded in another Java application

While JmolMultiTouchDriver is running, Jmol must be passed the command option -multitouch-sparshui.

Gestures

Implemented gestures include:

• single touch down/move/up (standard mouse down,drag,up -- does model rotation)
• single-touch click (standard mouse click)
• single-touch long click (does model translation)
• two-finger slide (does model translation)
• two-finger stretch (zooms in)
• two-finger pinch (zooms out)

Terminology

click

drag

slide

swipe: put finger on surface, drag in a single direction, while continuing the drag motion lift finger from surface

flick: swipe but done must faster -- Jmol does not differentiate this from swipe

pinch

stretch

Technical details of the implementation

An adaptation of Sparsh-UI to work with the Jmol application (either stand-alone or embedded in other Java programs) and with the Jmol applet. (But it does not use the standard Sparsh-UI gesture server.)

Uses a modified SparshUI interface over port 5946 (client) and 5947 (device)

After the small C++ program 'JmolMultiTouchDriver' is running, all messages from the touch-screen are passed to Jmol and interpreted as "gestures". Gestures include a standard two-finger spread/pinch for zoom in/out, a two-finger slide for translation, and a one-finger "flick" to start spinning.

The driver will cycle while waiting for a connection, and the first Jmol application or applet that loads will start a gesture server that connects with it. If additional applets or applications load, then their gesture server will not connect, but they will find the other gesture server.

The problem of multiple overlapping windows was solved easily by only allowing SparshUI action within a window if that window has the focus. So one needs to first click on a model to activate SparshUI action within that window.

There might be a problem if one applet starts a server and then closes while another is using it. In addition, multiple applets on the same page are no problem, but applets on different pages within the same tabbed browser system probably cause problems. Just closing a page doesn't necessarily work.

Custom gestures: SinglePointGesture and TwoPointGesture

Contributors

AngelHerraez