Apple Developer Tips: How We Are Updating my.OnSIP for the iPad

This blog is by Brian, Software Engineer here at Junction Networks. This is a good read about adapting a product to the iPad; it contains some tips for developers and is the first of a series Brian will write about updating my.OnSIP for the iPad.

Initial iPad Testing

When the iPad started shipping, our CTO, John, called me into his office and asked me if my.OnSIP would work on it. I told him it probably would and joked that if he wanted it working, he could buy me one to test the site on. We both laughed as I walked out of his office.

Near the end of the day, he called me in again to tell me he'd taken me up on my offer. A few days later a shiny, not-yet-greased-with-fingerprints iPad was in my hands, and I set to work.

I began testing, and I found my.OnSIP almost completely worked with Safari on the iPad. That is, there were a few hangups that took much more time than expected to work out. The following five features were missing:

• scrolling the various frames didn't seem to work,
• some buttons didn't work when touched,
• dragging calls for transfer didn't work,
• incoming instant messages didn't beep, and
• voicemail wouldn't play back.

In the first part of this series, I'm going to cover the first three issues; the latter two are related and will be covered in the next post.

Scrolling Frames

Above, I said that scrolling didn't seem to work. That's because I'd become used to the other iPad apps I've been playing with since receiving the device. In other apps, a simple single-finger flick would scroll the various sections in an app. When this didn't work on our site, I spent an inordinate amount of time trying to find a work-around. I was chasing a red herring, fed by the Apple developer documentation explaining how mimicking the native feeling of scrolling is made possible by the generated single finger events and certain JavaScript libraries.

Since my.OnSIP is already pretty heavy, I didn't want to load in another JavaScript library if I didn't have to. Not to mention: proper integration into the existing code base could have been problematic. After digging furiously for a while to find a clean, easy solution, I stumbled across a posting in Google groups that mentioned two-finger dragging as the built-in way to scroll frames in Mobile Safari (the Safari implementation on iPhone and iPad). Et voilà! A solution that required no code, work, and was, at least in theory, a standard UI interaction.

Perfect.

Pressing my Buttons

Most of the buttons on the site worked, but there were a few that didn't do anything when touched. It turns out that Mobile Safari only considers an element touchable on certain conditions according to the Safari Web Content Guide:

A clickable element is a link, form element, image map area, or any other element with mousemove, mousedown, mouseup, or onclick handlers.

We're using SproutCore as our JavaScript client framework, and it uses a common optimization of applying all its handlers to the document's body. The effect of this optimization is that Mobile Safari can't detect that a given element is clickable unless it's a link with a valid href attribute. Well, most of our buttons are valid links, but we had a few cases where we opted for a link without an href or a different tag altogether (such as <span>). So, I worked through the code and converted all our buttons to links with an href of "javascript:;", which doesn't do anything except indicate to Mobile Safari that an element can be clicked.

Dragging Calls

Dragging calls didn't work at all because Mobile Safari was interpreting the drag movements as finger swipes used to pan the page. However, after reading Handling Events in the Safari Web Content Guide, I found out that you can grab touch events from the user as they occur, which I could then turn around and use to calculate drags.

The mildly tricky part is that SproutCore already handles dragging and dropping using mouse events (onclick, et al.). In the case of the iPad and iPhone, your finger (mostly) operates the way a traditional mouse cursor would. As such, it made sense to shoe horn touch events into mouse events at the lowest levels of SproutCore (rather than monkey-patch many different events in the higher level code). It also turns out that SproutCore completely ignored touch events, so some modification would be necessary to it, anyway. So, I decided, I might as well do the small amount of work there and reap the rewards at countless points in the higher level interface.

N.B: We're using SproutCore 0.9.23, which is fairly old at this point. Since version 1.0 SproutCore has supported touch events, so this work would probably not be necessary if we were using a higher version. We're using an older version because there have been a large number of incompatible changes between 0.9.23 and 1.0 which, while not necessitating a complete rewrite, would take enough time that it's been pushed down on the priority list as we have a working version of our site on 0.9.23.

The first thing to do was to get SproutCore listening for touch events. Here's the diff:

diff --git a/globals/window.js b/globals/window.js
index f7d8814..87de82a 100644
--- a/globals/window.js
+++ b/globals/window.js
@@ -383,10 +383,25 @@ SC.window = SC.PaneView.extend({
   _ondrag: function() {
     return false;
   },
-  
+
+  /*
+   * Mobile WebKit touch events.
+   */
+  _ontouchstart: function () {
+    if (event.touches && event.touches.length == 1) {
+      return this._onmousedown.apply(this, arguments);
+    }
+  },
+
+  _ontouchend: function () {
+    return this._onmouseup.apply(this, arguments);
+  },
+
+  _ontouchmove: function () {
+    return this._onmousemove.apply(this, arguments);
+  },
+
   _hasFocus: NO,
   
-  _EVTS: ['mousedown', 'mouseup', 'click', 'dblclick', 'keydown', 'keyup', 'keypress', 'mouseover', 'mouseout', 'mousemove', 'resize', 'unload', 'focus', 'blur','drag','selectstart'],
+  _EVTS: ['mousedown', 'mouseup', 'click', 'dblclick', 'keydown', 'keyup', 'keypress', 'mouseover', 'mouseout', 'mousemove', 'resize', 'unload', 'focus', 'blur','drag','selectstart', 'touchstart', 'touchend', 'touchmove'],
 
   _listenerCache: [],

Now that SproutCore is listening for touch events, and those events are proxying for the mouse events, we have everything almost working. Also note how I'm specifically ignoring two finger events in _ontouchstart: if you'll recall, above I mentioned that two finger swiping is how one scrolls a pane, so I don't want to capture those, but instead let Mobile Safari handle them.

The only catch at this point is that mouse events store the coordinates of the mouse when the event occurred in slots on the event called pageX/pageY or clientX/clientY. Whereas touch events store the coordinates in the touches array to allow for multi-touch events. As I'm only interested in single-finger touches when dragging, I can simply use the first element of that array as the coordinates of the "mouse" which the finger is emulating. Here's the patch:

diff --git a/core.js b/core.js
index 292ed34..cc22342 100644
--- a/core.js
+++ b/core.js
@@ -624,13 +624,21 @@ Object.extend(Event,{
   },
   
   pointerLocation: function(event) {
-    var ret = {
-      x: event.pageX || (event.clientX +
-        (document.documentElement.scrollLeft || document.body.scrollLeft)),
-      y: event.pageY || (event.clientY +
-        (document.documentElement.scrollTop || document.body.scrollTop))
+    var ret;
+    if (event.touches && event.touches.length > 0) {
+      ret = {
+        x: event.touches[0].pageX,
+        y: event.touches[0].pageY
+      };
+    } else {
+      ret = {
+        x: event.pageX || (event.clientX +
+          (document.documentElement.scrollLeft || document.body.scrollLeft)),
+        y: event.pageY || (event.clientY +
+          (document.documentElement.scrollTop || document.body.scrollTop))
       
-    };
+      };
+    }
     return ret ;
   }

Next Time

That's it for this installment. I'll be covering the poorly-documented and hair-loss-inducing <audio> drama next week wherein I rant and rave about cross-browser compatibility, weirdness between DOM scripting, HTML, and Mobile Safari's media loading policy.