I have been thinking about the Apple iPad, specifically how it relates to other devices. I then recalled my undergraduate course in human computer interaction, and the work that was done on ubiquitous computing. I looked about me, and realised that I was surrounded by devices that were restricted to a few research labs fifteen years ago. The hardware is all there, with a few differences, but we are a long way from the vision of computing that came out of the last decade of the 20th century.
In the 1970s the Xerox Palo Alto Research Center (PARC) gave birth to the computing interface that would be used for the next 30 years (at least): the personal computer, modern graphical user interfaces, Ethernet networking, laser printers, and object-oriented programming. Famously, Apple stole most of these ideas to create the unsuccessful Lisa and highly successful Macintosh computers. As a follow-up, under the leadership of Mark Weiser, Xerox PARC started work on the next generation of devices.
What they came up with was the concept of ubiquitous computing
— where computers will be vastly better at getting out of the
way, allowing people to just go about their lives
(Weiser, 1993).
To achieve this Xerox PARC decided that we needed three devices to
supplement, or largely take over, from the personal computer: tabs,
pads and boards.
Tabs
Tabs were the smallest interface; about the same size as a Post-It Note. Originally they had very limited capabilities, with 128KB of store and the ability to play video at four frames a second. Tabs did have wireless networking, using a bespoke system that Xerox PARC developed, so they could work like a two-way pager (a device that the original tabs superficially resembled). Input was provided by a pen, but because of the limited processing power a simplified alphabet called Unistrokes was used (Goldberg and Richardson, 1993). Within an office building a tab could report its position, so your workmates will know where to find you, a bit like an advanced RFID security card.
Cell phones (alias mobile phones) are the same size as tabs. The intervening few years have seen store increase into hundreds of gigabytes, video increase to full-motion, three wireless protocols (Bluetooth, 802.11, and GSM) are typically used for communication, and GPS is used to track the location of the cell phone anywhere in the world (except in an office building, where satellite coverage can be a bit spotty).
Pads
Pads, or tablets, have had a longer history than tabs, as far as I can figure out. Commercial pen-based interfaces had been available for a couple of years by the time Xerox PARC started working on the MPad. Unlike the commercial offerings of the time, the MPad had wireless networking, multiprocessing (unlike the not-yet-shipped Apple iPad), and multimedia capabilities. However, it was the software that made the MPad dramatically different from the current crops of pads, but more on that later.
The pad has had a hard time commercially.
Microsoft has been pushing tablets for over twenty years without much
success (I can recall booting a Windows 3.1 for Pen Computing machine).
The Apple Newton was problematic enough to get
a mention on
The Simpsons.
Far more successful has been the netbook
, which has a user
interface similar to a desktop but it is as portable as a pad, for a
lot less money.
Interestingly, some netbooks are acquiring touch-screens, so they
can behave even more
like
tablets.
(The photo above is of a netbook, rather than an actual pad.)
Boards
Of the three interfaces I recall learning about as a student, the board is the first one I can recall ever seeing, sort of. The idea was to create a shared space where many people could interact — much like a whiteboard. The Xerox LiveBoard (Elrod et al, 1992) was the result of the PARC work on boards, which actually made it to market as a Xerox product. It could be used to record drawings, and make presentations, blazing a trail that Microsoft PowerPoint and Apple Keynote follow.
I first came across the LiveBoard concepts with the electronic whiteboards, which would record and play back the drawings that were made on it. However, game consoles are most common board-sized interface that I come across. They drive million-pixel displays, providing a shared space where people can interact using wireless controls.
The Hardware Difference: Keyboards
The tab developed by Xerox PARC did not have a keyboard as they have a
very small interaction area — too small for a keyboard
(Weiser, 1993).
The Palm Pilot and Apple Newton had pen-based input like the Xerox
tab.
However, most tab-sized devices are not pen-based: even the Apple
iPhone uses a keyboard when text needs to be entered.
I do not know why this is the case.
Maybe it was
a
long running patent suit (1997–2006) brought by Xerox
against Palm (over the Xerox Unistrokes system, on which Grafitti was
based) that made others wary.
Maybe keyboards are easier to implement and learn.
Maybe the tricks used by tab-like devices to allow text input
(like multi-tap and T9) are good enough:
the speed of input using T9 is
14.63wpm
(σ 1.09)
(Wobbrock et al, 2007)
compared to
15.8wpm (σ 4.02) for Unistrokes
(Castellucci et al, 2008).
The Unistrokes patent must be close to expiring, so we will see
soon if someone picks up the pen-input ball and runs with it.
Keyboards were optional on the MPad developed by Xerox, all netbooks have a physical keyboard, while the Apple iPad relies on an on-screen keyboard. As for boards, the Nintendo Wii, Sony Playstation 3, Microsoft XBox 360 consoles all have an optional keyboard. In addition, Microsoft PowerPoint and Apple Keynote presentations are usually presented on workstations acting as boards and the keyboards are vital to the creation of many presentations (sadly).
…But Where is the Software?
Reading back across the ubiquitous computing papers, old and new,
I am amazed at how far the hardware has come, and how little the
software has changed.
The original concept was to move beyond the intimate nature
of personal computing, and to create devices that were better suited to
communication and collaboration.
The software would allow tabs, pads, and boards to interact with each
other; not just your own devices, but those used by others.
A presentation could be displayed on a board, appear simultaneously
on pads (for annotation), and be controlled by a tab
(Myers, 2001).
A drawing made on one persons’s tab would appear on the shared board.
A running program would follow you (well, your tab) from your
workstation to another person’s office, where it will display itself so
you could discuss it, a bit like cut and paste writ large
(Rekimoto (1997)
dubbed it pick-and-drop).
There was excited (albeit awkward) talk of shared situations
and
collaboration.
However,
the seamless interplay between the multiple device surfaces
that Weiser imagined is still far from reality
(Klokmose et al,
2009).
Recent Apple products also lack the joy of ubiquitous computing. The language that Steve Jobs uses to talk about the iPad is still the language of the personal computer. Take the list of tasks that the iPad has to support:
- Browsing (which did not exist at the time of the Xerox MPad),
- Reading email,
- Viewing your photos,
- Viewing video,
- Listening to music,
- Playing games,
- Reading eBooks
There is no shared space, beyond email (which is older that I am)
and Web browsing (which is fifteen years old by my count).
Instead he claims that unless a pad is better than those tasks listed
above it has no reason for being
.
He then goes on to discuss how the iPad can be personalised
and
discusses the many single-user tasks for the iPad.
When he briefly discusses sharing, in the context of photos, the talk
is centred around a single device.
I am not a ubiquitous computing researcher (I studied undo) so I cannot offer any insight into why the software is not there, while the hardware is more than capable. Instead I take heart that there are still people working away at the problem — publishing papers about frameworks and software architectures, as user-interface researchers do when they need something to present and the coding is harder than expected. Software is also what free and open-source development is good at. With the free and open software present in tabs of all shapes and sizes, pads and boards I hope that some hackers may be inspired to take up the ubiquitous computing ideas, dodge the remaining patents, and create software that brings people together, allows them to share situations (not matter how awkward) and make the world a better place.
Me? I am still trying to sort out email…

