Proof of Concept Models
(Examples of my work)
Computer Science Department
Seeing Eye Mouse
The Seeing Eye Mouse was a sidebar project that I did while officially working on the "Desktop 370" project under Barry Goldstein. Eventually it became an official, funded project. My work on the Desktop 370 included building a read-by cache memory card for helping with the disk speed bottleneck, writing a memory manger for the cache (replaced in the final version with much better code by Dave Toll), and writing a windowing system. At the time all we had in the IBM product line were green screen text displays and the only way to attach a mouse was to use a device that translated the mouse stream into keyboard characters. Despite the level of the available technology, the user interface of my windowing system included sweeping gestures to scroll window content, change window location and change window size. The Seeing Eye Mouse came from thinking about how a mouse could be used as an output device while riding in an elevator that had floor numbers in Braille.
The first version of the Braille Display Mouse was build using N-Gauge model railroad switch track actuators. The alpha model of the device (seen below on the left) was build in the Central Scientific Services machine shop at Watson. The beta model (seen below on the right) was built by me using a single character display from an eighty character line braille terminal. Such terminals cost on the order of ten thousand dollars while the Braille Mouse could be sold for seventy-five dollars. I divided programming for the mouse with Bill Strohm. He did the input side, I did the electronics and the output software. This included character to Braille translation and a set of novel Braille glyphs for window borders and other screen graphics.
A dozen of the mice were built and field tested with IBM employees. At this point we learned that the vertical movement of the Braille pins could only be read by sweeping the fingertip over the display surface. That limited reading speeds. I designed a braille display that would give a virtual moving page but by that time management (no longer Barry) had lost interest and the project was terminated.
Simon - The first smart phone
Early in the 1990's I was contacted by an IBM development group in one of the southern states. They were interested in the Predictive Keyboard I invented while I was involved in the handwriting recognition department. I sent them the demo code and the predictive data sets. I didn't hear from them again until I was sent a Simon phone. The sad part of this is that the data set they worked with was built from my YKTVMV email archive. It was a proof-of-concept grade data set. The Predictive Keypad was good despite this, but it could have been much better. The version implemented for Simon lacked a lot, including the ability to improve its predictions by incorporating its observation of the user's wording choices into its prediction tables.
The pictures below show the Simon and a Simon screen with the Predictive Keypad.
Here is a youtube review in which the predictive keyboard component of the Simon interface is demonstrated.
A Basic Yorktown Security System (ABYSS)
The ABYSS project grew from lunch table conversations between Steve White and myself concerning the problem of software piracy. In a nutshell the problem was to find a way that software delivered anonymously in a shrink wrap box to a computer owned and operated by a potential pirate could be protected from illegal copying. At night, riding home in my car pool with Alex (Akhileshwari) Chandra, head of the Computer Systems Department , I would review the current state of my thinking on the problems brought up by Steve. Alex suggested using a privilege structure and in an instant the architecture became clear to me. The system could provide the "Glass House" sort of security found on mainframe computers by grafting an inverted branch onto the privilege tree. This branch made it possible for vendors to hide service providing parts of software packages in a physically secure (very hard to open without destroying its content) coprocessor that also provided key management and space management to keep vendors from "seeing" each others' code. The physical security system, of my invention, was secure in essentially the same way that cryptography secures information. Finally, software provided on floppy disk was shipped with a use-once hardware component (a "Token") of my invention and accessed by a protocol of my invention that made it essentially impossible to reproduce despite being executed in public. Prototype units were built and demonstrated. These included tokens (Bill Strohm) but the physical security system was kept at an intrusion detection level rather than a cryptographically secure level. When the system was reviewed by the IBM Science Advisory Committee, Frederick P. Brooks asked (upset by foot dragging) why it wasn't in IBM products yet. The SAC recommendation was that Don Coppersmith (one of the core DES designers) review the design. Steve and I had already passed the design by him. His reply to the Committee was that it worked the first time he looked at it and it still works.
In retrospect I can't help but feel that this architecture and mixed hardware-software method was a sufficient building block to solve problems as pressing today as computer viruses and some forms of identity theft. Certainly infrastructure security against hacking could also be effectively addressed. In the end, it died as a project for several reasons, backbiting politics, shortsighted middle management and an unwillingness to add about 0.25% to the cost of a personal computer. This failure of judgement has already cost hundreds of millions of dollars.
The picture below shows a finished processor cut lengthwise to show the security layer, the pair of processor boards, and a wire wrapped board pair ready for potting in epoxy.
The Personal Speech Assistant
The Personal Speech Assistant was a project aimed at bringing the spoken language user interface into the capabilities of hand held devices. David Nahamoo called a meeting among interested Research professionals, who decided that a PDA was the best existing target. I asked David to give me the Project Leader position, and he did.
On this project I designed and wrote the Conversational Interface Manager and the initial set of user interface behaviors. I led the User Interface Design work, set specifications and approved the Industrial Design effort and managed the team of local and offsite hardware and software contractors. With the support of David Frank I interfaced it to a PC based Palm Pilot emulator. David wrote the Palm Pilot applications and the PPOS extensions and tools needed to support input from an external process. Later, I worked with IBM Vimercati (Italy) to build several generations of processor cards for attachment to Palm Pilots. Paul Fernhout, translated (and improved) my Python based interface manager into C and ported it to the Vimercati coprocessor cards. Jan Sedivy's group in the Czech Republic Ported the IBM speech recognizer to the coprocessor card. Paul, David and I collaborated on tools and refining the device operation. I worked with the IBM Design Center (under Bob Steinbugler) to produce an industrial design. I ran acoustic performance tests on the candidate speakers and microphones using the initial plastic models they produced, and then farmed the design out to Insync Designs to reduce it to a manufacturable form. Insync had never made a functioning prototype so I worked closely with them on Physical UI and assemblability issues. Their work was outstanding. By the end of the project I had assembled and distributed nearly 100 of these devices. These were given to senior management and to sales personnel.
Click the links below to see the PSA components in operation. Return to this page by using your browser previous page key.
PSA breadboard - This video shows the PSA parts wired together for design testing. The parts include the processor board, switch and connector board, IBM Workpad (Palm Pilot 3), test base reflex speaker enclosure and stereo lithographic model case (for microphone testing).
Complete PSA - This video shows a more complete version of the PSA demonstrated by a Sales Rep at the 2000 Mobile insights conference.