Ricardo Sato

The Beta, tbeta for short, is a open source/cross-platform solution for computer vision and multi-touch sensing. It takes an video input stream and outputs tracking data (e.g. coordinates and blob size) and touch events (e.g. finger down, moved and released) that are used in building multi-touch applications. tbeta can interface with various web cameras and video devices as well as connect to various TUIO/OSC enabled applications and supports many multi-touch lighting techniques including: FTIR, DI, DSI, and LLP with expansion planned for the future (custom modules/filters).

At its core,the concept of Tangible Interfaces leverages the idea of using the movement of the body as an inherent part of the human side of a human-computer interaction, assuming that bodily engagement and tactile manipulation can facilitate deeper understanding and more intuitive experiences. However,as an interaction principle in our era of digital design,motion construction and control has been underutilized as a design tool,leaving open the possibilities of motion's natural ability to draw our attention,provide physical feedback,and convey information through physical change. This dissertation postulates that the ability to experiment,prototype,and model with programmable kinetic forms is becoming increasingly important as digital technology becomes more readily embedded in our objects and environments. The need for tools and systems with which to create, manipulate, and finesse physical motion in response to computational and material input remains an under-developed design area.

SiMa Systems is a pioneer of Multi-Touch and Multi-Force (MT/MF) touch sensor technologies. SiMa’s resistive MT/MF touch technologies have unique functional characteristics enabling a new dimension of touch, gesture and pen input. Resolution of up to 20 points/mm affords uncompromised accuracy even for the most demanding applications. SiMa’s touch technologies can be integrated into a broad range of products from displays to touch pads.

On MacBook Air and recent MacBook Pro systems, Adobe Photoshop CS4 reads multitouch gestures to rotate of the document canvas. But it's easy to accidentally rotate the canvas on newer trackpads.

Internally, our devs have created several different physics simulators based on the applications they are working on which always seem to make adults giddy as schoolkids.

If you're remotely like me, you'll also like what Dave Brown has created. Dave works in the UK at one of our Microsoft Technology Centers. His job is to create proof of concepts for organizations working with Microsoft products. He's also a part time wizard. Seriously. Not only is the simulator crazy fun, I'm told his code is very elegant as well.

These plugins can help you integrate Zotero with other software.

Andres Jimenez, Interaction Designer, led today’s brown bag lunch and walked us through various aspects of creating physical computing networks and devices for home automation, location awareness and status sharing. We were intrigued by his objects that posted mood and status updates to Twitter.

Inside the GPS revolution it's more than maps and driving directions: location-aware phones and apps now deliver the hidden information that lets users make connections and interact with the world in ways they never imagined. The future is here and it's in your pocket.

The power of ultrasonic waves has been harnessed to produce "virtual" objects in mid-air.

Shinoda Lab
(The University of Tokyo)
@SIGGRAPH2008 New Tech Demos

In an upcoming issue of IEEE Transactions on Robotics, Ig Mo Koo, Hyouk Ryeol Choi, and co-authors from Sungkyunkwan University and the University of Nevada explain how they have designed the innovative tactile display based on soft actuator technology. Overcoming the rigidity and bulkiness of current devices, the new display is soft and flexible enough to be wrapped around almost any part of the human body, such as the fingertip, palm, or arm.

At TAT, we thrive on innovating, creating new and unique ways of interacting with devices. We produce a number of concept-based futuristic UIs focusing on trends, predications, and possibilities that one can only dream about. We create opportunities for these to potentially exist and envision the way they could affect the mobile and social world.

We started doubleTwist because we were disappointed by the quality of software applications offered by the major device manufacturers. We wanted to build a universal media application that helps people quickly and easily play all their stuff, on all their devices and share their experiences with all their friends.
Our vision is to provide for media what the browser achieved for the web: a single, streamlined interface that connects to any device, media source or network.

I came up with a new display idea that changes shape (ShapeShift).
It would probably need to be limited to a preset form (not like the colours on a display that has pixels). Like an ice cube bag when you fill it with water the shape is predetermined by the space inside the bag

The vinyl is perfect because if you press it against the glass it adheres like a decal (because decals are made from the same material!).

The iPhone's beautiful glass screen makes it easy to cut the overlay – fire up the NES emulator, put the vinyl over the screen, and go at it with your Xacto knife. The screen won't scratch from the knife, so you can get a perfect overlay with little effort.

A variety of devices currently on the market provide simple forms of touch feedback, but none is an unqualified success – they all lack some aspect of physical experience, a correspondence with the way we actually interact with the world. Current tactile solutions fall short either in reconfigurability or in pre-interaction feedback. This pre-interaction feedback would provide the physical feeling of a button which the user can press or not, rather than just a tactile confirmation that they have just pressed that button. Reconfigurability would allow physically felt controls to change with the content of the display. In short, it’s easy to make static physical buttons, but not to make them disappear when not needed. And it’s easy to provide a physical sensation after the user interacts, but not to provide buttons that can be physically felt *before* the interaction is committed.

This is the third and final part of this introductory "mini-series". Part 1 introduced the value of ergonomics to interaction designers, and Part 2 discussed some of the challenges and methods of anthropometric design for a broad range of users. Now I am going to focus on how to identify ergonomic issues in observational and lab testing contexts.

There are several body measurements that could be relevant for reaching a touch screen, but a practical one would be Forward Grip Reach distance – roughly the distance from the shoulder axis to the palm of the hand. With those two metrics in mind – eye height and forward grip reach – you could picture any user as the function of two perpendicular lines – a vertical line, representing the individual's eye height, and a horizontal line representing arm reach. This is illustrated above for a range of three different users – note that the wheelchair user has a sitting eye height compared with the two standing users.

The continuing convergence of digital interfaces with physical products is putting interaction designers in a position where knowledge of anthropometrics, kinesthetics, and other non-cognitive human capabilities is valuable for creating effective design solutions.

Here we find a transparent music player that slaps onto your wrist. Don’t want it clear; just opt for an image or color graphics instead and use it as a wrist-gear. Teamed with earphones that bring the “concert performance” to your ears, the trip sounds bombastic! I did mention futuristic right? To justify that Mac includes a built-in mic that helps locate and play a song on your playlist, once you hum the tune into it.

While we're still waiting for our electronic broadsheet (hell, we'll settle for a tabloid) it looks like folks in Toshima will be seeing quite a bit of the e-paper as they hit the streets of this Tokyo ward. In a test conducted by the Ministry of Internal Affairs and Communications, the signage has been installed in the proximity of the Toshima Life and Industry Plaza, where a wireless network was established to provide updates in case of an emergency.

The “Colorsonic”: an MP3 player that displays its playlist with color. Each color represents a music genre. When you want to play some songs n stuff, choose 1 (or more) color(s) from your wheel, and a list will be automatically picked for you made of songs from the genre colors you chose.

The wide-ranging discussion focused on how the mobile Web is becoming increasingly important, and in many countries, the primary way people connect to the Internet. A good chunk of the panel was spent talking about the implications of sharing where you are and what you are doing all the time because mobile phones make it easier to do so. Chad Hurley noted that the rate at which YouTube is serving videos to mobile devices is growing at a faster rate than the site as a whole. Mike asked Chad Hurley how long would it be before people started using their mobile phones to upload videos to YouTube in a serious way.

Although CES 2009 was undoubtedly smaller and perhaps a little more subdued than last year's HDTV-dominated extravaganza, the products we did see were a lot more interesting — and of course, Palm stole the show with its blockbuster Pre announcement. We've rounded up the highlights below, make sure you didn't miss anything!

Founded in 2006 – Natural User Interface or ~ NUI Group is an interactive media community researching and creating open source machine sensing techniques to benefit artistic, commercial and educational applications.

SHiFT has always been about emerging technologies, whether they’re the latest internet trend or the latest social or engineering development.

Since 2001, OFFF festival has been held in Barcelona, becoming the globally recognized and trendsetting event it is today. The three-day festival showcases top digital artists, web, print and interactive designers, motion graphics studios, and new music adventurous. OFFF festival provides insight into all culture media platforms.

a recently born interaction design group that explores natural communication between people and technology. We design experiences that merge real and digital into a creative environment where people are invited to touch, play, move, feel as they do in the real world.

MT Barcelona, one of our wonderful community member's group, has really out done themselves in creating their final demo reel for their latest project. The touch wall was designed for the Red Bull Music Academy 08. Guten Touch is an interactive installation that involves people into a natural relationship with technology. A two projected display system plus a 3m x 2m multitouch wall showcase applications designed to engage us into human friendly experiences rather than flashy and jaw-dropping visualizations. Space Invaders hitted by foam balls, pixel paintings created with brushes and digital objects held by hands try to blur boundaries between real and digital.

Robin Chase founded Zipcar, the world’s biggest car-sharing business. That was one of her smaller ideas. Here she travels much farther, contemplating road-pricing schemes that will shake up our driving habits and a mesh network vast as the Interstate.

We can’t see how the street is immersed in a twitching, pulsing cloud of data. This is over and above the well-established electromagnetic radiation, crackles of static, radio waves conveying radio and television broadcasts in digital and analogue forms, police voice traffic. This is a new kind of data, collective and individual, aggregated and discrete, open and closed, constantly logging impossibly detailed patterns of behaviour. The behaviour of the street.

The point here is that basic structure is the same — all buildings get added to. There needs to be some kind of honoring of the original style.
pace layers

* fashion
* commerce
* infrastructure
* governance
* culture
* nature

Reality Mining defines the collection of machine-sensed environmental data pertaining to human social behavior. This new paradigm of data mining makes possible the modeling of conversation context, proximity sensing, and temporospatial location throughout large communities of individuals. Mobile phones (and similarly innocuous devices) are used for data collection, opening social network analysis to new methods of empirical stochastic modeling.

MIT Media Lab group. The goal of the Fluid Interfaces research group is to radically rethink the human-machine interactive experience. By designing interfaces that are more immersive, more intelligent, and more interactive we are changing the human-machine relationship and creating systems that are more responsive to people's needs and actions, and that become true "accessories" for expanding our minds.

Pulp-Based Computing is a series of explorations that combine smart materials, papermaking and printing. By integrating electrically active inks and fibers during the papermaking process, it is possible create sensors and actuators that behave, look, and feel like paper. These composite materials, not only leverage the physical and tactile qualities of paper, but can also convey digital information, spawning new and unexpected application domains in ubiquitous and pervasive computing at extremely affordable costs.

With Invisible Media we can augment objects around us to make them sensitive to, and able to inform, the focus of our attention in order to provide relevant content. The system is built to minimize bulky wearable gear, and allows the user to navigate this extra channel of physically-situated information with speech commands, keeping their hands available for manual manipulation of the objects themselves. Information is presented to people auditorily, resulting in a user-system dialog that attempts to mimic a domain expert or recommender who knows what objects are in view of the user and can suggest relevant content. We have created Engine-Info, a training application that teaches the components of an internal combustion engine, as well as My-ShoppingGuide, a personalized shopping scenario that can suggest appropriate foods in a supermarket based on a person's preferences and health needs.

MIT grad student David Merrill demos Siftables — cookie-sized, computerized tiles you can stack and shuffle in your hands. These future-toys can do math, play music, and talk to their friends, too. Is this the next thing in hands-on learning?

In this work, a sheet of paper is introduced as a haptic IO device.
In general, there are not so many kinds of interaction techniques on a multitouch screen. (Everything people do on it is zooming and rotating images.)
However, introducing real objects to interaction like this expands the capability of tabletop device.

The Gesture and Activity Recognition Toolkit (GART) (formerly Georgia Tech Gesture Toolkit) is a toolkit to allow for rapid prototyping of gesture-based applications. There are two versions of the toolkit, a Linux shell scripting based version and the more refined Java version.

Associate Professor & Canada Research Chair

It’s extremely processor-intensive computer vision, happening in a video stream, all with JavaScript worker threads. That is, this is possible because the next version of Firefox, version 3.1, allows for multiple threads processing the video instead of trying to do everything in succession. HTML5 + Firefox 3.1 + some not-terribly-backwards-compatible code = basic vision. It looks like it’s pretty simple frame differencing with a threshold, then a bounding area drawn around the spot that changes.

Experiment using Oblong Ind. Interaction system.

Continuing our multi-touch research, we’ve been working on turning an off-the-shelf rear projection TV into a multi-touch display. This screen has the best width-to-depth ratio of any multitouch system (67″ diagonal viewing area and only 16″ deep). This is also the first example of hacking a multi-touch system into an off-the-shelf television. The system we settled on uses very few additional components and could potentially be applied to any rear-projection TV.

Here is an example of using a gesture recognition library in Processing. The Gesture and Activity Recognition Toolkit (GART) is from http://wiki.cc.gatech.edu/ccg/projects/gt2k/gt2k. It makes use of a Hidden Markov Model toolkit (HTK). Both training and recognition are supported.

Here is a very primitive and limited Processing library simpleARToolKit made as a wrapper from another open source project jARToolKit which is a Java port of the original ARToolKit. It only works with an OpenGL sketch in Windows. The download includes the following two example sketches.

Interested in performing high-performance, high-quality video processing, computer vision, motion tracking, and analysis? And want to do it in the friendly Processing coding environment – an ideal place to start, even for non-programmers?

In this tutorial, I will show you how to use a thresholded frame difference (motion) image in order to perform collision detection with objects onscreen. Essentially we will be creating something similar to one of the old webcam games where you can ‘pop bubbles’ with your hands (or indeed anything that moves).

It’s a relatively easy thing for computers to “see” video, but “computer vision” goes a step further, applying a wide range of techniques by which computers can begin to understand and process the content of a video input. These techniques tend toward the primitive, but they can also produce aesthetically beautiful results. The best place to start with computer vision has long been the standard library, OpenCV. A free (as in beer and freedom) library developed by Intel and with ongoing use in a variety of applications, OpenCV is a terrific, C/C++-based tool not just for things like motion tracking, but video processing in general. OpenCV gets a lot of support in the C++-based OpenFrameWorks, but that doesn’t mean Java and Processing have to be left out of the fun.

OpenCV is an open source computer vision library originally developed by Intel. It is free for commercial and research use under a BSD license. The library is cross-platform, and runs on Mac OS X, Windows and Linux. It focuses mainly towards real-time image processing, as such, if it finds Intel's Integrated Performance Primitives on the system, it will use these commercial optimized routines to accelerate itself.

OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real time computer vision.
Example applications of the OpenCV library are Human-Computer Interaction (HCI); Object Identification, Segmentation and Recognition; Face Recognition; Gesture Recognition; Motion Tracking, Ego Motion, Motion Understanding; Structure From Motion (SFM); Stereo and Multi-Camera Calibration and Depth Computation; Mobile Robotics.

Projecting a narrow band of light onto a three dimensionally shaped surface produces a line of illumination that appears distorted from other perspectives than that of the projector, and can be used for an exact geometric reconstruction of the surface shape (light section).

OpenProcessing is a ‘flickr’ish place for Processing addicted generation.

DAVID-laserscanner is a very low-cost system for contact-free scanning of 3d objects. The only hardware requirements are a simple commercial hand-held laser and a standard camera.

Kyle McDonald sends us a hacked-together 3D scanner. I love that it’s slightly inaccurate in aesthetically-pleasing ways, I love that it’s something you can put together using stuff you already have at the ready, and I love that it’s powered by Processing. The applications could range from 3D models to motion graphics and animation to assistance in mapping projections to 3D objects.

Go mobile. JavaFX is the best way to bring expressive, feature-rich content to mobile devices. JavaFX uniquely leverages the Java ME platform's compelling mix of ubiquity, capability, expressiveness, and performance.
Get started today. The JavaFX SDK has the essential set of technologies, tools and resources required for developers and designers to create and deploy expressive and powerful mobile content.

Clutter is an open source software library for creating fast, visually rich and animated graphical user interfaces.
Clutter uses OpenGL (and optionally OpenGL ES for use on Mobile and embedded platforms) for rendering but with an API which hides the underlying GL complexity from the developer. The Clutter API is intended to be easy to use, efficient and flexible.

Sequencing, scripting, and more for Max/MSP

If you’ve worked at all with patching your own creations for music, visuals, and control, this has probably happened to you: you’ve made some change, and forgot what you did. You think of something you did some time ago – and forget what it was. Or you want to be able to easily collaborate with other people, and that means a lot of files flying around and no idea which file has which change. All of these problems are familiar to programmers. The solution: a technique called version control. Sounds fancy, but it’s really accessible to anyone, not just advanced programmers. And once you try it, you’ll never go back.

Baby’s cries have a general pattern for its needs: hunger, discomfort (wet diaper, for example), sickness, sleepiness, and tiredness. This pattern enables BabySays to translate baby’s language into adult’s language as well as monitoring the baby, and help parents respond to baby’s cry appropriately and promptly.

The I-Quad uses tile shaped LED electronic boards held together by a simple frame and interacts with external devices via USB or wireless. Each tile is capable of a low-res, almost dot matrix-like resolution. Essentially anything can be displayed across these tiles; from communication, entertainment, even ambient lighting.

Your hands are gesturing but I do not understand! Is it me or are your hands telling me something?
Comprehending sign language is not an easy task, but should that be a hindrance for communication between you and a vocally challenged person?
I guessed not, hence this Sign Language Translator will be the apt solution for you to transcend borders of gesture-based communication.

I am that stage in my life where I couldn’t care a less about the make or model of the mobile phone I use. I simply want functionality and that’s it! However according to my son, if he doesn’t get the latest model, his social life will be dead and he probably will be an outcast for holding that hands-me-down number! Do I have to get him a new mobile, every time the fad changes? No, not if something like the Infinite Sustainable Mobile Phone pops up.

In Designing with RFID we explore the potential for RFID objects in everyday contexts. Because RFID is a wireless, radio-based technology it is inherently invisible once embedded, and this raises issues around visibility and interaction. How does the addition of hidden interactive qualities influence the design of physical RFID objects? There is a need to develop tangible design qualities such as shape, materials, build quality and affordances for RFID-enabled objects.

In Designing with RFID we explore the potential for RFID objects in everyday contexts. Because RFID is a wireless, radio-based technology it is inherently invisible once embedded, and this raises issues around visibility and interaction. How does the addition of hidden interactive qualities influence the design of physical RFID objects? There is a need to develop tangible design qualities such as shape, materials, build quality and affordances for RFID-enabled objects.

This was an outdoor advertising campaign done by Inwindow Outdoor for the release of the animated film, Coraline. It appeared in 7 cities, at 14 locations and used a variety of technologies including video, audio, gesture, holograms and augmented reality.

These templates have been built upon the work of Nathan Smith and his 960 Grid System using effects from the Mootools JavaScript library. The idea for building these templates was inspired by Andy Clarke, author of Transcending CSS, who advocates a content-out approach to rapid interactive prototyping, crediting Jason Santa Maria with the grey box methodology.

2008, Bonn, Germany February 18 – 20, 2008

The 'iPoint 3D' allows people to communicate with a 3-D display through simple gestures – without touching it and without 3-D glasses or a data glove. What until now has only been seen in science fiction films will be presented at CeBIT from March 3-8 by experts from the Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI.

Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves
2002, Minneapolis, Minnesota, USA April 20 – 25, 2002

Impress is the deliverance of the touch screen from its technical stiffness, coldness and rigidity. It breaks the distance in the relationship of human and technology, because it is not any longer the user which is subjected to technology, but in this case the display itself has to cave in to the human. Impress is a chance of approach of user and technology, above all, from technology.

It is a matter of a flexible display consisting of foam and force sensors which is deformable and feels pleasantly soft. Impress works with the parameters position and time like other touch screens as well, but in addition to that, it reacts, above all, on the intensity of pressure.

Talk to me. Not at me.
Thank goodness for modern web services. Apps like Twitter, and Tumblr have taught us all new ways to communicate briefly, and succinctly. It's time that the business world started embracing the new way of talking. If you're stuck in: account/client/project/type-of-data/some-specific-list – Pull your head out. There's a lot of work and communication going on around you. Stay on top of it.

re/touch brings together hundreds of cross-cultural examples of social norms and values involving touch—all categorised according to actions related to touching.
A collection of quotes from ethnographic accounts written between the late 1800s and the present, re/touch encourages designers and researchers to explore how touch is used by people to relate to one another and the worlds in which we live.

As a collaborative learning center, the SMART Table enables engaging and motivating small-group learning experiences. Up to eight students can use their fingers intuitively to sweep, slide and spin objects on the interactive screen. The SMART Table's ready-made activities help primary students gain and further their skills in areas like counting and reading.

Chris Harrison and Scott Hudson (Carnegie Mellon University) came up with a way to use any flat surface as a gesture based input device (Scratch input). An absolute breakthrough which makes extraordinary interactions, like for example controlling your television or music player with your wooden kitchen table, possible.

Scratch input is specifically designed to use augmented existing, passive surfaces as an input device. Using a simple sensor which is sticked on the surface, the sound-waves are captured. Due the fact that every movement produces a different sound-pattern, gestures can be recognized.

Although gesture based interacting is really hot at the moment, we still don’t fully use the power of gestures. That’s what London based designer Zinc Chan must have thought when she came up with the gorgeous looking telephone Icono. Her design was based upon the “iconic hand gesture people use to represent the telephone.” When you want to make a phonecall you don’t pick up the handset like on current telephones, but you put the earphone and microphone around your fingers. By making the telephone gesture you create a phone with your hands.

TEI'09 is the third international conference dedicated to presenting the latest results in tangible and embedded interaction.
Computing is progressively moving beyond the desktop into new physical and social contexts. One key area of innovation has been around tangible computing, which pushes the user interface beyond the digital into the physical world by means of mobile devices, graspable interfaces, physical computing, and interactive surfaces. A closely related topic is that of embedded interaction where the everyday objects and environments we interact with are computationally augmented in new ways. Designing such systems requires interdisciplinary thinking. Their creation must not only encompass software, electronics, and mechanics, but also the system's physical form and behaviour, and its social impacts.

Initially we only had a keyboard for the command line and text entry. Then the mouse appeared for navigating two dimensional plains of UI. Now the field of computing has a new input toy to play with; our hands. Touch, multi-touch and gestural computing, also known as Natural User Interface (NUI) has become the newest input craze. Excitement around this has even spurred comments predicting the demise of the mouse in the next 3-5 years1. Computer designers (and engineers) have become engrossed with the ability to touch the screen with multiple fingers and control software by waving their arms. However in this excitement, have designers overlooked how to properly engage users and use multi touch to create useful, innovative, and interesting experiences? Perhaps touch and gesture are simply the new shiny objects in the room, soon to be discarded for the next new thing.

Siftables are small devices which have a graphical display, a number of sensors and wireless communication capabilities. They are small tangible user interfaces which can function individually or in a group, and can be manipulated to interact with digital information and media.

Zipiko lets you see what your friends are planning to do and allows you to quickly invite them to join you.

mixin lets you share your daily activities and intentions to get together more often with your friends

Metadesign (or meta-design) is an emerging conceptual framework aimed at defining and creating social, economic and technical infrastructures in which new forms of collaborative design can take place. As a methodology, its aim is to nurture emergence of the previously unthinkable as possibilities or prospects through the collaboration of designers within interdisciplinarity 'metadesign' teams.

‘Metadesign must allow a social mode of existence that is flexible and based on mutual processes of affecting and being affected, rather than on a juridical model. According to Thacker, metadesign represents a critical and creative investigation into the possibilities of transformation of human beings and culture’.

Need to show something to someone? iShowU is your answer! iShowU is designed to record anything on your screen, instantly — both audio, and video!
If a picture is worth a thousand words, then a movie is worth a million. Imagine the time you'll save writing Grandma an email about how to magnify her dock when you can just shoot a movie in seconds.