Tag: Multi-touch

Delta + WIRED Store = Touch the Future of Travel

Mar 08, 2011 by in Experience Design, Multi-touch, Portfolio, Touchscreen

When it comes to travel, people care more about where they’re going than how they’re getting there. Delta Airlines understands this and asked Razorfish’s Emerging Experiences team to create an engaging experience for the WIRED Holiday Store in NYC. We wanted to tap into users’ imagination and sense of playfulness so that they walk away from the experience thinking about what kind of destinations they want to go to next and, of course, Delta.

In four (4) weeks we concepted, designed, developed and launched The Untravel Idea – a new, personal way for leisure travelers to encounter destinations. We wanted users to touch the future of travel.

The experience gives people an open-ended, creative experience that puts the user in control. First users choose the type mood they are looking for on their next getaway. From there users can select from a wide variety of relevant words that match their mood, and when put together, show them a range of destination possibilities. The result is a beautiful montage of photographic imagery that will transport the user’s imagination.

To extend the experience beyond the store, users are prompted to use their mobile device to snap a pic of a QR tag associated with each destination that allows them to explore additional destination info, video and travel packages.

The event was a complete success. People couldn’t wait to see where Delta would take them next. The Takeaway – Delta is not just an airline; they’re giving me new ways to discover travel destinations.

This is the future of travel … and it’s just the beginning.


CES 2011 Recap

Jan 13, 2011 by in News, Technology

The Consumer Electronics Show was back for 2011 and our team was on the ground in Las Vegas. We have a number of initiatives going on at CES this year.

First, our team was involved in the Microsoft Surface 2.0 launch. We’ve been working with the Surface team for a few months on the next generation of Surface. We’ve been porting our applications to run on the latest version. We can proudly announce that we are Surface 2.0 ready and we look forward to supporting the new platform and bringing the solution to our clients. The Microsoft Surface announcement caught the media by surprise – it’s been over 3 years since the original Surface was announced. The new device is faster, leaner and costs less than the previous version of Surface. We’ll have an in-detail analysis of Microsoft Surface 2.0 posted on the blog shortly.

Second, we were involved in the launch of another experience for one of our clients. We created a solution that will be experienced by millions of consumers in the market. Unfortunately our involvement must remain confidential so we can’t go into too many details. Let’s just say it was definitely one of those opportunities that we could not pass up!

We took the opportunity to explore the trade show floor in an effort to educate ourselves on the latest technology offerings. We hope to bring some of these technologies to our clients in 2011. Here are some of the technologies that we’ve got our eye on.

Tablets

2011 has been declared the “year of the tablet”. There were certainly no shortage of tablets at CES. In fact, about 80 new tablet form-factor devices were announced at CES this year.

From a hardware perspective, tablets are getting thinner, lighter and more powerful thanks to innovation around chip technology from companies like Intel, ARM, nVidia and Qualcomm. There are a variety of new form-factors hitting the market. The Eee Pad Transformer tablet can be docked in a base which transforms the device into a traditional laptop form-factor. The Dell Inspiron Duo tablet features a reversible screen to accomplish the same thing.

There were a variety of different screen sizes available. One of the interesting debates between amongst members of our team was around the usefulness of the small screen tablets. These “tweener” devices feature screens between the size of a typical phone and an iPad. The smaller size means they are more portable than an iPad, however they still can’t fit in your pocket and they can’t make phone calls.

One of the most impressive devices was the BlackBerry Playbook. The device features a brilliant user interface which makes use of NUI design principals – direct interaction of content through the use of gestures. In addition, the performance of the device was exceptional. We can’t wait to start developing for this platform.

For the first time, we had the opportunity to see the new Android Honeycomb tablet OS. The exerience is decidedly Android retaining much of the same design language. Improvements have been made to the user interface to take into account the additional tablet real-estate. In all honesty, we were slightly disappointed with the user interface. We were hoping for something game-changing from Google and instead, they delivered an experience that was transitional, not transformational.

One of the major disappointments was the lack of direction from Microsoft on tablet devices. We were crossing our fingers for an announcement around a tablet operating system that was lightweight and provided an exceptional user experience similar to what is being provided on Windows Phone 7 platform. And we wanted this platform soon.

Microsoft did acknowledge they are behind in the space. Right now, their story is positioning Windows 8 as the solution for tablets by supporting system-on-a-chip architecture. By supporting this hardware platform, Microsoft will be able to deliver Windows experiences on tablet devices while taking into account battery life and OS performance.

Unfortunately, no announcements were made around the Windows 8 user interface. Delivering an exceptional tablet UI will be essential to their strategy. It is likely Microsoft will adopt the “Metro” design language currently being used for Windows Phone 7 and Microsoft Surface 2.0.

Gesture Control and Natural Interaction

With the release and success of Xbox Kinect, the gesture control market is heating up. Much like the original iPhone brought touch interaction into the mainstream by putting millions of devices in the hands of consumers, Xbox Kinect will do the same for gesture control. The way we interact with computers is fundamentally changing and we are getting in on the ground floor.

We’ve taken the opportunity to develop for the Kinect platform, however we were looking for a commercial-grade solution to bring to our clients. Enter PrimeSense.

PrimeSense licenses their technology to Microsoft for use in the Xbox Kinect, therefore they seemed like the perfect partner to deliver the hardware and software to support commercialized gesture control solutions. We are actively working with PrimeSense to develop for their platform. Their OpenNI initiative hopes to create a framework for standardization of natural interface development across devices.

We see gesture control technology being used in an in-home setting and also in retail environments. This technology can be utilized to create at-home shopping experiences which combine natural interaction and augmented reality. Imagine being able to virtually try on clothes from the comfort of your own home. Or order a pizza with a flick of the wrist from the comfort of your couch.

We have been champions of the use of interactive experiences in the retail environment and we have the statistics to prove it. To date, the majority of our experiences have utilized touch. This technology provides a new user interaction paradigm and offers an entirely new world of possibilities in the retail space.

Touch Screens

Touch screen technology is evolving rapidly. Devices are becoming larger, cheaper and more reliable. Exciting new form-factors and multi-touch hardware will help us deliver new experiences to our clients in 2011.

3M Touch Systems has exciting new hardware and form-factors hitting the market which utilize their massively multi-touch projected capacitive technology. This technology provides extremely stable multi-touch that supports a large amount of touch points. 3M is brining 23” and 32” screen sizes to the market. In additional, the screens can be integrated into a multi-device array to build large size touch wall and table experiences.

We also had some hands-on time with systems from PQ Labs and Multitouch.fi. Both vendors offer touch solutions that are unique and exceptional. We look forward to working with these companies in the future.

Display Technology

Displays are getting thinner, lighter and more energy efficient. 3D technology is also evolving quickly. Much like last year, 3D display technology was everywhere. The most impressive innovation in the 3D TV space comes from LG. They demonstrated how their 3D technology has been standardized – every TV on display in their booth could utilize the same pair of glasses to deliver an exceptional 3D experience. They also demonstrated flicker-less 3D which produced a better 3D picture than we’ve seen on any other consumer device.

The glasses-less 3D technology was a disappointment. There isn’t enough discernable depth with the current iteration of the technology. Certainly this will change over time, however the promise of ditching the glasses has yet to be fulfilled. We wouldn’t be surprised if this changes in 2011.

In-Car Technology

Ford had the major innovations in the automotive space. The Ford Focus Electric vehicle was announced along with an update to the MyFord Touch interface. The interface features a number of enhancements including the ability to visualize your destination and alert the driver if there isn’t adequate charge in the vehicle’s battery. In addition, an efficiency coach monitors your driving habits to advise changes to your driving style and an “Emotive Display” visualizes butterflies when you are driving in a way that adds range to your vehicle.

MyFord Mobile was also announced. The app allows you to locate charging stations, unlock doors and find the location of the vehicle. In addition, the app goes social with driving behavior monitoring – achievements are awarded once certain milestones are met. These achievements can be shared on Facebook.


Windows Phone 7 Launch – Behind the Scenes Video

May 25, 2010 by in Multi-touch, Portfolio, Technology, Touchscreen

Our team was asked to help launch the Microsoft Windows Phone 7 project at Mobile World Congress 2010. The project was a whirlwind experience – starting with 5 weeks of design/development and 11 days of deployment and support that spanned 2 continents and countless late nights. It was all worth it when Steve Ballmer made the introduction and we were all a part of history as the next generation in mobile experiences was announced to the 50k MWC attendees and a larger worldwide audience.  The people lucky enough to be in attendance couldn’t wait to get their hands on the experience we built.

We setup 10 touchscreens in 2 locations and the experiences were in constant use. Microsoft has since taken the touchscreens to countless other events including MIX10, SXSW 2010, CES, CTIA and many many more.

In addition to the touchscreen experiences, we also worked with our Seattle team to produce a microsite experience that would allow those not in attendance to get a taste of the phone.


RockstAR on Tour: Web 2.0 Expo San Francisco

May 09, 2010 by in Augmented Reality, Mobile, Multi-touch, Technology, Touchscreen

We took the show on the road for the Web 2.0 Expo in San Francisco. We worked with the Microsoft Tag team to bring the RockstAR augmented reality experience to the event.

web20-1

Since we were running the experience in the Microsoft booth, we decided to add some new characters – the most popular of which being Steve Ballmer:

ballmer_shot2

We used the experience as a way to engage with conference attendees and demonstrate an innovative use of Microsoft Tag technology. As conference attendees had their RockstAR snapshot taken, we’d ask them to download the tag reader application to their mobile device. Afterwards, they could take a snapshot of the Microsoft Tag and retrieve their photo. We took over 300 photos at the event.

web20-2

The RockstAR experience is another example of how you can use tag technology to extend an interactive in-store experience to a customers’ mobile device. Wishlists, shopping carts, mobile content delivery, product ratings & reviews and wayfinding are some of the examples of how tag technology can be used to change the way people shop in retail.

Check out our pictures from the event.


The Technology Behind RockstAR

Apr 13, 2010 by in Augmented Reality, Lab, Multi-touch, Technology

We recently had the opportunity to debut the RockstAR experience at SXSW – check out video of the experience in action. We like to think of it as the classic photo booth taken to the next level with augmented reality, multi-touch and social integration. Let’s go behind-the-scenes and take a look at both the software and hardware that brings this experience to life.

RockstAR

First, let’s talk software. The application was built on the recently announced Razorfish Vision Framework. The framework provides a platform to power augmented reality, gestural and other vision-based experiences. For the RockstAR experience, we are analyzing each frame coming from an infrared camera to determine if faces are found in the crowd. Once a face is detected, it is assigned a unique ID and tracked. Once we receive a lock on the face, we can pass position and size information to the experience where we can augment animations and graphics on top of the color camera feed. This technology has a variety of uses. For instance, face tracking can be used to track impressions on static or interactive digital experiences in the retail environment. Here is a screenshot taken from the debug-mode of the experience which shows the face tracking engine at work using the infrared camera.

face tracking

In addition to the vision-based technology, the experience was fully multi-touch enabled – users can gesture on a virtual joystick to swap out bands and snap pictures.

joystick

Because the classic photo booth experience is a social activity, we took it to the next level with twitter and Flickr integration. As pictures were snapped, we’d immediately make them available online. A QR code was rendered with each picture to quickly allow users to navigate to the RockstAR photo on their mobile device. Once the experience is extended to mobile, users can email the pictures to their friends, set it as wallpaper, re-tweet it to their twitter followers, etc.

RockstAR twitter and flickr

Let’s move on to hardware. Unfortunately, you can’t purchase infrared AR-ready cameras at your local Walmart… at least not until Project Natal comes out later this year. Therefore, we needed to build a dual-camera system that would support the face tracking in infrared and the color video feed for display on the screen. We decided to go with 2 commercial-grade Firefly MV cameras with custom lenses.

camera

One of the cameras we modified to see only infrared light by replacing the IR-blocking filter with a IR band-pass filter. This allows only a narrow range of infrared light to reach the camera CCD.

infrared filter

We also purchased and tested a variety of infrared illuminators. These are used to illuminate the environment with invisible infrared light allowing the infrared camera to accurately track faces in low-light conditions.

infrared illuminator

Sparks were flying as we fused the color and infrared cameras together — just another day at the office.

We created a portable rig for the camera and infrared illuminators. Adjustable camera mounts and industrial strength velcro provide flexibility and portability across a variety of installations.

rig2

We used a presentation remote clicker as an alternative way to drive the experience. We primarily used it as a remote camera trigger which allowed us to quickly snap pictures of unsuspecting people from a distance.

clicker

The experience was powered by a 55″ multi-touch screen and a CPU provided by DFI Technologies. We’ve been working with DFI to build PCs that will power the next-generation of interactive experiences. These PCs are small form factor and can be mounted behind the multi-touch screen.

dfi

Last but not least, we bring you the pink rug. We can’t reveal too much information about this technology… we need to keep some things secret. Just know that it is critical to the overall experience.

rug


Audi A1 Multi-Touch Configurators at Geneva Motor Show 2010

Mar 08, 2010 by in Microsoft Surface, Multi-touch, News, Portfolio, Touchscreen

The Razorfish team in Germany partnered with Realtime Technology AG to build configuration experiences for the Audi A1 world premiere at the international motor show in Geneva. They are designed to attract and engage young people and to demonstrate the wide range of customization possibilities of the new Audi.

The first experience is located on the main stage, featuring a 24″ Multi-Touch display allowing users to interact with the car configuration and an additional 65″ display with synchronized high-definition 3d-rendering in real-time to garner even more attention. The complex configuration scenario is wrapped in a simple and easy-to-use interface. The application is based on Windows 7 and the Razorfish Touch Framework.

A1 to drive

A1 to drive

Audi A1

A1 all

The second configurator runs on Microsoft Surface and is based on the Audi A4 configurator. The multi-user environment allows individuals to place physical tokens on the table and configure their favorite A1 in a collaborative way. The extravagant competition kit adds exciting new possibilities to spice up the user’s virtual car.

A1 Surface

A1 Surface

A1 Surface

Both configurators can be seen live at Geneva Motor Show until March 14, 2010.


Windows Phone 7 Series Launch – Day 3

Feb 18, 2010 by in Mobile, Multi-touch, News, Portfolio

Before we left for the evening, we recorded a quick walkthrough of the Windows Phone booth and EMC (Executive Meeting Center) locations where we have touch experiences deployed to support the Windows Phone 7 Series launch event.

Members of the press and blogging community have been recording video of the experience throughout the conference. These videos have begun appearing online – here are a couple of the videos we’ve found:


Windows Phone 7 Series Launch – Day 2

Feb 17, 2010 by in Mobile, Multi-touch, News, Portfolio

day2-boothday2-booth3

After a long night of celebrating the successful launch of Windows Phone 7 Series in Barcelona, we are back at the Windows Phone booth at Mobile World Congress. The crowds are still huge and the experiences are running great. Each experience is collecting touch and interaction information in the background – we are going to begin processing this information to determine how many sessions we are seeing, average session time, the most popular areas of the experience, etc. We will use this information as a guide to optimize the experience for the next event.

day2-booth2day2-crowd

The Windows Phone team is showing live projected demonstrations of the device in the theatre area – these demonstrations are attracting huge crowds.


Windows Phone 7 Series Launch – Demo of Razorfish Touch Experience

Feb 16, 2010 by in Experience Design, Multi-touch, News

1:13pm (+1 GMT) – Day 2, Mobile World Congress. Video of Albert Shum, Director of Mobile Experience Design at Microsoft, using the multi-touch experience built by Razorfish as a tool to explain some of the thinking that Microsoft put into the user interface. Razorfish partnered with the Microsoft team to deliver multi-touch experiences which emulate a Windows Phone 7 Series device. The experience is deployed on multiple 40″ multi-touch displays at Mobile World Congress in Barcelona.


Windows Phone 7 Series Launch – Day 1

Feb 16, 2010 by in Mobile, Multi-touch, News, Portfolio

day1-conference1

Members of the press camped out at the Windows Phone press lounge located across the plaza from Mobile World Congress. Because of the huge turnout for the announcement, much of the press watched the launch event live from the downstairs press lounge. After the show, we launched 6 experiences at this location allowing members the press to touch and interact with Windows Mobile 7 Series for the first time.

day1-conference21

Members of the press who weren’t able to watch the event in the theatre or the press lounge huddled around screens outside in the reception area. We went live with 2 experiences at this location.

day1-crowds1

Conference attendees watching the event live at the Windows Phone booth at Mobile World Congress. We had an additional 2 experiences running at this location.

day1-pictures1day1-pictures2

Cameras were out as the interface was unveiled for the first time. The phone interface design was kept a secret up until launch day. Preventing pictures and other leaks of information from making it to the press turned out to be a huge undertaking. The Windows Phone team went to great lengths to prevent leaks – in fact, many of the Microsoft employees working on the team never had the opportunity to see the interface until launch day. We based our experience off of some hands-on time in Redmond and videos of the experience. Our team was able to reverse-engineer the design, animation and interaction of the user interface. Accuracy was extremely important and we had to ensure the design and motion in our experience was a perfect re-creation of the experience on the actual device. We built the experience on top of the Razorfish Touch Framework. Using the framework allowed us to rapidly develop the application from scratch in under 4 weeks.

The product launch was a huge success and the Windows Phone team has been celebrating in Barcelona. The reaction from the press and blog community has been overwhelmingly positive. The conference is far from over but so far we are off to a great start!