Fresh out of R&D from the Razorfish Emerging Experiences team is a product code-named “5D”. 5D started out as an idea to re-invent personal shopping. Our goal was to create a retail experience platform for both consumers and sales associates that enables multi-channel sales through immersive and connected digital devices in retail environments. And the only way to do it is to seamlessly integrate five key components – devices, content, experiences, analytics and CRM with a touch of digital magic!

The team announced 5D at the 2012 NRF Convention & Expo in New York City in partnership with NEC and Microsoft. Leveraging Windows Embedded, Microsoft Surface, MS Tag, Windows Phone and Kinect for Windows we created a prototype around a fictitious brand “Razorfashion” that demonstrates how various touch points along the customer journey can attract consumers into the store, drive product engagement and arm store associates with more contextualized digital tools.

You can read the full press release here

This year, Razorfish sent several of our people to MIX 11, the annual Microsoft sponsored conference in Las Vegas for developers and designers.

So much happened during our week at MIX  that it is difficult to summarize it all thematically.   There were announcements and sessions on several major topics: IE9, HTML5, ASP.NET MVC 3, Silverlight 5, Windows Phone Mango release, and the Kinect SDK. In addition, there were also appearances from MS Surface v2, Windows Azure, oData and Sharepoint as well as a remarkable set of UX presentations.

The word on everyone’s lips seemed to be fragmentation, whether in reference to the expected HTML5 compatibility issues between future browsers (which the emphasis on the IE9 “native” browser experience only exacerbated) or to the greater array of Microsoft development technologies fighting for developers’ attentions.

What the four Razorfish attendees at MIX saw, on the contrary, were patterns of evolution.  The much ballyhooed struggles between the Windows Team and the Development Team inside Microsoft for the future of HTML5 and Silverlight indicate to us that Microsoft can still respond to a rapidly changing worldwide technology ecosystem.  When a product is struggling in the niche it was doing fine in a year ago, it can be refitted to survive in a new niche. Such is the case with Silverlight, originally intended as a Flash-killer.  Silverlight developers never truly adopted the original Flash-killer strategy and instead used Silverlight to develop more sophisticated and interesting line-of-business applications.  The problem is that LOB applications do not really belong on the web.  They belong behind firewalls.  The lack of casual games written in Silverlight likely affected the ability of Silverlight to force downloads and gain browser share.  So instead, the strengths of Silverlight are being moved to the desktop as well as specialized platforms such as Windows Phone, the XBOX (?), and possibly Windows vNext.

WPF, which was once the pre-eminent desktop development platform, is in turn becoming a specialized tool for NUI development for multi-touch, Surface and Kinect.  The announcement of the Kinect SDK itself demonstrates Microsoft’s continuing ability to innovate and surprise.  It is, in the best sense of the term, a fortuitous mutation.

This all leaves HTML5 as the preferred technology for the web.  We of course see the early signs of browser compatibility issues. At the same time, though, we have each been through this before and survived. The extra gyrations developers will have to go through will, in the end, provide the illusion consumers desire – that the same application can run similarly on any operating system and any device.  As one MIX speaker put it, “The technology you use impresses no one.  The experience you create with it is everything.”

Speaking of devices, we are excited to see that the WP7 team is not only going for parity with other smart phones but is firing warning shots across their bows with the much touted Mango release.  Features we’re used to like multitasking are being expanded beyond current implementations with updating live tiles and “Live Agents” which allow for more full-featured multitasking.

There was naturally some complaining about the placement of various keynotes and sessions.  With the multiple announcements and cross-blocking sessions, isn’t there a danger that individual messages will get drowned out in the general cacophony?  We find that the panoply of conflicting viewpoints is one of the chief charms of MIX. Microsoft is not Apple.  To borrow from Isaiah Berlin’s famous title, Apple is the hedgehog that does one thing well; Microsoft is the fox that explores all avenues and experiences.  The great strength of Microsoft is its ability to challenge developers and create new harmonies out of these encounters. Should MIX ever be split up into different web, Silverlight, Windows Phone and UX conferences, we would all be poorer for it since all we would ever get would be our own opinions reflected back on ourselves – an echo chamber effect that will only serve to make us all deaf.

The overall quality of all sessions and boot camps were extremely high this year.  In the past, we have been happy with a 60% success rate on talks.  This year roughly 85% of the talks rang our internal bells. Certain sessions deserve a special shout out, however.

While all the UX lightning talks were extraordinary,  August de los Reyes’s 21st Century Design (10’ 45”) talk took it to a different level.  In the live session, the slide deck itself was the star with the brilliant August narrating it much as Peter Jones was the voice of the book in the old Hitchhiker’s Guide to the Galaxy television series.

Despite its inauspicious title, Ivan Tashev’s talk Audio for Kinect revealed what a truly remarkable device the Kinect really is. We honestly didn’t understand half of the technical stuff and we became queasy when formulas started flying across the screen. What we learned, though, was that only a fragment of the Kinect’s full audio capability is currently being used.  Dr. Tashev demonstrated the ability of the Kinect’s audio algorithms to pick out two separate speakers, one reading Chinese and the other reading Korean, and separate them into different channels.  All of this cool functionality will, moreover, be handed over to developers when the Kinect SDK beta is released at the end of spring.

Finally, we cannot say enough good things about Luis Cabrera and his willingness to demonstrate the Surface 2 at work in A Whole NUI World. Razorfish, of course, has a special affinity for anything Surface. What was outstanding in this presentation was not only the beauty and power of the new Surface devices but also the amount of thought that has gone into the tooling. Kudos to the Surface team, they’re reaching for a goal that is more than just a new technology but a new way for people to interact with computers and each other.

By the end of MIX, we were all quite exhausted mentally and physically. It may take us a full year – until the next MIX – to finish ingesting everything that we learned and experienced at MIX11.

So long, Microsoft, and thanks for all the Kinects.

One of the questions we constantly asked ourselves was how we could utilize Microsoft Surface as an easy and collaborative sharing platform. Surface is a product designed for social environments, attracting and engaging people to take a seat around the table and interact. As smartphones take over the mobile market, it’s important to recognize that people expect to have intuitive ways to connect their handsets and share pictures, information or other data. But we still haven’t seen a solution which leverages the whole Surface potential to create a compelling mobile sharing experience.

This is where our newly developed Amnesia Connect platform comes in. It demonstrates the most seamless and visual sharing opportunities a Surface table can probably provide. With support for any number of mobile devices, users can literally see-through their device and share content as easy and tactile as it can get. It is perfectly suited for sharing visual data like images and video, but works for any other type of content as well.

At the moment we support iPhone and iPad devices and are currently evaluating other platforms as well. But we can already tell that it’s heart-warming to see Microsoft’s and Apple’s devices playing so nicely together and we can’t wait to throw other players into the mix as well.

Read the full press release here.

The Consumer Electronics Show was back for 2011 and our team was on the ground in Las Vegas. We have a number of initiatives going on at CES this year.

First, our team was involved in the Microsoft Surface 2.0 launch. We’ve been working with the Surface team for a few months on the next generation of Surface. We’ve been porting our applications to run on the latest version. We can proudly announce that we are Surface 2.0 ready and we look forward to supporting the new platform and bringing the solution to our clients. The Microsoft Surface announcement caught the media by surprise – it’s been over 3 years since the original Surface was announced. The new device is faster, leaner and costs less than the previous version of Surface. We’ll have an in-detail analysis of Microsoft Surface 2.0 posted on the blog shortly.

Second, we were involved in the launch of another experience for one of our clients. We created a solution that will be experienced by millions of consumers in the market. Unfortunately our involvement must remain confidential so we can’t go into too many details. Let’s just say it was definitely one of those opportunities that we could not pass up!

We took the opportunity to explore the trade show floor in an effort to educate ourselves on the latest technology offerings. We hope to bring some of these technologies to our clients in 2011. Here are some of the technologies that we’ve got our eye on.

Tablets

2011 has been declared the “year of the tablet”. There were certainly no shortage of tablets at CES. In fact, about 80 new tablet form-factor devices were announced at CES this year.

From a hardware perspective, tablets are getting thinner, lighter and more powerful thanks to innovation around chip technology from companies like Intel, ARM, nVidia and Qualcomm. There are a variety of new form-factors hitting the market. The Eee Pad Transformer tablet can be docked in a base which transforms the device into a traditional laptop form-factor. The Dell Inspiron Duo tablet features a reversible screen to accomplish the same thing.

There were a variety of different screen sizes available. One of the interesting debates between amongst members of our team was around the usefulness of the small screen tablets. These “tweener” devices feature screens between the size of a typical phone and an iPad. The smaller size means they are more portable than an iPad, however they still can’t fit in your pocket and they can’t make phone calls.

One of the most impressive devices was the BlackBerry Playbook. The device features a brilliant user interface which makes use of NUI design principals – direct interaction of content through the use of gestures. In addition, the performance of the device was exceptional. We can’t wait to start developing for this platform.

For the first time, we had the opportunity to see the new Android Honeycomb tablet OS. The exerience is decidedly Android retaining much of the same design language. Improvements have been made to the user interface to take into account the additional tablet real-estate. In all honesty, we were slightly disappointed with the user interface. We were hoping for something game-changing from Google and instead, they delivered an experience that was transitional, not transformational.

One of the major disappointments was the lack of direction from Microsoft on tablet devices. We were crossing our fingers for an announcement around a tablet operating system that was lightweight and provided an exceptional user experience similar to what is being provided on Windows Phone 7 platform. And we wanted this platform soon.

Microsoft did acknowledge they are behind in the space. Right now, their story is positioning Windows 8 as the solution for tablets by supporting system-on-a-chip architecture. By supporting this hardware platform, Microsoft will be able to deliver Windows experiences on tablet devices while taking into account battery life and OS performance.

Unfortunately, no announcements were made around the Windows 8 user interface. Delivering an exceptional tablet UI will be essential to their strategy. It is likely Microsoft will adopt the “Metro” design language currently being used for Windows Phone 7 and Microsoft Surface 2.0.

Gesture Control and Natural Interaction

With the release and success of Xbox Kinect, the gesture control market is heating up. Much like the original iPhone brought touch interaction into the mainstream by putting millions of devices in the hands of consumers, Xbox Kinect will do the same for gesture control. The way we interact with computers is fundamentally changing and we are getting in on the ground floor.

We’ve taken the opportunity to develop for the Kinect platform, however we were looking for a commercial-grade solution to bring to our clients. Enter PrimeSense.

PrimeSense licenses their technology to Microsoft for use in the Xbox Kinect, therefore they seemed like the perfect partner to deliver the hardware and software to support commercialized gesture control solutions. We are actively working with PrimeSense to develop for their platform. Their OpenNI initiative hopes to create a framework for standardization of natural interface development across devices.

We see gesture control technology being used in an in-home setting and also in retail environments. This technology can be utilized to create at-home shopping experiences which combine natural interaction and augmented reality. Imagine being able to virtually try on clothes from the comfort of your own home. Or order a pizza with a flick of the wrist from the comfort of your couch.

We have been champions of the use of interactive experiences in the retail environment and we have the statistics to prove it. To date, the majority of our experiences have utilized touch. This technology provides a new user interaction paradigm and offers an entirely new world of possibilities in the retail space.

Touch Screens

Touch screen technology is evolving rapidly. Devices are becoming larger, cheaper and more reliable. Exciting new form-factors and multi-touch hardware will help us deliver new experiences to our clients in 2011.

3M Touch Systems has exciting new hardware and form-factors hitting the market which utilize their massively multi-touch projected capacitive technology. This technology provides extremely stable multi-touch that supports a large amount of touch points. 3M is brining 23” and 32” screen sizes to the market. In additional, the screens can be integrated into a multi-device array to build large size touch wall and table experiences.

We also had some hands-on time with systems from PQ Labs and Multitouch.fi. Both vendors offer touch solutions that are unique and exceptional. We look forward to working with these companies in the future.

Display Technology

Displays are getting thinner, lighter and more energy efficient. 3D technology is also evolving quickly. Much like last year, 3D display technology was everywhere. The most impressive innovation in the 3D TV space comes from LG. They demonstrated how their 3D technology has been standardized – every TV on display in their booth could utilize the same pair of glasses to deliver an exceptional 3D experience. They also demonstrated flicker-less 3D which produced a better 3D picture than we’ve seen on any other consumer device.

The glasses-less 3D technology was a disappointment. There isn’t enough discernable depth with the current iteration of the technology. Certainly this will change over time, however the promise of ditching the glasses has yet to be fulfilled. We wouldn’t be surprised if this changes in 2011.

In-Car Technology

Ford had the major innovations in the automotive space. The Ford Focus Electric vehicle was announced along with an update to the MyFord Touch interface. The interface features a number of enhancements including the ability to visualize your destination and alert the driver if there isn’t adequate charge in the vehicle’s battery. In addition, an efficiency coach monitors your driving habits to advise changes to your driving style and an “Emotive Display” visualizes butterflies when you are driving in a way that adds range to your vehicle.

MyFord Mobile was also announced. The app allows you to locate charging stations, unlock doors and find the location of the vehicle. In addition, the app goes social with driving behavior monitoring – achievements are awarded once certain milestones are met. These achievements can be shared on Facebook.

The launch of Xbox Kinect has caused much excitement in the open source community. In the last few weeks, developers have managed to tap into the hardware with impressive results. We’ve seen applications ranging from gesture-based experiences to 3D imaging.

We’ve taken this exciting opportunity to port our popular DaVinci experience to the Kinect platform. Gestures are used to create objects and control the physics of the environment. Your hands appear in the interface which allows you to literally grab objects out of thin air and move them in the environment. Additional gestures allow you to affect the gravity, magnetism and “planetary attraction”.

To date, many of the experiments in gestural interface development have not taken into account the hands. Unfortunately, the result is an experience that isn’t precise – users have no context of where they are interacting in the virtual space and 1-to-1 manipulation of objects in a scene proves difficult. By using an clenched hand to signify “grabbing” an object and an open hand to signify “releasing” an object, we are able to create experiences with an higher level of precision which can mimic a touch based experience. In fact, we’ve created a Kinect plugin to enable our entire suite of touch-based experiences to work with gestures – more videos to come!

Gesture-based interaction is great when touch isn’t practical. For instance, on a large screen projected display as shown in the video above it is difficult or physically impossible to control the entire area using touch. Using a technology like Kinect, we can create a virtual canvas in mid-air in front of the user. Interactions within this virtual canvas space are projected into the experience as shown in the DaVinci example.

To be honest, we had a blast playing with this experience. It definitely fulfilled all of our Star Wars fantasies of controlling objects with your mind. We’ll be adding more features in the coming weeks including the Darth Vader death grip. Stay tuned!

“Control, control, you must learn control.” – Yoda

The Razorfish team in Germany partnered with Realtime Technology AG to build configuration experiences for the Audi A1 world premiere at the international motor show in Geneva. They are designed to attract and engage young people and to demonstrate the wide range of customization possibilities of the new Audi.

The first experience is located on the main stage, featuring a 24″ Multi-Touch display allowing users to interact with the car configuration and an additional 65″ display with synchronized high-definition 3d-rendering in real-time to garner even more attention. The complex configuration scenario is wrapped in a simple and easy-to-use interface. The application is based on Windows 7 and the Razorfish Touch Framework.

The second configurator runs on Microsoft Surface and is based on the Audi A4 configurator. The multi-user environment allows individuals to place physical tokens on the table and configure their favorite A1 in a collaborative way. The extravagant competition kit adds exciting new possibilities to spice up the user’s virtual car.

Both configurators can be seen live at Geneva Motor Show until March 14, 2010.

We’ve been busy traveling the globe demonstrating our touch experiences. The reaction from clients and the industry has been great – we’ve met some wonderful people along the way. Recently we were at the Microsoft Vendor Fair showing our Surface and vertical multi-touch experiences. With the Windows 7 release, the excitement within Microsoft is understandable. Much like our other clients, we’re working with Microsoft in realizing the potential of touch related experiences from a sales and marketing perspective. To see evidence of this look no further than the Microsoft retail store.

Razorfashion goes international. We jumped at a chance to support our Neue Digitale team in Germany at DMEXCO (Digital Marketing Exposition Conference). To support the effort, we’ve been working with a company in Germany on an experimental multi-touch device named dreaMTouch. Once we figured out how to get the pre-release device through customs, we were able to integrate the hardware with the Microsoft-based Razorfish Touch Framework in under a day. Since integrating with the hardware was so easy, we decided to feature the Razorfashion application during the conference as well.

Neue Digitale / Razorfish developed the first realtime 3d configurator for Microsoft Surface which was showcased at this year’s IAA – the world’s leading automotive fair.

Developed in collaboration with Realtime Technology AG, the application allows multiple users to configure an Audi A4 simultaneously by changing the car’s paint finishes, rims and by selecting and coloring style package components. The configured A4 is experienced in an immersive 3d environment, in which users can navigate seamlessly by zooming and panning using a multitouch-enabled interface. Special perspectives allow a more detailed view for specific components, for example for configuring rims or taking a closer look at interior details.

To control the actual car configuration process users can place multilingual tokens on the table. All available options appear around the token and can be selected by touching or rotating the physical token.

The final car configuration is projected on an external High Definition display, attracting and engaging other visitors.

Iain McDonald with Amnesia Razorfish introduced the Lonely Planet Surface application at Remix Australia – check out the video!

Check out the above technology demonstration of the multi-touch Virtual Earth control and the Razorfish Touch Framework. This video demonstrates a weather experience and the cross-platform capabilities of the framework on Microsoft Surface or other touch-enabled devices.