Fresh out of R&D from the Razorfish Emerging Experiences team is a product code-named “5D”. 5D started out as an idea to re-invent personal shopping. Our goal was to create a retail experience platform for both consumers and sales associates that enables multi-channel sales through immersive and connected digital devices in retail environments. And the only way to do it is to seamlessly integrate five key components – devices, content, experiences, analytics and CRM with a touch of digital magic!

The team announced 5D at the 2012 NRF Convention & Expo in New York City in partnership with NEC and Microsoft. Leveraging Windows Embedded, Microsoft Surface, MS Tag, Windows Phone and Kinect for Windows we created a prototype around a fictitious brand “Razorfashion” that demonstrates how various touch points along the customer journey can attract consumers into the store, drive product engagement and arm store associates with more contextualized digital tools.

You can read the full press release here

We recently partnered with London-based technology company, BodyMetrics, to develop a means for online shoppers to buy clothes from the comfort of their couch. Whattya mean big deal? Well, did we mention that the clothes are guaranteed to fit?

Yup, thanks to BodyMetrics’ 3D body-scanning technology, which is based off of the same PrimeSense scanners and camera tech as the Microsoft Kinect, shoppers are able to have their body dimensions scanned in and saved to an online profile. Just think of it like the transport room in Star Trek … if Scotty had a bit of an online shopping problem.

Once users have created their profile and saved their body data, they can virtually try on a wide range of clothing types such as jeans, dresses, skirts and tops from tons of partner retailers. As each piece of clothing is mapped to the on-screen avatar’s body, the user is able to see the exact fit of the item thanks to a visual overlay that depicts the tight spots of the garment. No more guessing games when you buy that pair of jeans online – you get the perfect fit, every time.

The icing on the cake – retailers get to benefit from a drastic drop in their store return rates since their customers can finally purchase with confidence. That, coupled with the exponential momentum and increased basket-size of eCommerce purchases means great things for apparel companies. Plus, you don’t have to listen to some phony sales associate squawking about how fabulous you look in those jeans – just take a look for yourself!

AT&T’s Small Business division asked the Emerging Experiences Group within Razorfish to create an in-store experience as part of a 30 store pilot program to create better awareness and engagement with customers as well as provide associates more powerful sales tools. The project has been a big success and you should be able to take a spin in a store near you soon.

Occassionally we get downtime on the Emerging Experiences team — but not very much.  After a recent project we decided to play with some of the technology we have lying around.  One of our favorite toys for 2012 is the high-end projector (pick yours up for Christmas now!).  With a powerful projector you can create great interactive experiences on glass, on store windows, on mist, on water and so on.  When a projector is pointed at a flat surface, it creates the illusion of depth.  Lately we have been playing with the idea of shining a projector on surfaces that already have depth.  Next step — hooking this up to an array of Kinect sensors to see what projecting on 3D surfaces that interact with 3D depth cameras will look like.

Over the last few weeks Amnesia Razorfish had the opportunity to collaborate with the University of Sydney and Publicis Mojo for an innovative take on how near field communication can transform tasks in our daily lives. For this concept designed for food courts Stephen Davis created a Brand Table prototype that not only simplifies the process of displaying menus of restaurants on your phone, but also allows you to place your order and pay instantly.

Steve has a nice write-up on his website about the benefits of this approach and we were happy to see it being picked by a number of online publications including TechCrunch and Engadget.
With the wide-spread adoption of NFC at the horizon we’ll hopefully see concepts like this becoming reality soon.

Before the BUILD conference, the one thing we all knew was that Microsoft needed a multitouch tablet strategy to compete with Google and Apple and in order to maintain the future viability of the Windows operating system. What we were not sure of was how Microsoft would achieve this goal while preserving backwards compatibility for all of our previous Windows applications in the office as well as in the home. The challenge at first blush seemed insurmountable: provide something completely new to the Windows world while preserving everything that went before.

At BUILD, Microsoft revealed that they have actually accomplished this goal by providing what is basically two side-by-side operating systems. They have also signaled that the primary challenge for application creators going forward will not be technical but rather design-focused. Microsoft, which in the past has tended to side-line user experience, now puts design front and center with “Windows 8.”

One of the two Win8 interfaces is a slightly souped up version of Windows 7 that looks familiar and runs just about anything I could think to try installing on it: Zune, Dropbox, iTunes, Kindle for PC, and even the software for the Kinect SDK. “Windows 8” ran each of them without complaint. The desktop shell works best with a mouse and keyboard, though it also supports and has been redesigned to support multitouch also.

The other is a Metro inspired immersive experience that works best using touch. Instead of an explorer based file system with icons, the Metro shell is designed around interactive tiles, familiar from Windows Phone 7, that launch discrete apps. The Metro shell revolves around a new Windows Store (the equivalent of the iPad’s App Store and WP7’s Marketplace) that allows consumers to download games and apps.

Dual Mode with Dock

One could easily think of this as two solutions in one: a consumer platform designed for the tablet and a desktop platform designed for the PC. What is unusual about these side-by-side solutions is that, with the flick of a finger, the tablet user can bring up the desktop UI and the desktop user can bring up the Metro UI. The two operating systems are not something one configures through the control panel the way one might configure a background theme. Instead, both UIs are effectively always alive and always immediately accessible.

Microsoft generously provided each attendee with a new Samsung tablet installed with a pre-Beta build of “Windows 8” and accessorized with a wireless keyboard, a stylus and a dock. The dock is by far the most intriguing – and least discussed – piece of hardware provided as it offers an indication of how Microsoft envisions “Windows 8” being used in the future. A tablet may be inserted into the docking station with a monitor and mouse in an office setting, at which point the desktop UI can be brought up and the user has an experience for the most part indistinguishable from what he is currently used to. The tablet can then be undocked and switched to the Metro style with all the previously running applications still running.

The notion of two different operating systems goes all the way down to the development stack with two different platforms: one based on the traditional tools of Microsoft development such as Silverlight, WPF and WinForms targeting the desktop UI and another, completely new, development stack built around something called the Windows Runtime (WinRT) and programmable using C# and VB with a XAML-style UI language or C++ or Html with javascript compiling to a combination of native and dynamic code.

Initially the expectation is that the .NET tools of the past ten years will be used to write business, productivity and data-entry intensive applications while the new tools will be used for games, social apps and everything else one might expect to find on a smartphone or an iPad.

In a mixed-OS experience like the one described above using a dock, a more likely setup would be a full .NET style business app for the desktop with a lighter-weight Metro style version of the same app on the Metro UI. This allows users to quickly switch back and forth between a tablet and a desktop scenario using the same device. The test of this will likely come when Microsoft reveals its plans for Microsoft Office. We would expect Microsoft to provide both a classic and a Metro version of their Office suite. How well they implement this will in turn provide a roadmap for how other vendors will cater to the enterprise in their software solutions. In other words, will “Windows 8” for the enterprise have enterprise applications for the desktop only or for both the desktop and for the Metro UI.

There is a third possibility also. It may be possible to build full enterprise applications targeting Metro only. The WinRT platform combined with Microsoft’s Azure offering supports this.

The challenge in creating sophisticated apps for Metro is not primarily a technical challenge. It is primarily a User Experience challenge. Can we create multitouch enabled data grids? Can we come up with new navigation patterns to replace the standard enterprise application with hundreds of unique windows? Can we find ways to create great experiences that combine both multitouch and keyboard interaction?

Jensen Harris

While Microsoft has been tagged with a reputation for not understanding UX over the past decade, this has seemed to change. At BUILD, the speakers were all aware of the importance of UX while speakers like Jensen Harris demonstrated that Microsoft not only knew that UX problems were important but that they also had the chops to solve them. In this context, BUILD has been a watershed event. If Microsoft has tended to admire and promote smart programming in the past, after BUILD it will become more important to be savvy about design. The days when design could be dismissed as merely prettying up an application are over. After this week, design on Windows is front and center. This is good news for agencies like Razorfish which are strong in both design and technology. It will be a challenge for software consultancies that have only been paying lip service to UX until now as they attempt to establish themselves as Metro experts.

On the technology front, as mentioned above, Microsoft is supporting three platforms: one targeted at C++ developers, one at XAML developers (Silverlight and WPF), and one targeted at web developers. The tack of using web technologies for building native Metro apps for the “Windows 8” tablet currently makes the most sense. A common path for developing apps for multiple platforms like the iPad and iPhone, Android and Windows is to first create a web application that can run on all these platforms, then after looking at web analytics data and determining which platforms use the web app most, building native apps for each of the top platforms. In the case of the Metro UI, it will be easiest to port code from web apps to the native web development tools on Windows 8 rather than attempt to build a brand new project in either C++ or XAML. Again, this type of development is already familiar to digital agencies but likely to be a challenge for other organizations.

BUILD also quietly announced improvements and fixes to WPF in the new .NET 4.5 framework being released with “Windows 8.” This is exciting for the Razorfish Emerging Experiences group as WPF is our main development platform for Surface applications and multitouch experiences.

Search Results for Silverlight at BUILD

The story for Silverlight is a bit more ambiguous. Currently “Windows 8” offers two different versions of IE 10 – one for the desktop UI and one for the Metro UI. The Metro UI version does not support plugins. Consequently neither Flash nor Silverlight applications will run in Metro IE. This is a difficult position since it entails Silverlight does not work as a multi-platform solution even on “Windows 8,” i.e. it supports only one of the two Win8 platforms. Silverlight-out-of browser is still viable on the desktop UI. It must compete, however, with both WPF – which is more feature rich – and WinForms – which has a significantly larger developer base. It has been suggested that the main benefit of Silverlight as a desktop technology solution will be that, since it, like XAML for WinRT, is only a subset of WPF, this will make things easier when it comes time to port an application over to the Metro UI. In porting from either Silverlight or WPF, however, some rewriting will have to occur as XAML for WinRT actually introduces interesting new XAML features – such as markup for localization – currently missing from both Silverlight and WPF.

There are still several open questions remaining with regard to Windows 8. Two have already been mentioned:

1. Is the Metro UI for the enterprise or for consumers only?
2. What are Microsoft’s plans for Microsoft Office?

Waiting in line for Win8 Tablets

A third open question is What are Microsoft’s plans for Windows Phone? While the Samsung tablets given to attendees at BUILD are Intel-based, Microsoft’s ultimate goal is to provide an ARM-based version of “Windows 8.” The great advantage of an ARM architecture is that it allows “Windows 8” to be placed on a variety of hardware platforms including smart phones. Currently, however, Microsoft has dropped no hint that it plans to release “Windows 8” phones, however, and the concern would be that such an announcement would damage sales of Windows Phone 7.5, which will be released sometime in 2011. On the other hand, it doesn’t seem to make sense to have completely different operating systems for the Microsoft tablet and Microsoft’s phone. Apple has benefited greatly by having one OS for both form factors and Microsoft strategists, no doubt, are well aware of this.

This year, Razorfish sent several of our people to MIX 11, the annual Microsoft sponsored conference in Las Vegas for developers and designers.

So much happened during our week at MIX  that it is difficult to summarize it all thematically.   There were announcements and sessions on several major topics: IE9, HTML5, ASP.NET MVC 3, Silverlight 5, Windows Phone Mango release, and the Kinect SDK. In addition, there were also appearances from MS Surface v2, Windows Azure, oData and Sharepoint as well as a remarkable set of UX presentations.

The word on everyone’s lips seemed to be fragmentation, whether in reference to the expected HTML5 compatibility issues between future browsers (which the emphasis on the IE9 “native” browser experience only exacerbated) or to the greater array of Microsoft development technologies fighting for developers’ attentions.

What the four Razorfish attendees at MIX saw, on the contrary, were patterns of evolution.  The much ballyhooed struggles between the Windows Team and the Development Team inside Microsoft for the future of HTML5 and Silverlight indicate to us that Microsoft can still respond to a rapidly changing worldwide technology ecosystem.  When a product is struggling in the niche it was doing fine in a year ago, it can be refitted to survive in a new niche. Such is the case with Silverlight, originally intended as a Flash-killer.  Silverlight developers never truly adopted the original Flash-killer strategy and instead used Silverlight to develop more sophisticated and interesting line-of-business applications.  The problem is that LOB applications do not really belong on the web.  They belong behind firewalls.  The lack of casual games written in Silverlight likely affected the ability of Silverlight to force downloads and gain browser share.  So instead, the strengths of Silverlight are being moved to the desktop as well as specialized platforms such as Windows Phone, the XBOX (?), and possibly Windows vNext.

WPF, which was once the pre-eminent desktop development platform, is in turn becoming a specialized tool for NUI development for multi-touch, Surface and Kinect.  The announcement of the Kinect SDK itself demonstrates Microsoft’s continuing ability to innovate and surprise.  It is, in the best sense of the term, a fortuitous mutation.

This all leaves HTML5 as the preferred technology for the web.  We of course see the early signs of browser compatibility issues. At the same time, though, we have each been through this before and survived. The extra gyrations developers will have to go through will, in the end, provide the illusion consumers desire – that the same application can run similarly on any operating system and any device.  As one MIX speaker put it, “The technology you use impresses no one.  The experience you create with it is everything.”

Speaking of devices, we are excited to see that the WP7 team is not only going for parity with other smart phones but is firing warning shots across their bows with the much touted Mango release.  Features we’re used to like multitasking are being expanded beyond current implementations with updating live tiles and “Live Agents” which allow for more full-featured multitasking.

There was naturally some complaining about the placement of various keynotes and sessions.  With the multiple announcements and cross-blocking sessions, isn’t there a danger that individual messages will get drowned out in the general cacophony?  We find that the panoply of conflicting viewpoints is one of the chief charms of MIX. Microsoft is not Apple.  To borrow from Isaiah Berlin’s famous title, Apple is the hedgehog that does one thing well; Microsoft is the fox that explores all avenues and experiences.  The great strength of Microsoft is its ability to challenge developers and create new harmonies out of these encounters. Should MIX ever be split up into different web, Silverlight, Windows Phone and UX conferences, we would all be poorer for it since all we would ever get would be our own opinions reflected back on ourselves – an echo chamber effect that will only serve to make us all deaf.

The overall quality of all sessions and boot camps were extremely high this year.  In the past, we have been happy with a 60% success rate on talks.  This year roughly 85% of the talks rang our internal bells. Certain sessions deserve a special shout out, however.

While all the UX lightning talks were extraordinary,  August de los Reyes’s 21st Century Design (10’ 45”) talk took it to a different level.  In the live session, the slide deck itself was the star with the brilliant August narrating it much as Peter Jones was the voice of the book in the old Hitchhiker’s Guide to the Galaxy television series.

Despite its inauspicious title, Ivan Tashev’s talk Audio for Kinect revealed what a truly remarkable device the Kinect really is. We honestly didn’t understand half of the technical stuff and we became queasy when formulas started flying across the screen. What we learned, though, was that only a fragment of the Kinect’s full audio capability is currently being used.  Dr. Tashev demonstrated the ability of the Kinect’s audio algorithms to pick out two separate speakers, one reading Chinese and the other reading Korean, and separate them into different channels.  All of this cool functionality will, moreover, be handed over to developers when the Kinect SDK beta is released at the end of spring.

Finally, we cannot say enough good things about Luis Cabrera and his willingness to demonstrate the Surface 2 at work in A Whole NUI World. Razorfish, of course, has a special affinity for anything Surface. What was outstanding in this presentation was not only the beauty and power of the new Surface devices but also the amount of thought that has gone into the tooling. Kudos to the Surface team, they’re reaching for a goal that is more than just a new technology but a new way for people to interact with computers and each other.

By the end of MIX, we were all quite exhausted mentally and physically. It may take us a full year – until the next MIX – to finish ingesting everything that we learned and experienced at MIX11.

So long, Microsoft, and thanks for all the Kinects.

When it comes to travel, people care more about where they’re going than how they’re getting there. Delta Airlines understands this and asked Razorfish’s Emerging Experiences team to create an engaging experience for the WIRED Holiday Store in NYC. We wanted to tap into users’ imagination and sense of playfulness so that they walk away from the experience thinking about what kind of destinations they want to go to next and, of course, Delta.

In four (4) weeks we concepted, designed, developed and launched The Untravel Idea – a new, personal way for leisure travelers to encounter destinations. We wanted users to touch the future of travel.

The experience gives people an open-ended, creative experience that puts the user in control. First users choose the type mood they are looking for on their next getaway. From there users can select from a wide variety of relevant words that match their mood, and when put together, show them a range of destination possibilities. The result is a beautiful montage of photographic imagery that will transport the user’s imagination.

To extend the experience beyond the store, users are prompted to use their mobile device to snap a pic of a QR tag associated with each destination that allows them to explore additional destination info, video and travel packages.

The event was a complete success. People couldn’t wait to see where Delta would take them next. The Takeaway – Delta is not just an airline; they’re giving me new ways to discover travel destinations.

This is the future of travel … and it’s just the beginning.

A wide range of Oakley products are designed for sports fans and outdoor living people who are dependent on their equipment when practising their passion and living their dreams. Choosing the right sunglass lens makes a significant difference when sports and outdoor activities are taken seriously. To guide the consumer through this decision process, we have implemented an iPhone and iPad App which simulates realistic scenarios by using engaging 3d-panorama landscapes wrapped in an intuitive touch- and accelerometer-based interface.

The overall experience features more than 18 lens tints in spectacular environments and various weather conditions. Once the perfect lens is selected, a detailed product information is just one touch away.

Available on the AppStore for iPad and iPhone.

We recently created an experience named RockstAR which features augmented reality and multi-touch technology. It is the classic photo booth experience taken to the next level with interactive technology, social integration (currently the experience posts to twitter, twitpic and flickr), good ole fashioned Rock n Roll and a little 80s video game nostalgia. We also can’t leave out the pink rug – one of the most important parts of any experience.

The application is the first demonstration of the Razorfish Vision Framework (RVF) and it is integrated with our Razorfish Touch Framework (RTF). The experience was featured at several SXSW Interactive Conference 2010 events including the Razorfish and Microsoft parties.

Stay tuned as we’ll be posting a behind-the-scenes tech walkthrough in the next week.