Multi-touch

Bringing 5D to Life at NRF

Jan 14, 2013 by in 5D, Augmented Reality, Experience Design, Kinect, Microsoft Kinect, Microsoft Surface, Multi-touch, Near Field, News, Portfolio, Retail, Technology, Touchscreen

Get your hands on the 5D experience by embarking on a unique shopping journey that utilizes a variety of platforms and technologies, including a first of it’s kind, seamlessly-synchronized transparent interactive display wall. It’s located in the Microsoft booth (1005) on Level 3. And to see more of 5D in action, head on over to emergingexperiences.com/5D.

Tags:

We’re building the future of retail (and it’s kind of messy)

Jan 11, 2013 by in 5D, Augmented Reality, Experience Design, Kinect, Lab, Multi-touch, Near Field, News, Portfolio, Retail, Technology, Touchscreen

Our lab is buzzing with activity as the team prepares for the National Retail Federation’s 102nd Annual Convention & EXPO in New York. On display will be the latest iteration of Razorfish 5D— the world’s first cross-device, cross-OS, connected retail platform. Launched at last year’s NRF convention, 5D has already been launched in several markets and was used to create Audi City London, a one-of-a-kind immersive virtual showroom. This year we’re showing how our platform can power customized, personalized and seamlessly synchronized shopping experiences. We threw in some augmented reality and a bunch of transparent displays as well.

Our team will be demonstrating the 5D experience in booth #1005 on Level 3 of the Jacob K. Javits Convention Center. If you can’t make the show, be sure to follow us on Twitter to get the latest updates.


Audi City London is Contagious

Dec 12, 2012 by in 5D, Experience Design, Kinect, Multi-touch, News, Retail, Technology, Touchscreen

Most Contagious, that is.

We’re excited to announce that Audi City London has claimed Contagious Magazine’s Most Contagious Retail Award at a ceremony today in London. This experience was a year-long collaborative effort between Audi and a wide range of partners, and was launched near Piccadilly Circus just ahead of the summer Olympics. It is delivered by one of the most technologically advanced retail environments ever created and features a variety of multi-touch displays for configuring your Audi from millions of possible combinations. Once you’ve created your personalized Audi at this groundbreaking dealership, you can toss it onto one of the floor-to-ceiling digital “powerwalls” to visualize and explore your configuration at a true 1:1 scale. Audi City London is a true dealership of the future and an effort we were proud to be part of.

Photo: Gaurav Singh

Tags: ,

Enhanced Consumer Connections, Powered by Razorfish 5D

Nov 15, 2012 by in 5D, Experience Design, Kinect, Lab, Microsoft Surface, Mobile, Multi-touch, Near Field, Retail, Technology, Touchscreen

When we’re playing in our Lab, we’re always looking for creative ways to push the limits of technology. Some of our projects are just for fun, and others, like London’s Audi City, completely reinvent the way people shop. We were even thinking about digital wallets before they were cool. So when we set out to create the Razorfish 5D platform, our goal was to design a powerful and highly immersive way for brands to connect with consumers—before, during and after the shopping experience. In our latest video, we show how our 5D platform seamlessly connects a variety of digital devices to better attract consumers into the store, drive product engagement and arm store associates with more contextualized digital tools. The end result is a fun and personal experience, the way shopping should be.


Hidden Tech Features in Windows 8

Nov 12, 2012 by in Microsoft Surface, Multi-touch, Technology, Touchscreen

 

With Microsoft’s Build 2012 over and Surface now available for purchase, it seems like the only thing Microsoft developers can talk about is building Windows 8 store apps. Creating Windows apps that can be monetized using the same wildly successful revenue sharing model that Apple pioneered is compelling. However, in the rush to cash in on this new frontier it’s easy for developers to lose sight of the fact that Windows 8 is far more than the addition of a new shell and a new application model.

As a product, Windows has been continuously developed for over fifteen years (which is far longer than Apple has been using the revenue sharing monetization model), but this time Microsoft has built some great technologies that have received far less attention.

The first that comes to mind is DirectX. When most developers think of DirectX they typically think of 3D applications and games, and while DirectX is a great platform for building 3D applications and games, it is capable of doing much more. In Windows 7, Microsoft added Direct2D to the DirectX family of technologies, and as you would expect Direct2D finally adds the ability to execute drawing commands against a 2D surface. In many ways Direct2D is being setup to take the place of GDI, and because it is built on top of Direct3D and DXGI it is also hardware-accelerated and runs on the GPU. Microsoft has also added DirectWrite, which now provides developers a way to layout and render high quality text while making full use of the GPU. Before DirectWrite developers either had to use GDI or rig their own system to render text; now with DirectWrite, developers have access to a rich API that supports layout, international text, and sub-pixel anti-aliasing that integrated easily into the rest of their application whether they are using GDI or DirectX.

Even though DirectX is an amazing API and has only grown in capabilities over the years, there are still numerous applications that don’t use DirectX. Most of these applications use GDI (a much older technology ) to paint to the screen, and yet other applications use other rendering libraries like WPF. It used to be the case that once a developer chooses a core rendering technology to build an app, it was impractical to leverage any other rendering technology. Fortunately Microsoft has built a new technology into Windows 8 called Direct Composition that does away with this limitation. At its core, Direct Composition is simply a bitmap compositing engine. By using Direct Composition it is now possible to use WPF to build the bulk of your application, and to sprinkle in some DirectX code to give your app that extra sparkle that would otherwise be too difficult or which would run too slowly if attempted using WPF. Additionally, because Direct Composition is baked deep into Windows 8 it is possible to compose applications and effects that are generated from code running in separate processes, which opens up a whole new front in software engineering.

Direct Manipulation

While certainly not the last hidden technology in Windows 8, the last I’ll cover is Direct Manipulation. Most developers are extremely comfortable in the world of mouse driven user interfaces—we all understand the concepts of click, hover, right click, move, etc. But the world of touch driven user interfaces is largely uncharted by many developers and most will find that it is far more complicated and difficult than the mouse driven world. In a touch user interface, there may be one, two, five, or no touches on the screen. The user may be pinching to zoom or he or she may just want to move an object but by using two fingers instead of one. Of course it would be possible for developers to build state machines that were able to process and interpret these user gestures and to continue to build their touch based application, but it turns out that building these state machines is not an easy task. Fortunately Microsoft has done the hard work and has included the Direct Manipulation technology in Windows 8. Direct Manipulation is essentially just a touch input state machine that frees developers from the details of interpreting user input. Instead, by using Direct Manipulation, developers can be notified when a user is engaging in a common gesture like pinching, sliding, or rotating. In fact much of the Direct Manipulation API is the same as what you would find in WPF or in WinRT Xaml, and it is my guess that the WinRT Xaml stack is actually built on top of Direct Manipulation.

There are of course many other  new technologies that Microsoft has created for Windows 8 and still more existing technologies that Microsoft has improved. The Windows Store app model opens up entirely new markets for Microsoft developers, but let’s not forget that Windows 8 is still a great platform for building desktop apps.


Audi City: Inventing the Dealership of the Future

Jul 19, 2012 by in 5D, Experience Design, Kinect, Microsoft Kinect, Mobile, Multi-touch, News, Portfolio, Retail, Technology, Touchscreen

We’re excited by the launch of a revolutionary showroom experience for a premiere automotive brand. After a year of collaboration between Audi and a wide range of partners, Audi City has launched near Piccadilly Circus in London, ahead of the 2012 Olympics.

Piccadilly Circus in London

Audi City London is a groundbreaking dealership experience delivered by one of the most technologically advanced retail environments ever created. The digital environment features multi-touch displays for configuring your Audi vehicle from millions of possible combinations. Your personalized car is visualized in photorealistic 3D using real-time render technology, making the Audi City vehicle configurator the most advanced in the world. After personalizing your Audi, you can toss your vehicle onto one of the floor-to-ceiling digital “powerwalls” to visualize your car configuration in life-size scale. From here, you can use gestures to interact with your personalized vehicle, exploring every angle and detail in high resolution using Kinect technology.

credit: Audi

A purely digital showroom can’t deliver on the tactile experience of buying a car. Therefore, a store associate can save your configuration on a RFID-enabled USB stick and guide you into a personal consultation area that features a variety of tactile objects. These objects help the customers get hands-on with the materials of the vehicle including car exterior color and finish options and interior upholstery options. Each of these tangible objects are digitally-tagged through RFID technology. You can bring bring any of these physical objects over to the configurator experience and the corresponding exterior paint finishes and interior options will automatically update your vehicle configuration.

credit: Audi

When purchasing a car, the customer journey occurs across multiple channels. In order to integrate and simplify the car buying process, we’ve allowed customers to retrieve their online car configurations in the showroom environment. In addition, any car configuration made in the showroom is synchronized to your personal USB stick. Simply pop in the USB stick at home and the web-based configurator is automatically launched with the exact car configuration you created in the showroom. This allows Audi to deliver a “start anywhere, end anywhere” buying cycle for the customer, which has proven elusive for retailers.

Not only is Audi City a premier showroom environment, the dealership concept represents a fundamental shift in retail strategy for the brand. This new small-footprint retail format brings Audi closer to their customers, not only geographically but also emotionally. The smaller-footprint concept will launch in metropolitan environments and reach a younger urban and digitally-enabled demographic. After hours, the environment will serve as a cultural center in the larger community by playing host to readings, round-table discussions and art exhibitions.

credit: Audi

“Audi City combines the best of two worlds – digital product presentation and personal contact with the dealer” says Peter Schwarzenbauer, Member of the Board of Management at Audi. “People are placing greater emphasis than ever before on a direct and personal bond of trust with their vehicle brand – especially in respect of the increasing variety of products and available information. Thus, with Audi City we are creating a one-stop-shop for experiencing our brand. It is right in the midst of our customers’ lives, yet seamlessly connected to the online range offered by the four rings.”

Audi announced at the London launch that 20 showrooms in other major international cities will follow by 2015.

Tags:

DeltaZone @ Madison Square Garden

Apr 15, 2012 by in Microsoft Kinect, Mobile, Multi-touch, Portfolio, Touchscreen

Anticipation has been building for years.

The expectation has always been that our lives will be transformed by new technologies. Everything from travel to sports and entertainment would be made new again…redefined.

And now, thanks to Delta and Madison Square Garden in partnership with Razorfish, that time has finally arrived.

Delta Air Lines’ Touch the Future of Travel and a newly refreshed yet still iconic Madison Square Garden is here.

In addition to the 11,000 square foot lounge which features select menus, multi-screen event coverage, and a clear view of professional athletes entering the arena through a glass hallway, we’ve created a unique experience for VIPs.

A personalized, curated way for travelers to discover new destinations, collecting content from around the globe and enjoying fantastic vistas that transport them into the magic of destination travel and discovery.

Delta’s Touch the Future of Travel is about unique inspiration, easier access to what you want, when you want it, and sharing travel ideas with friends…and Razorfish with Delta is making it all happen.


The Microsoft BUILD conference, Windows 8, and the new UX Challenge

Sep 20, 2011 by in Experience Design, Multi-touch, Technology

Before the BUILD conference, the one thing we all knew was that Microsoft needed a multitouch tablet strategy to compete with Google and Apple and in order to maintain the future viability of the Windows operating system. What we were not sure of was how Microsoft would achieve this goal while preserving backwards compatibility for all of our previous Windows applications in the office as well as in the home. The challenge at first blush seemed insurmountable: provide something completely new to the Windows world while preserving everything that went before.

At BUILD, Microsoft revealed that they have actually accomplished this goal by providing what is basically two side-by-side operating systems. They have also signaled that the primary challenge for application creators going forward will not be technical but rather design-focused. Microsoft, which in the past has tended to side-line user experience, now puts design front and center with “Windows 8.”

One of the two Win8 interfaces is a slightly souped up version of Windows 7 that looks familiar and runs just about anything I could think to try installing on it: Zune, Dropbox, iTunes, Kindle for PC, and even the software for the Kinect SDK. “Windows 8” ran each of them without complaint. The desktop shell works best with a mouse and keyboard, though it also supports and has been redesigned to support multitouch also.

The other is a Metro inspired immersive experience that works best using touch. Instead of an explorer based file system with icons, the Metro shell is designed around interactive tiles, familiar from Windows Phone 7, that launch discrete apps. The Metro shell revolves around a new Windows Store (the equivalent of the iPad’s App Store and WP7’s Marketplace) that allows consumers to download games and apps.

Dual Mode with Dock

Dual Mode with Dock

One could easily think of this as two solutions in one: a consumer platform designed for the tablet and a desktop platform designed for the PC. What is unusual about these side-by-side solutions is that, with the flick of a finger, the tablet user can bring up the desktop UI and the desktop user can bring up the Metro UI. The two operating systems are not something one configures through the control panel the way one might configure a background theme. Instead, both UIs are effectively always alive and always immediately accessible.

Microsoft generously provided each attendee with a new Samsung tablet installed with a pre-Beta build of “Windows 8” and accessorized with a wireless keyboard, a stylus and a dock. The dock is by far the most intriguing – and least discussed – piece of hardware provided as it offers an indication of how Microsoft envisions “Windows 8” being used in the future. A tablet may be inserted into the docking station with a monitor and mouse in an office setting, at which point the desktop UI can be brought up and the user has an experience for the most part indistinguishable from what he is currently used to. The tablet can then be undocked and switched to the Metro style with all the previously running applications still running.

The notion of two different operating systems goes all the way down to the development stack with two different platforms: one based on the traditional tools of Microsoft development such as Silverlight, WPF and WinForms targeting the desktop UI and another, completely new, development stack built around something called the Windows Runtime (WinRT) and programmable using C# and VB with a XAML-style UI language or C++ or Html with javascript compiling to a combination of native and dynamic code.

Initially the expectation is that the .NET tools of the past ten years will be used to write business, productivity and data-entry intensive applications while the new tools will be used for games, social apps and everything else one might expect to find on a smartphone or an iPad.

In a mixed-OS experience like the one described above using a dock, a more likely setup would be a full .NET style business app for the desktop with a lighter-weight Metro style version of the same app on the Metro UI. This allows users to quickly switch back and forth between a tablet and a desktop scenario using the same device. The test of this will likely come when Microsoft reveals its plans for Microsoft Office. We would expect Microsoft to provide both a classic and a Metro version of their Office suite. How well they implement this will in turn provide a roadmap for how other vendors will cater to the enterprise in their software solutions. In other words, will “Windows 8” for the enterprise have enterprise applications for the desktop only or for both the desktop and for the Metro UI.

There is a third possibility also. It may be possible to build full enterprise applications targeting Metro only. The WinRT platform combined with Microsoft’s Azure offering supports this.

The challenge in creating sophisticated apps for Metro is not primarily a technical challenge. It is primarily a User Experience challenge. Can we create multitouch enabled data grids? Can we come up with new navigation patterns to replace the standard enterprise application with hundreds of unique windows? Can we find ways to create great experiences that combine both multitouch and keyboard interaction?

Jensen Harris

Jensen Harris

While Microsoft has been tagged with a reputation for not understanding UX over the past decade, this has seemed to change. At BUILD, the speakers were all aware of the importance of UX while speakers like Jensen Harris demonstrated that Microsoft not only knew that UX problems were important but that they also had the chops to solve them. In this context, BUILD has been a watershed event. If Microsoft has tended to admire and promote smart programming in the past, after BUILD it will become more important to be savvy about design. The days when design could be dismissed as merely prettying up an application are over. After this week, design on Windows is front and center. This is good news for agencies like Razorfish which are strong in both design and technology. It will be a challenge for software consultancies that have only been paying lip service to UX until now as they attempt to establish themselves as Metro experts.

On the technology front, as mentioned above, Microsoft is supporting three platforms: one targeted at C++ developers, one at XAML developers (Silverlight and WPF), and one targeted at web developers. The tack of using web technologies for building native Metro apps for the “Windows 8” tablet currently makes the most sense. A common path for developing apps for multiple platforms like the iPad and iPhone, Android and Windows is to first create a web application that can run on all these platforms, then after looking at web analytics data and determining which platforms use the web app most, building native apps for each of the top platforms. In the case of the Metro UI, it will be easiest to port code from web apps to the native web development tools on Windows 8 rather than attempt to build a brand new project in either C++ or XAML. Again, this type of development is already familiar to digital agencies but likely to be a challenge for other organizations.

BUILD also quietly announced improvements and fixes to WPF in the new .NET 4.5 framework being released with “Windows 8.” This is exciting for the Razorfish Emerging Experiences group as WPF is our main development platform for Surface applications and multitouch experiences.

Search Results for Silverlight at BUILD

Search Results for Silverlight at BUILD

The story for Silverlight is a bit more ambiguous. Currently “Windows 8” offers two different versions of IE 10 – one for the desktop UI and one for the Metro UI. The Metro UI version does not support plugins. Consequently neither Flash nor Silverlight applications will run in Metro IE. This is a difficult position since it entails Silverlight does not work as a multi-platform solution even on “Windows 8,” i.e. it supports only one of the two Win8 platforms. Silverlight-out-of browser is still viable on the desktop UI. It must compete, however, with both WPF – which is more feature rich – and WinForms – which has a significantly larger developer base. It has been suggested that the main benefit of Silverlight as a desktop technology solution will be that, since it, like XAML for WinRT, is only a subset of WPF, this will make things easier when it comes time to port an application over to the Metro UI. In porting from either Silverlight or WPF, however, some rewriting will have to occur as XAML for WinRT actually introduces interesting new XAML features – such as markup for localization – currently missing from both Silverlight and WPF.

There are still several open questions remaining with regard to Windows 8. Two have already been mentioned:

1. Is the Metro UI for the enterprise or for consumers only?
2. What are Microsoft’s plans for Microsoft Office?

Waiting in line for Win8 Tablets

Waiting in line for Win8 Tablets

A third open question is What are Microsoft’s plans for Windows Phone? While the Samsung tablets given to attendees at BUILD are Intel-based, Microsoft’s ultimate goal is to provide an ARM-based version of “Windows 8.” The great advantage of an ARM architecture is that it allows “Windows 8” to be placed on a variety of hardware platforms including smart phones. Currently, however, Microsoft has dropped no hint that it plans to release “Windows 8” phones, however, and the concern would be that such an announcement would damage sales of Windows Phone 7.5, which will be released sometime in 2011. On the other hand, it doesn’t seem to make sense to have completely different operating systems for the Microsoft tablet and Microsoft’s phone. Apple has benefited greatly by having one OS for both form factors and Microsoft strategists, no doubt, are well aware of this.


Delta + WIRED Store = Touch the Future of Travel

Mar 08, 2011 by in Experience Design, Multi-touch, Portfolio, Touchscreen

When it comes to travel, people care more about where they’re going than how they’re getting there. Delta Airlines understands this and asked Razorfish’s Emerging Experiences team to create an engaging experience for the WIRED Holiday Store in NYC. We wanted to tap into users’ imagination and sense of playfulness so that they walk away from the experience thinking about what kind of destinations they want to go to next and, of course, Delta.

In four (4) weeks we concepted, designed, developed and launched The Untravel Idea – a new, personal way for leisure travelers to encounter destinations. We wanted users to touch the future of travel.

The experience gives people an open-ended, creative experience that puts the user in control. First users choose the type mood they are looking for on their next getaway. From there users can select from a wide variety of relevant words that match their mood, and when put together, show them a range of destination possibilities. The result is a beautiful montage of photographic imagery that will transport the user’s imagination.

To extend the experience beyond the store, users are prompted to use their mobile device to snap a pic of a QR tag associated with each destination that allows them to explore additional destination info, video and travel packages.

The event was a complete success. People couldn’t wait to see where Delta would take them next. The Takeaway – Delta is not just an airline; they’re giving me new ways to discover travel destinations.

This is the future of travel … and it’s just the beginning.


OakleyView for iPhone and iPad

Oct 12, 2010 by in Augmented Reality, Experience Design, Mobile, Multi-touch, Touchscreen

A wide range of Oakley products are designed for sports fans and outdoor living people who are dependent on their equipment when practising their passion and living their dreams. Choosing the right sunglass lens makes a significant difference when sports and outdoor activities are taken seriously. To guide the consumer through this decision process, we have implemented an iPhone and iPad App which simulates realistic scenarios by using engaging 3d-panorama landscapes wrapped in an intuitive touch- and accelerometer-based interface.

The overall experience features more than 18 lens tints in spectacular environments and various weather conditions. Once the perfect lens is selected, a detailed product information is just one touch away.

Available on the AppStore for iPad and iPhone.