Mar 25, 2014 by in Kinect, Microsoft Kinect, News, Projection, Technology, Travel

As passengers of San Francisco International Airport (SFO) walk into the newly renovated Terminal 3E they are greeted by a 16-foot wide glowing data visualization that hovers above six multi-touch screens, the Flight Deck. The Flight Deck highlights SFO’s global reach, services, amenities, and museum exhibits. We want to share the backstory about the development of the interactive data visualization component of the Flight Deck.


The Projection

Our goal was to create a visual beacon in an airport that stood out from other digital displays and had the ability to be seen on both sides. We envisioned a uniquely shaped floating image without bezels that was constantly moving to capture the attention of passengers whether they are arriving or departing. To meet this challenge, we experimented with several concepts but decided on attempting a 24/7 rear-projection in a bright airport terminal. Rear-projections in backlit environments are difficult since projectors don’t project black. Initial tests (images below) showed that a projection in the space would be possible given the proper projector, proper dark grey rear-projection film, and window treatment. We worked with the architects to specify a frit for the windows immediately behind the projection. We used a 20,000-lumen projector, the brightest rear-projection film we could find, and optimized the placement of the glass and projector for the visitor approach. Visitors approach the Flight Deck parallel to the projector, so that they experience the brightest portion of the projection when walking toward the Flight Deck.


The Experience

We wanted to create an ever-changing experience where a Kinect camera would use people’s presence to alter the background, and real-time data from flight paths would highlight the global reach of SFO from moment to moment. Key rewards were also synchronized with the touch screens so that visitors are able to take over the entire projection if they collect hidden rewards within the touch screen experience. The software was developed using Cinder, and used OSC for network communication. An early decision was to heavily thread our architecture so that we could maintain a minimum of 60 fps for low interaction latency and smooth animation. Unfortunately several components in Cinder were not properly written for heavy multithreading such as the Console or the Kinect class, so we added locks and properly threaded components as needed. Early interaction prototypes tested the limits of our frame rate and latency to determine areas for optimization.


Kinect Interaction

As curious visitors approach the Flight Deck a Kinect camera captures their form and creates triangles and shapes that play with their form using an attraction based physics system, as seen in the image above. As visitors move left to right the triangles leave trails of triangles on the screen that orbit the space, creating constantly changing visuals. Visitors can also use their arms to attract and control the shapes around them. A rear mounted Kinect camera captures approaching visitors and uses computer vision algorithms to convert the depth image to blobs. The computer vision allows for tracking of any number of people, vs two people for skeleton tracking. In addition, we also use depth data from a depth range that is normally unreliable for skeleton tracking. We modified the Kinect API for Cinder to use an extended depth range to capture people who are up to 16 feet away and to see people within a 15 foot wide area.

Data Visualization

Live flight data is visualized on screen as well as inbound and outbound activity from SFO in the last 24 hours. Positional data from flights is received in a polar coordinate system and is converted to a Cartesian coordinate system on the GPU. Our data visualization uses the positional data from flights so that flight paths can be visualized with bends and waviness vs perfectly smoothed flight paths. The challenge with positional data was that live flight data is not always available in all positions of a flight so some interpolation had to be mixed with the actual data. SLERP was utilized for interpolating positions and real data was combined with interpolated data using a gated smoothing algorithm. The approach allowed for us to properly visualize flights, for example flights to Asia, where positions are occasionally lost but picked up at various locations.

Looking Forward

The interactive data visualization represents only one aspect of the Flight Deck. The Flight Deck also consists of a multi-touch experience and a mobile experience and was a collaboration of creative and technical talent. Since the launch, we’ve heard a lot of feedback where people are comparing the Flight Deck to their favorite sci-fi shows and movies. We hope that we can continue to create forward thinking experiences for the public.


Audi City Berlin Launched

Feb 05, 2014 by in 5D, Experience Design, Microsoft Kinect, Mobile, Multi-touch, News, Portfolio, Retail, Technology, Touchscreen

First London, then Beijing and now Berlin: Audi City has transformed the dealership experience in ways never before seen. It’s delivered by one of the most technologically advanced retail environments ever created and features a variety of multi-touch displays for configuring your Audi from millions of possible combinations. Personalize your Audi then toss it onto one of the floor-to-ceiling digital “powerwalls” to visualize and explore your configuration at a true 1:1 scale. Audi City is a true dealership of the future and an effort we are proud to be part of.






RZRShop Highlights from NRF

Jan 27, 2014 by in 5D, Augmented Reality, Experience Design, Kinect, Mobile, Multi-touch, News, Portfolio, Projection, Retail, Technology, Touchscreen

From a surfboard designer that comes to life right before your eyes, to a one-of-a-kind, dual screen Kinect beach soccer game, RZRShop was a huge hit at this year’s National Retail Federation Annual Convention and Expo. Check out some of the highlights.

Tags: ,

Experience The Future of Travel with San Francisco International Airport’s Flight Deck

Jan 24, 2014 by in Experience Design, Kinect, Multi-touch, News, Portfolio, Projection, Retail, Technology, Touchscreen, Travel

The assignment was, well, not so simple. Design and build a fully immersive, best in class, innovative future travel experience for passengers bustling through one of the busiest and fastest growing airports in the country — San Francisco International. The Flight Deck will serve as the gateway experience to the newly renovated Terminal 3, Boarding Area E (or Terminal 3E as it’s known as).

When the Emerging Experiences team was contracted by Hensel-Phelps, one of the largest general contractors and construction managers in the United States, to create this experience we could not have been more excited. For a multi-disciplinary team like us, built of experience designers, creatives, techies, and passionate people that love to collaborate and deliver unexpected, delightful experiences, this project screamed open canvas. Our team is built for these challenging, yet experimental and fun engagements. And the fruits of the tight collaboration between tech and creative shine bright. Literally. Through a 20,000 lumen projector onto oversized panels of glass displaying real-time flight data.


So just what is Flight Deck? The SFO Flight Deck is composed of three distinct but connected digital experiences — an interactive and real time large scale projection (you can’t miss it), six multi-touch kiosks that are rich with beautiful content, and a mobile takeaway component for those on the go. The entire experience resides in Terminal 3, but the projected visualization serves as a beacon calling all SFO guests to contribute to the global SFO travel story. Content in all three experiences encompasses the entire airport and extends its reach into the city of San Francisco and global destinations.

In order to serve the wide gamut of travel guests, the team operated under the following experience pillars at all times. If an idea did not fit under two or more of these pillars, it hit the floor.

1. Quick and meaningful interactions should maintain quality and depth.
2. Playful collaborations should be maintained throughout the experience.
3. Your personal narrative is part of the SFO story.

Why? As mentioned earlier, the assignment was not so simple. Today’s travelers are in a rush. How and why do we make busy passengers take the time to stop and engage? Inspired by the golden age of travel, when it was considered a magical thing and an experience to do so, we wanted to bring back the joy of the travel experience.

SFO is the just the airport to foster that joy. Featuring an on-site museum with collections that could rival most of the best metropolitan museums we’ve ever visited, we were allowed to dig in to the archives of hundreds of photos of the museum’s aviation collection. And we bring a lot of these golden moments to guests in Flight Deck, hoping they take the time to smile and remember the transformative and magical experiences that travel can bring.

Grand Opening

Terminal 3E is scheduled to open for passengers on January 28, but folks were able to get a sneak peek at the T3E Community Day on Saturday, January 25th. The Flight Deck was a huge hit for folks of all ages, some of which said they plan on coming a bit early on their next flight out of SFO just so they could explore it more. Check out our updates from the event on Twitter and Facebook, and be sure to vote for your favorite feature in Terminal 3 here (guess what we voted for?).





The Biggest Thing in Retail is Actually Pretty Small: Welcome to #RZRShop

Jan 13, 2014 by in 5D, Experience Design, Kinect, Microsoft Surface, Mobile, Multi-touch, Near Field, News, Portfolio, Retail, Technology, Touchscreen


For this year’s National Retail Federation Annual Convention and Expo, we’ve built a highly interactive, surf-and-sand-inspired shopping experience. Built inside a uniquely portable container, RZRShop creates big moments inside a smaller, more nimble retail footprint. Digital meets physical with a Perceptive Pixel-powered surfboard designer that comes to life right before your eyes. And battle lines are drawn with a one-of-a-kind, dual screen Kinect beach soccer game and smart vending experience. Powered by the Razorfish 5D Retail Platform, RZRShop seamlessly connects a variety of devices to attract customers, drive product engagement and arm store associates with more contextualized tools. The end result is a fun and personal experience—the way shopping should be.

Visit for more information, and visit us in Microsoft booth #2703 to experience RZRShop for yourself.

Big Ideas Come in Small Packages

Jan 08, 2014 by in 5D, Experience Design, Microsoft Kinect, News, Retail, Technology, Touchscreen

NRF Container

Our container arrived at New York’s Javits Center this morning where our team is prepping for next week’s National Retail Federation annual convention and expo. What’s in the container? Stay tuned.

Tags: , ,

Creativity on the Vine

Jun 07, 2013 by in News

At our recent Razorfish Client Summit in Las Vegas, Emerging Experiences asked our fellow fish to submit 6 seconds of creativity for our clients and partners to vote on within “The Lab”. Utilizing the Vine application, our employees around the world created 6 second shorts that expressed their creative side. We then fed specifically tagged Vine videos (#razorfishcreativity) into a unique multitouch experience for our clients and partners to vote on. Vine doesn’t have an official API, so we had to get a bit “creative” on the technology side to pull this off.

What were the results? Well, lets just say the entries were varied… From stop-motion Rubik’s Cubes, to creative mashups of Bob Lord over lunch, each entry showcased individuality and our collective creative spirit. Overall our clients loved the experience and their ability to vote on their favorites. So which fish won?

(insert drumroll here)

1st Place: Moritz Bosselmann
View Entry

2nd Place: Matisse Miller
View Entry

3rd Place: Tomoko Fushimi-Haack
View Entry

Who said it doesn’t pay to be creative? Thank you to all the fish that entered and to our clients and partners that participated in the experience.


Adweek Feature Story: Emerging Experiences goes coast-to-coast with a new Lab in San Francisco.

Jun 04, 2013 by in Lab, News


Razorfish Emerging Experiences has opened a new lab in our San Francisco Razorfish office across from Pier 39 and the renowned Fisherman’s Wharf. Equal parts workspace and client demonstration area, the lab is invaluable for our team to design, build and test some of the most engaging and transformational experiences in the marketplace. Leveraging the success of our Atlanta Lab and its evolution over the past 5 years, the San Francisco Lab is our newest digital sandbox.

In two related articles referencing the work in our new Lab, Christopher Heine writes in Adweek that “employing the latest technology at point of sale is nothing new—for years businesses from car rental companies to Nordstrom department stores have unhooked from the wires. But the trend has gone from merely ringing up sales via mobile devices to a deeply immersive in-store experience—fully digitized but crucially featuring that face-to-face element…”

The Lab showcases 360 degree video content across multiple displays and projection surfaces, features emerging technologies such as transparent displays and multi-touch and gesture-based sensors powered by our proprietary Razorfish 5D Platform. Watch an Audi be configured in precise detail through the application that empowers Audi City or sit back and watch as data is visualized through one of our latest projects.

Physical meets digital and the customer’s journey will never be the same. Innovating Tomorrow, Today. For appointments please contact Wade Forst (, our Director of Emerging Experiences in San Francisco.

From The Razorfish Client Summit: An Interview with Jonathan Hull and Jeremy Lockhorn

May 14, 2013 by in News


Emerging Experiences Lab at Converge: Razorfish Client Summit 2013

May 01, 2013 by in Advertising, Experience Design, Lab, News, Technology

Take a look behind the scenes of the Lab at the ARIA in Las Vegas. A true manifestation of what we do in the Emerging Experiences group, the Lab set-up brings to life the ideas behind this year’s Client Summit theme, Convergence. To learn more about the ideas that drive our passions, read more about what Razorfish’s Global CEO, Bob Lord and Global CTO, Ray Velez have to say in their new book.