Author Archive

Hidden Tech Features in Windows 8

Nov 12, 2012 by in Microsoft Surface, Multi-touch, Technology, Touchscreen

 

With Microsoft’s Build 2012 over and Surface now available for purchase, it seems like the only thing Microsoft developers can talk about is building Windows 8 store apps. Creating Windows apps that can be monetized using the same wildly successful revenue sharing model that Apple pioneered is compelling. However, in the rush to cash in on this new frontier it’s easy for developers to lose sight of the fact that Windows 8 is far more than the addition of a new shell and a new application model.

As a product, Windows has been continuously developed for over fifteen years (which is far longer than Apple has been using the revenue sharing monetization model), but this time Microsoft has built some great technologies that have received far less attention.

The first that comes to mind is DirectX. When most developers think of DirectX they typically think of 3D applications and games, and while DirectX is a great platform for building 3D applications and games, it is capable of doing much more. In Windows 7, Microsoft added Direct2D to the DirectX family of technologies, and as you would expect Direct2D finally adds the ability to execute drawing commands against a 2D surface. In many ways Direct2D is being setup to take the place of GDI, and because it is built on top of Direct3D and DXGI it is also hardware-accelerated and runs on the GPU. Microsoft has also added DirectWrite, which now provides developers a way to layout and render high quality text while making full use of the GPU. Before DirectWrite developers either had to use GDI or rig their own system to render text; now with DirectWrite, developers have access to a rich API that supports layout, international text, and sub-pixel anti-aliasing that integrated easily into the rest of their application whether they are using GDI or DirectX.

Even though DirectX is an amazing API and has only grown in capabilities over the years, there are still numerous applications that don’t use DirectX. Most of these applications use GDI (a much older technology ) to paint to the screen, and yet other applications use other rendering libraries like WPF. It used to be the case that once a developer chooses a core rendering technology to build an app, it was impractical to leverage any other rendering technology. Fortunately Microsoft has built a new technology into Windows 8 called Direct Composition that does away with this limitation. At its core, Direct Composition is simply a bitmap compositing engine. By using Direct Composition it is now possible to use WPF to build the bulk of your application, and to sprinkle in some DirectX code to give your app that extra sparkle that would otherwise be too difficult or which would run too slowly if attempted using WPF. Additionally, because Direct Composition is baked deep into Windows 8 it is possible to compose applications and effects that are generated from code running in separate processes, which opens up a whole new front in software engineering.

Direct Manipulation

While certainly not the last hidden technology in Windows 8, the last I’ll cover is Direct Manipulation. Most developers are extremely comfortable in the world of mouse driven user interfaces—we all understand the concepts of click, hover, right click, move, etc. But the world of touch driven user interfaces is largely uncharted by many developers and most will find that it is far more complicated and difficult than the mouse driven world. In a touch user interface, there may be one, two, five, or no touches on the screen. The user may be pinching to zoom or he or she may just want to move an object but by using two fingers instead of one. Of course it would be possible for developers to build state machines that were able to process and interpret these user gestures and to continue to build their touch based application, but it turns out that building these state machines is not an easy task. Fortunately Microsoft has done the hard work and has included the Direct Manipulation technology in Windows 8. Direct Manipulation is essentially just a touch input state machine that frees developers from the details of interpreting user input. Instead, by using Direct Manipulation, developers can be notified when a user is engaging in a common gesture like pinching, sliding, or rotating. In fact much of the Direct Manipulation API is the same as what you would find in WPF or in WinRT Xaml, and it is my guess that the WinRT Xaml stack is actually built on top of Direct Manipulation.

There are of course many other  new technologies that Microsoft has created for Windows 8 and still more existing technologies that Microsoft has improved. The Windows Store app model opens up entirely new markets for Microsoft developers, but let’s not forget that Windows 8 is still a great platform for building desktop apps.


Leveraging the Windows 8 Start Screen

Nov 11, 2011 by in Technology


Ever since Microsoft started leaking details about its upcoming version of their flagship product, Windows 8, there has been firestorm of controversy among Microsoft’s faithful.  Many Silverlight application developers and publishers feel like they have been willfully misled into investing in a technology that Microsoft is now apparently abandoning. Many IT Pros dislike and fear the retraining efforts they will have to make with new Start Screen and other Windows shell changes. Finally, many ASP.NET web developers don’t see how Windows 8 relates to them despite the fact that Microsoft is adding “WinJS”, a runtime that allows Web developers to leverage their existing skills to build native applications.  On the ground, it may seem like things are going badly for Windows 8, but with a little developer ingenuity and a lot more communication and documentation from Microsoft, Windows 8 could be the product that saves Microsoft from being a victim of their own success.

Take the Start Screen for example—In order to finally enable OEMs to build devices that can truly be considered a “Tablet PC”, Microsoft has to provide a way for users to launch applications.  One might be tempted to think that the Start Menu in Windows 7 could be adapted to serve this purpose, but fingers are just not good at tapping on small icons or icons that are densely packed.  Making the Start Menu a full screen experience is really the only way to get enough space to create a truly usable touch-optimized experience.  We in the Emerging Experience group have known this for years as practically every single touch based application that we have built has been a full screen app.  On top of this, Microsoft’s Start Screen’s animations are extremely fluid and natural, and so to us it seems like a natural platform from which to launch our showcase applications.

To give a little background about ourselves and our applications, we are a technology agile group, which means that we use the technology that creates the best experience for our customers. Many of our apps are built using WPF but we also have apps that are built using Flash.  Obviously attempting to port our applications would not be a good strategy for the Flash apps, but even after a brief investigation, I quickly decided that attempting to port all of our WPF applications was a non-starter.  The Metro APIs are far too different and who knows if, after porting the WPF applications, I would even end up with an app that worked?  The solution, it was decided, was to leave the existing showcase applications as they were but to simply create live tiles for them so that they could be invoked.

The problem with this solution is that it is not possible to really take advantage of the Live Tile infrastructure from a Win32 app.  In a Metro-style (WinRT) application you can supply different resolution images for the tiles by altering the AppX manifest, but Win32 Applications don’t have AppX manifests.  It might seem trivial to simply create a WinRT application that upon launch invokes one of our showcase applications, and to use the WinRT app’s AppX manifest to customize the Live Tile, but unfortunately the relationship between WinRT and Win32 is significantly more complex than that.  First of all, WinRT applications can call some Win32 APIs, but it explicitly cannot create new processes—this is part Microsoft security model for WinRT apps.  On top of that, even though WinRT apps can call many Win32 APIs, many of those calls either fail outright or fail to have the desired result.  Clearly this is an area where Microsoft can do a much better job in providing documentation. 

To work around these limitations, I decided to create a WPF application that lives in the System Tray as a notification icon.  The entire purpose of this WPF application is to listen for network calls and then launch and activate the requested application.  At this point our WinRT “launcher” application was simply responsible for initiating the network call and then close itself down.

While this worked beautifully in the debugger, I was surprised to find that it did not work once the applications were freed from the debugger.  Sure, the Launcher application still made a network call to the WPF application and the WPF application still launched the showcase application, but the showcase application was never displayed.  The problem, it turns out, is that the Win32 function “SetForegroundWindow” on which my WPF application was indirectly relying behaves differently if the calling application is being debugged.  Clearly the Windows shell makes use of a facility to show the desktop when the user clicks on the Desktop tile in the Start Screen, but when I asked Microsoft about this and SetForegroundWindow,  I was essentially told that this was by design and that only the end user should control which window has focus.  I understand the wisdom of this decision, but this answer didn’t get me any closer to being able to launch our showcase applications from nice looking Live Tiles.

While I wouldn’t propose that developers do this in production applications, Windows 8 isn’t a production OS itself—and I still hold out hope that will make this whole endeavor moot by they release Windows 8.  With the disclaimer in effect, the way that I solved this problem was to create a third Windows Forms application whose sole responsibility is to run CDB, the command-line debugger, and automate it to launch and attach to the WPF application.  Because the WPF application has a debugger attached, it is now able use the SetForegroundWindow API and the entire system works as expected.  In fact, by not creating a Window in the Windows Forms application and launching CDB without a console window the entire hack is invisible to the user and everything transparently works as expected.

While there probably are other ways to achieve the same effect, this really should highlight noteworthy holes in Microsoft’s messaging and documentation.  First of all, I can understand that Metro-style apps won’t be backwards compatible on Windows 7, but this is no reason not to provide a facility to make Win32 apps forward compatible the Windows 8 Start Screen’s Live Tiles.  Also, while I understand that, generally speaking, it is not good design to allow application developers to control which application is active, at the same time there are always exceptions and this functionality should be enabled/disabled in registry settings or group policy and not be decided in a one-size-fits-all fashion.  But with this said, Windows 8 is actually proving to be a natural evolution of Windows 7 and a good platform on which to build touch-based experiences.  With all the talk Silverlight, Javascript, and strategy, it’s easy to forget that applications are only valued for creating good user experiences and that Windows 8 delivers a great user experience for a pre-Beta operating system.