SharpGIS

#GIS from a .NET developer's perspective

Creating a simple push service for home automation alerts

In my intro post for my home automation project, I described a part of the app that sends temperature, humidity and power consumption measurements and pushes them to my phone. In this blogpost, we’ll build a simple console-webservice that allows a phone to register itself with the service, and the service can push a message to any registered device using the Windows Notification Service. Even if you’re not building a home automation server, but just need to figure out how to push a message to your phone or tablet, this blogpost is still for you (but you can ignore some of the console webservice bits).

To send a push message via WNS to an app, you need the “phone number of the app”. This is a combination of the app and the device ID. If you know this, and you’re the owner for the app, you are able to push messages to the app on the device. It’s only the app on the device that knows this phone number. If the app wants someone to push a message to the device, it will need go share it with that someone. But for this to work, you will first have to register the app on the Microsoft developer portal and associate your project with the app in order to be able to create a valid “phone number”.

Here’s the website after registering my app “Push_Test_App”. You simply create a new app, and only need to complete step 1 to start using WNS.

image

Next you will need to associate your app with the app you created in the store from the following menu item:

image

Simply follow the step-by-step guide. Note that the option is only available for the active startup-up project. Repeat this for both the store and phone app if you’re creating a universal project (make sure to change the startup project to get the menu item to relate to the correct project).

This is all we need to do to get the app’s “phone number”  the “channel uri”. We can get that using the following line of code:

var channel = await PushNotificationChannelManager.CreatePushNotificationChannelForApplicationAsync();
var uri = channel.Uri;

Notice that the channel.Uri property is just a URL. This is the URL to the WNS service where we can post stuff to. The WNS service will then forward the message you send to your app on the specific device.

Next step is to create a push service. We’ll create two operations: A webservice that an app can send the channel Uri to, and later an operation to push messages to all registered clients.

We’ll first build the simple console app webserver. Some of this is in part based http://codehosting.net/blog/BlogEngine/post/Simple-C-Web-Server.aspx where you can get more details.

The main part to notice is that we’ll start a server on port 8080, and we’ll wait for a call to /registerPush with a POST body containing a json object with channel uri and device id:

using System;
using System.Net;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
using System.Threading;

namespace PushService
{
    internal class HttpWebService
    {
        private HttpListener m_server;
        
        public void Start()
        {
            ThreadPool.QueueUserWorkItem((o) => RunServer());
        }

        private void RunServer()
        {
            Int32 port = 8080;
            m_server = new HttpListener();
            m_server.Prefixes.Add(string.Format("http://*:{0}/", port));
            m_server.Start();
            while (m_server.IsListening)
            {
                HttpListenerContext ctx = m_server.GetContext();
                ThreadPool.QueueUserWorkItem((object c) => ProcessRequest((HttpListenerContext)c), ctx);
            }
        }

        public void Stop()
        {
            if (m_server != null)
            {
                m_server.Stop();
                m_server.Close();
            }
        }

        private void ProcessRequest(HttpListenerContext context)
        {
            switch(context.Request.Url.AbsolutePath)
            {
                case "/registerPush":
                    HandlePushRegistration(context);
                    break;
                default:
                    context.Response.StatusCode = 404; //NOT FOUND
                    break;
            }
            context.Response.OutputStream.Close();
        }

        private void HandlePushRegistration(HttpListenerContext context)
        {
            if (context.Request.HttpMethod == "POST")
            {
                if (context.Request.HasEntityBody)
                {
                    System.IO.Stream body = context.Request.InputStream;
                    System.Text.Encoding encoding = context.Request.ContentEncoding;
                    System.IO.StreamReader reader = new System.IO.StreamReader(body, encoding);
                    DataContractJsonSerializer s = new DataContractJsonSerializer(typeof(RegistrationPacket));
                    var packet = s.ReadObject(reader.BaseStream) as RegistrationPacket;
                    if (packet != null && packet.deviceId != null && !string.IsNullOrWhiteSpace(packet.channelUri))
                    {
                        if (ClientRegistered != null)
                            ClientRegistered(this, packet);
                        context.Response.StatusCode = 200; //OK
                        return;
                    }
                }
            }
            context.Response.StatusCode = 500; //Server Error
        }

        /// <summary>
        /// Fired when a device registers itself
        /// </summary>
        public event EventHandler<RegistrationPacket> ClientRegistered;


        [DataContract]
        public class RegistrationPacket
        {
            [DataMember]
            public string channelUri { get; set; }
            [DataMember]
            public string deviceId { get; set; }
        }
    }
}

Next let’s start this service in the console main app. We’ll listen for clients registering and store them in a dictionary.

 

private static Dictionary<string, Uri> registeredClients = new Dictionary<string, Uri>(); //List of registered devices

static void Main(string[] args)
{
    //Start http service
    var svc = new HttpWebService();
    svc.Start();
    svc.ClientRegistered += svc_ClientRegistered;

    Console.WriteLine("Service started. Press CTRL-Q to quit.");
    while (true)
    {
        var key = Console.ReadKey();
        if (key.Key == ConsoleKey.Q && key.Modifiers == ConsoleModifiers.Control)
            break;
    }
    //shut down
    svc.Stop();
}
        
private static void svc_ClientRegistered(object sender, HttpWebService.RegistrationPacket e)
{
    if(registeredClients.ContainsKey(e.deviceId))
        Console.WriteLine("Client updated: " + e.deviceId);
    else
        Console.WriteLine("Client registered: " + e.deviceId);
    
    registeredClients[e.deviceId] = new Uri(e.channelUri); //store list of registered devices
}

Note: You will need to launch this console app as admin to be able to open the http port.

Next, let’s add some code to our phone/store app that calls the endpoint and sends its channel uri and device id (we won’t really need the device id, but it’s a nice way to identify the clients we have to push to and avoid duplicates). We’ll also add a bit of code to handle receiving a push notification if the server was to send a message back (we’ll get to the latter later):

private async void ButtonRegister_Click(object sender, RoutedEventArgs e)
{
    var channel = await PushNotificationChannelManager.CreatePushNotificationChannelForApplicationAsync();
    var uri = channel.Uri;
    channel.PushNotificationReceived += channel_PushNotificationReceived;
    RegisterWithServer(uri);
}

private async void RegisterWithServer(string uri)
{
    string IP = "192.168.1.17"; //IP address of server. Replace with the ip/servername where your service is running on
    HttpClient client = new HttpClient();

    DataContractJsonSerializer s = new DataContractJsonSerializer(typeof(RegistrationPacket));
    RegistrationPacket packet = new RegistrationPacket()
    {
        channelUri = uri,
        deviceId = GetHardwareId()
    };
    System.IO.MemoryStream ms = new System.IO.MemoryStream();
    s.WriteObject(ms, packet);
    ms.Seek(0, System.IO.SeekOrigin.Begin);
    try
    {
        //Send push channel to server
        var result = await client.PostAsync(new Uri("http://" + IP + ":8080/registerPush"), new StreamContent(ms));
        Status.Text = "Push registration successfull";
    }
    catch(System.Exception ex) {
        Status.Text = "Push registration failed: " + ex.Message;
    }
}

//returns a unique hardware id
private string GetHardwareId()
{
    var token = Windows.System.Profile.HardwareIdentification.GetPackageSpecificToken(null);
    var hardwareId = token.Id;
    var dataReader = Windows.Storage.Streams.DataReader.FromBuffer(hardwareId);

    byte[] bytes = new byte[hardwareId.Length];
    dataReader.ReadBytes(bytes);
    return BitConverter.ToString(bytes);
}

[DataContract]
public class RegistrationPacket
{
    [DataMember]
    public string channelUri { get; set; }
    [DataMember]
    public string deviceId { get; set; }
}

//Called if a push notification is sent while the app is running
private void channel_PushNotificationReceived(PushNotificationChannel sender, PushNotificationReceivedEventArgs args)
{
    var content = args.RawNotification.Content;
    var _ = Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
    {
        Status.Text = "Received: " + content; //Output message to a TextBlock
    });
}

Now if we run this app, we can register the device with you service. When you run both, you should see a “client registered” output in the console. If not, check your IP and firewall settings.

Lastly, we need to perform a push notification. The basics of it is to simply post some content to the url. However the service will need to authenticate itself using OAuth. To authenticate, we need to go back to the dev portal and go to the app we created. Click the “Services” option:

image

Next, go to the subtle link on the following page (this link isn’t very obvious, even though it’s the most important thing on this page):

image

The next page has what you need. Copy the highlighted Package SID + Client Secret on this page. You will need this to authenticate with the WNS service.

image

The following helper class creates an OAuth token using the above secret and client id, as well as provides a method for pushing a message to a channel uri using that token:

using System;
using System.IO;
using System.Net;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
using System.Text;
using System.Web;

namespace PushService
{
    internal static class PushHelper
    {
        public static void Push(Uri uri, OAuthToken accessToken, string message)
        {
            HttpWebRequest request = HttpWebRequest.Create(uri) as HttpWebRequest;
            request.Method = "POST";
            //Change this depending on the type of notification you need to do. Raw is just text
            string notificationType = "wns/raw";
            string contentType = "application/octet-stream";
            request.Headers.Add("X-WNS-Type", notificationType);
            request.ContentType = contentType;
            request.Headers.Add("Authorization", String.Format("Bearer {0}", accessToken.AccessToken));

            byte[] contentInBytes = Encoding.UTF8.GetBytes(message);
            using (Stream requestStream = request.GetRequestStream())
                requestStream.Write(contentInBytes, 0, contentInBytes.Length);
            try
            {
                using (HttpWebResponse webResponse = (HttpWebResponse)request.GetResponse())
                {
                    string code = webResponse.StatusCode.ToString();
                }
            }
            catch (Exception)
            {
                throw;
            }
        }

        public static OAuthToken GetAccessToken(string secret, string sid)
        {
            var urlEncodedSecret = HttpUtility.UrlEncode(secret);
            var urlEncodedSid = HttpUtility.UrlEncode(sid);

            var body =
              String.Format("grant_type=client_credentials&client_id={0}&client_secret={1}&scope=notify.windows.com", urlEncodedSid, urlEncodedSecret);

            string response;
            using (var client = new System.Net.WebClient())
            {
                client.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
                response = client.UploadString("https://login.live.com/accesstoken.srf", body);
            }
            using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(response)))
            {
                var ser = new DataContractJsonSerializer(typeof(OAuthToken));
                var oAuthToken = (OAuthToken)ser.ReadObject(ms);
                return oAuthToken;
            }
        }
    }

    [DataContract]
    internal class OAuthToken
    {
        [DataMember(Name = "access_token")]
        public string AccessToken { get; set; }
        [DataMember(Name = "token_type")]
        public string TokenType { get; set; }
    }
}

Next let’s change our console main method to include pushing a message to all registered clients when pressing ‘S’. In this simple sample we’ll just push the current server time.

static void Main(string[] args)
{
    //Start http service
    var svc = new HttpWebService();
    svc.Start();
    svc.ClientRegistered += svc_ClientRegistered;

    Console.WriteLine("Service started. Press CTRL-Q to quit.\nPress 'S' to push a message.");
    while (true)
    {
        var key = Console.ReadKey();
        if (key.Key == ConsoleKey.Q && key.Modifiers == ConsoleModifiers.Control)
            break;
        else if(key.Key == ConsoleKey.S)
        {
            Console.WriteLine();
            PushMessageToClients();
        }
    }
    //shut down
    svc.Stop();
}

private static void PushMessageToClients()
{
    if (registeredClients.Count == 0)
        return; //no one to push to

    //Message to send to all clients
    var message = string.Format("{{\"Message\":\"Current server time is: {0}\"}}", DateTime.Now);

    //Generate token for push

    //Set app package SID and client clientSecret (get these for your app from the developer portal)
    string packageSid = "ms-app://s-X-XX-X-XXXXXXXXX-XXXXXXXXXX-XXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX";
    string clientSecret = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
    //Generate oauth token required to push messages to client
    OAuthToken accessToken = null;
    try
    {
        accessToken = PushHelper.GetAccessToken(clientSecret, packageSid);
    }
    catch (Exception ex)
    {
        Console.WriteLine("ERROR: Failed to get access token for push : {0}", ex.Message);
        return;
    }

    int counter = 0;
    //Push the message to all the clients
    foreach(var client in registeredClients)
    {
        try
        {
            PushHelper.Push(client.Value, accessToken, message);
            counter++;
        }
        catch (Exception ex)
        {
            Console.WriteLine("ERROR: Failed to push to {0}: {1}", client.Key, ex.Message);
        }
    }
    Console.WriteLine("Pushed successfully to {0} client(s)", counter);
}

Now run the console app and the phone/store app. First click the “register” button in your app, then in the console app click “S” to send a message. Almost instantly you should see the server time printed inside your app.

image

 

Note: We’re not using a background task, so this will only work while the app is running. In the next blogpost we’ll look at how to set this up, as well as how to create a live tile with a graph on it.

You can download all the source code here: Download (Remember to update the Packet SID/Client Secret and associate the app with your own store app).

Fixing Visual Studio’s auto generated code

I usually have a zero-tolerance when it comes to build warnings. While granted often the warnings are benign, having a lot will very often hide the important ones. Sometimes I even set the option “treat warnings as errors” to help me enforce this. When you’re building a library, you should also add XML doc comments to your classes and members, so you get full intellisense support in the projects you’re using it from. Just remember to check off the following check box:

image

This also have the benefit of giving you warnings for missing doc comments on public members, so you remember to write proper doc. However if you’re building for Windows Store or Phone, there’s a good chance you’ll see these four warnings now:

image

These are coming from code auto-generated by the compiler which exposes a set of public classes. You can even see these in intellisense:

image

This is a major issue in my mind. First of all it introduces warnings in code you didn’t write, and it pollutes your class library with methods that are not meant to be used. Here’s what this auto-generated class looks like:

image

The fix seems simple: Just go in and add the doc comments, and problem is solved, right? Not really. The problem is any changes you make to this file is overwritten every time you build. So we need to tweak the file right after it’s being generated, but right before it’s being compiled, but who’s that fast?

Enter: The Build Task

We can use a build task to do exactly that, and tell Visual Studio to let us know that its about to compile the code and fix it ourselves.

First create a new empty Windows Class Library, and add the following two references:

image

Next add the following class and compile:

XamlTypeInfoBuildTask.cs

using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;
using System;

namespace XamlBuildTasks
{
    public class XamlTypeInfoBuildTask : Task
    {
        private const string XamlTypeInfoFileName = "XamlTypeInfo.g.cs";

        [Required]
        public string IntermediateOutputPath { get; set; }       
       
        public override bool Execute()
        {
            string filename = IntermediateOutputPath + XamlTypeInfoFileName;
            if (!System.IO.File.Exists(filename))
                return false;
            string code = System.IO.File.ReadAllText(filename);

            if (code.StartsWith("#pragma warning disable 1591")) //Already modified
                return true;
            int idx = code.IndexOf("[System.CodeDom.Compiler.GeneratedCodeAttribute");
            if (idx < 0)
                return false;
            string insert = "[System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]\n";
            code = "#pragma warning disable 1591\n" + code.Substring(0, idx) + insert + code.Substring(idx) +
                "#pragma warning restore 1591\n";
            System.IO.File.WriteAllText(filename, code);
            return true;
        }
    }
}

What does this class do? Simple: It has one property: The folder where the intermediate files including XamlTypeInfo.g.cs is. It then opens the file to be modified, and first injects “#pragma warning disable 1591” at the header which disables doc warnings, and re-enables it again at the bottom. At the same time we hide the class from intellisense, by setting the EditorBrowsable attribute on the class. This doesn’t really remove the class from the assembly – it just tells Visual Studio to skip showing this for intellisense.

To use this build-task, we need to add a little bit to the project file that causes this issue. We’ll first create a .targets file with the parameters for buildtask. Place this next to the compiled DLL (and use the name of the dll where highlighted):

XamlTypeInfoBuildTask.targets

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask TaskName="XamlTypeInfoBuildTask"
            AssemblyFile="$(MSBuildThisFileDirectory)XamlBuildTask.dll" />
  <Target Name="MyBuildTask" AfterTargets="MarkupCompilePass2" BeforeTargets="CoreCompile">
    <XamlTypeInfoBuildTask IntermediateOutputPath="$(IntermediateOutputPath)" />
  </Target>
</Project>

This tells the project to run this before the compilation after the markup has been processed. It also sets the IntermediateOutputPath property on our build task, so the build task can find the file.

Now the last step is to reference this .targets file from the csproj file. Open the csproj file up in notepad, and scroll down to the <Import…/> tags, and add the following (remember to modify the highlighted paths to where the targets file is:

  <Import Project="..\..\..\build\XamlTypeInfoBuildTask.targets" Condition="Exists('..\..\..\build\XamlTypeInfoBuildTask.targets')" />

Now when we build, we get a nice pretty build with no warnings:

image

And here’s all the auto-generated classes, gone from intellisense:

image

Making all this simpler

Naturally this is a bit of a pain having to set up over and over again. So unless you ever have to do a similar build task, you can forget everything you just read (sorry Smile), and just use the Nuget package I created that’ll do all this for you.

Simply add a reference to this nuget package, and the build task dll is downloaded and the .targets file auto-referenced.

And if you want the source code? It’s all available here on Github: https://github.com/dotMorten/XamlTypeInfoBuildTask

Enjoy!

Create Universal Map App In Under 2 Minutes

We just released the second beta drop of the 'ArcGIS Runtime for .NET SDK', which now supports both Windows Store 8.1, Windows Phone 8.1 in addition to WPF. This means you can now build universal apps with a map control that supports all these platforms. We also released the package on NuGet for your convenience.

As a demonstration how quick it is to create a Windows Phone and Windows Store app, here's a little video creating a universal map app from scratch in 80 seconds:

 

Note: The nuget package currently doesn't support WPF. You will need to download the full setup to get WPF support as well, instead of using the nuget version.

The layout of an ExtensionSDK

Extension SDKs are a very powerful way distribute your control libraries for use in Windows Store and Windows Phone apps.
This article will go through the layout of the extension sdk, and later take that knowledge to build an extension sdk from an already released app.

An ExtensionSDK essentially consists of 3 parts:

  • Files to use during design time
  • Files to deploy as content
  • Assemblies to use for reference

In addition there's a metadata file 'SDKManifest.xml' that describes the content.

The root layout then looks like the following:
    \EXTENSIONNAME\VERSION\DesignTime\
    \EXTENSIONNAME\VERSION\Redist\
    \EXTENSIONNAME\VERSION\References\
    \EXTENSIONNAME\VERSION\SDKManifest.xml
…where 'EXTENSIONAME' is the name of your extension, and VERSION is version number in the format "1.2.3.4".
   
For each of these groups you can control what gets deployed in debug and release or both. If you don't want to control whether you use debug or release, you will below these folders use the folder 'CommonConfiguration'. For debug specific configuration use 'Debug', and for release configuration use 'Retail'. In most case you will be using 'CommonConfiguration' though.
This means our folder structure now looks like this:
    \EXTENSIONNAME\VERSION\DesignTime\CommonConfiguration\
    \EXTENSIONNAME\VERSION\Redist\CommonConfiguration\
    \EXTENSIONNAME\VERSION\References\CommonConfiguration\

Next level down in the folders describe if files are related to AnyCPU, x86, x64 or ARM builds (the latter is very useful for C++ projects). For AnyCPU use 'neutral', meaning it doesn't matter. So use this for .NET Assemblies compiled for AnyCPU, image resources, winmd files etc. You will want to use the architecture specific folder if you deploy binaries that are architecture specific.

So what goes in what folders:

  • DesignTime: This is where you will put .Design assemblies if you have specific design time binaries for your assemblies, as well as Generic.xaml. You only need to deploy 'neutral' and/or 'x86' architectures, since VS runs in a 32bit process.
  • Redist: Images, shaders, Generic.xbf, videos etc, AND C++ binaries.
  • References: .NET DLLs, C++ WinMDs, xml doc.

Here's an example layout of an extension sdk that consists of two libraries: One C++ WinRT component (NativeLib) and a Managed library:
    \MyControlLib\1.0.0.0\SDKManifest.xml
    \MyControlLib\1.0.0.0\DesignTime\CommonConfiguration\neutral\ManagedLib\Themes\Generic.xaml
    \MyControlLib\1.0.0.0\DesignTime\CommonConfiguration\x86\ManagedLib.Design.dll
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\ManagedLib.pri
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\NativeLib.pri
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\ManagedLib\Icon.png
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\ManagedLib\Themes\Generic.xbf
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\NativeLib\shaders\PixelShader.cso
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\NativeLib\shaders\VertexShader.cso
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\ARM\NativeLib.dll
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\x86\NativeLib.dll
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\x64\NativeLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\neutral\NativeLib.winmd
    \MyControlLib\1.0.0.0\References\CommonConfiguration\ARM\ManagedLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\ARM\ManagedLib.xml
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x64\ManagedLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x64\ManagedLib.xml
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x86\ManagedLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x86\ManagedLib.xml

The SDKManifest could look like the following:

    <?xml version="1.0" encoding="utf-8" ?>
    <FileList
      xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
      xsi:noNamespaceSchemaLocation="SDKManifest.xsd"
      DisplayName="My Super Duper Control Library"
      ProductFamilyName="MyControlLib"
      Description="My Control Library"
      MinVSVersion="12.0"
      Identity="MyControlLib, Version=1.0.0.0"
      MinToolsVersion="12.0"
      AppliesTo="WindowsAppContainer + ( Managed )"
      SupportedArchitectures="x86;x64;ARM"
      DependsOn="Microsoft.VCLibs, version=12.0"
      SupportsMultipleVersions="Error">
        <File Reference="NativeLib.winmd" Implementation="NativeLib.dll" />
        <File Reference="ManagedLib.dll"/>
    </FileList>

   
Note that if you don't have native dependencies, this would change quite a lot. The full set of properties are pretty poorly documented today, so generally I download and install a wealth of extension sdks and look at them and see if they do similar things to me and then copy from that.

Building an Extension SDK from an installed app

So now that we know the layout of an extension sdk, let us apply that to 'reverse-engineering' an already deployed app into an extension sdk and use that to build our own app on top. Because Windows Store apps aren't fully encrypted, this means you can often take parts of an app that's separated out into libraries and build a new app from these libraries. This is something to consider when you build your app - if you are really good are separating your stuff into sub-libraries, you also make it easier for others to reuse your stuff. As an example let's download the Bing Maps Preview app and reverse it into an SDK and build our own 3D Map App.

When you installed the app, you will be able to access a folder with a name similar to the following with administrator rights:
"c:\Program Files\WindowsApps\Microsoft.Maps3DPreview_2.1.2326.2333_x64__8wekyb3d8bbwe\"

In here we'll find a lot of logic for the app, but the main one we are interested in is the "Bing.Maps" folder and the Bing.Maps dll+winmd. The folder is essentially the content and is image resources and shaders. The Bing Maps.dll and Winmd are C++ WinRT components. Since the dll is C++, the architecture will either be ARM, x86 or x64 depending on what PC you downloaded it on. In my case it's x64 so I should be able to build an extension sdk that will support 64 bit PCs from this alone. If I want to support more, I will have to install the app on a x86 or ARM PC and copy the dll from there as well (the other files are neutral and will be the same).

So let's first create the following folder : "Bing.Maps\1.0.0.0\".

Next, let's copy the "Bing.Maps" folder that has all the images and shaders into
    \Bing.Maps\1.0.0.0\Redist\CommonConfiguration\neutral\Bing.Maps\
Next copy the Bing.Maps.dll into (x64 if that's what you have, change/add ARM/x86 if your binary isn't x64)
    \Bing.Maps\1.0.0.0\Redist\CommonConfiguration\x64\Bing.Maps.dll
Lastly, copy the Bing.Maps.winmd into:
    \Bing.Maps\1.0.0.0\References\CommonConfiguration\neutral\Bing.Maps.winmd

Lastly we need to create a new file in \Bing.Maps\1.0.0.0\SDKManifest.xml to describe the SDK:

    <?xml version="1.0" encoding="utf-8" ?>
    <FileList
      xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
      xsi:noNamespaceSchemaLocation="SDKManifest.xsd"
      DisplayName="Bing Maps"
      ProductFamilyName="Bing.Maps"
      MinVSVersion="12.0"
      Identity="Bing.Maps, Version=1.0.0.0"
      MinToolsVersion="12.0"
      AppliesTo="WindowsAppContainer"
      SupportedArchitectures="x86;x64;ARM"
      DependsOn="Microsoft.VCLibs, version=12.0"
      SupportsMultipleVersions="Error">
        <File Reference="Bing.Maps.winmd" Implementation="Bing.Maps.dll" />
    </FileList>

Voila! We now have an ExtensionSDK. There's several ways you can 'install' this into Visual Studio. The simplest way is to copy the folder into your user folder under %USERPROFILE%\AppData\Local\Microsoft SDKs\<target platform>\v<platform version number>\ExtensionSDKs.
In this case %USERPROFILE%\AppData\Local\Microsoft SDKs\Windows\v8.1\ExtensionSDKs\

If you're building an installer you can also install it into
    %Program Files%\Microsoft SDKs\Windows\v8.1\ExtensionSDKs
Or specify a link to the location of the folder in a registry key:
    HKLM\Software\Microsoft\Microsoft SDKs\Windows\v8.1\ExtensionSDKs\Bing.Maps\1.0.0.0\

Lastly you can do this in your project file    by using the 'SDKReferenceDirectoryRoot' tag. Add the following right before the <Target Name="BeforeBuild"/> tag at the very bottom.

  <PropertyGroup>
    <SDKReferenceDirectoryRoot>c:\myfolder\my_sdks\;$(SDKReferenceDirectoryRoot)</SDKReferenceDirectoryRoot>
  </PropertyGroup>

Note that for the latter, the folder should point to a root extension sdk folder, meaning the SDK above must be located in a certain tree under this folder. In this case:
    c:\myfolder\my_sdks\Windows\v8.1\ExtensionSDKs\Bing.Maps

When you've done any of these install options, you can now get started building an app. Go to "Add References" and the Bing Maps entry should show up.

image

Now add the following XAML to your page:

        <bing:Map x:Name="map" xmlns:bing="using:Bing.Maps">
            <bing:Map.MapProjection>
                <bing:ThreeDimensionalMapProjection />
            </bing:Map.MapProjection>
        </bing:Map>

And in code-behind after "InitializeComponents":

            map.BaseLayers = Bing.Maps.BaseLayers.CreatePhotoRealisticOverlay();

Run the app and you should see a 3D globe!

image

Note: This Bing Maps SDK is not based on anything officially released but on a un-finished app. This is by all means a giant hack and only meant as an exercise to build an Extension SDK. Use this at your own risk and don’t attempt to publish any apps using it.

References:

Building A Powerful Platform for Windows Store Mapping apps

When Microsoft announced WinRT at the Build conference in Anaheim, I instantly started researching and prototyping what this new platform could mean for the company I’m working for. The promise of integrating legacy native code with XAML and .NET seemed to finally be the exactly what I’ve been looking for. Also the tight integration between XAML and DirectX, something which is really complex to get working in WPF was intriguing, since we were often hitting the limit of what XAML could do.

We have a huge amount of native code that does a lot of advanced spatial analysis, advanced map rendering, spatial databases, etc. Even better was that most of it is written in a cross-platform way using C++ and was already running on Windows Classic, Linux, iOS, Android and several other platforms.

In hindsight I’m really amazed how quickly this work can go. Granted a lot of time was spent on researching, prototyping, ‘socializing the idea’ etc, but after we had the bases right, we were actually able to move very fast, and in less than 3 months create a whole new SDK geared for the Windows Store (albeit beta).

The things that made this go so fast was:

  1. We had lots of C++ code that was already written to be compiled cross-platform, so most of the development time was spent on exposing this via the WinRT interface and C++/CX.
  2. We chose to build a hybrid SDK based on both C++ and C#. This enabled us to port large amount of our existing C# code from our WPF and Silverlight SDKs. It also allowed us to not be limited by the inheritance limitations that WinRT has by simply creating .NET class libraries rather than WinRT class libraries, which in turn greatly simplifies the API for other developers.

Things that set us back:

  1. Our rendering engine only supported DirectX 9 and OpenGL. Windows Store apps require DirectX 11, which is quite different from v9, so a lot of work had to be done there, because we wanted to do it in a way that wasn’t being hurt by the least common denominator (ie. if DX11 supports a feature that DX9 or OpenGL doesn’t, it shouldn’t hold us back from using it). In the end, our map rendering engine became better because of it for all the platforms.
  2. The documentation on SurfaceImageSource (the glue behind DirectX and XAML) was very limited.
  3. Some time was spent on making sure the native code passes certification, although not too bad.

Several people both within the company, from Microsoft, MVPs etc has been extremely helpful getting us through those setbacks. Thank you! You know who you are (and yes I owe you beer :-)

So enough about that. Instead, I would really encourage you to go download our SDK. It’s actually free! Just go to our developers siteand hit the download button. You’ll be required to register/sign in – don’t worry – as far as I know we don’t spam :-)

Grab the download, install it, and create a new C# or VB.NET Windows Store app. Add a reference to the ArcGIS Runtime SDKs, set the build target to x86, x64 or ARM (AnyCPU won’t work since this has cool C++ code in its guts).

image

And now code away. There’s a few samples on how to get started with the API as well as a library reference on the developers site.  We know the documentation is a little slim at this point – we’re hard at work improving that. However we do have a getting started tutorial here: http://developers.arcgis.com/en/windows-store/guide/add-a-map-to-your-app.htm

In addition we have a set of samples in the form of a sample browser available today on GitHub: https://github.com/Esri/arcgis-samples-winstore

There’s also a toolkit with some controls here that you are encouraged to fork and contribute to: https://github.com/Esri/arcgis-toolkit-winstore

I did a short introduction to the API at our plenary about 4 mins into this video:

You can download the source code for the sample I built on stage on my github account here: https://github.com/dotMorten/WinStore-Routing-Sample

Go download and code away today, and ask questions and give us feedback in the forum.

Also please go read the official SDK release announcement here: http://blogs.esri.com/esri/arcgis/2013/03/25/arcgis-runtime-sdk-for-windows-store-apps-beta-is-available/

Hacking the Silverlight Unit Tests to support returning Task

The Silverlight/Windows Phone unit test framework has always supported running asynchronous tests – a feature that until recently wasn’t there in WPF without jumping some really ugly (and flaky) hoops. Basically you can write a silverlight and windows phone unit test like this:

[TestClass]
public class TestClass1 : SilverlightTest
{
    [TestMethod]
    [Asynchronous] 
public void Test1() { DispatcherTimer timer = new DispatcherTimer() { Interval = TimeSpan.FromSeconds(2) }; timer.Tick += (a, b) => { timer.Stop(); base.TestComplete(); }; timer.Start(); } }

The problem with this code though is that this is only for Silverlight and Windows Phone. If you are cross-compiling for multiple platforms and want to run on WPF this wouldn’t work. It’s also not pretty that you have to inherit from SilverlightTest, remember to decorate the class with [Asynchronous] as well as calling TestComplete. Even worse, if you forget to stop the timer, it would CRASH the entire test run. The unit test framework is a little flaky when it comes to a task accidentally completing twice (instead of reporting it as an error, it crashes the entire test run and you’ll never get your daily test report…).

With Visual Studio 2012 and .NET 4.5 we can now simply return an object of type ‘Task’ and we would be good to go. This is awesome for testing your new async/await based stuff that returns task. So in WPF you would simply return your task object. As an example, let’s say we have the following really advanced computing task:

public static Task<int> Compute(int input)
{
    TaskCompletionSource<int> tcs = new TaskCompletionSource<int>();
    DispatcherTimer timer = new DispatcherTimer() { Interval = TimeSpan.FromSeconds(2) };
    timer.Tick += (a, b) =>
    {
        timer.Stop();
        if (input <= 0)
            tcs.SetException(new ArgumentOutOfRangeException("Number must be greater than 0"));
        else 
            tcs.SetResult(input);
    };
    timer.Start();
    return tcs.Task;
}

Now to test this in .NET 4.5 (including Windows Store Apps) you can simply write the following unit test:

[TestClass] 
public class TestClass1
{
[TestMethod] public async Task Test42() { var result = await Utility.Compute(42); Assert.AreEqual(result, 42); }
}

Nice and simple. However in Silverlight and Windows Phone you would have to write the following instead (I highlighted the extra or changed code required):

[TestClass]
public class TestClass1 : SilverlightTest
{
    [TestMethod]
    [Asynchronous]
    public async void Test42()
    {
        var result = await Utility.Compute(42);
        Assert.AreEqual(result, 42);
        base.TestComplete();
    }
}

Wouldn’t it be nice if the unit test I just wrote for WPF would work as is in Silverlight and on Windows Phone? Of course you could create a SilverlightTest class that has an empty TestComplete method, define an AsynchronousAttribute just for fun, and sprinkle a compiler conditional around the void/Task return type, but that just feels messy to me.

Fortunately the unit test framework for Silverlight is open source, so it’s possible to hack it in there. There are two main places you will need to change, which I will go through here. Note this is based on changeset #80285.

In the file “\Microsoft.Silverlight.Testing\UnitTesting\UnitTestMethodContainer.cs” we add the highlighted code to the method that detects if the Asynchronous attribute is on a method:

private bool SupportsWorkItemQueue()
{
    if (_testMethod != null)
    {
        if (_testMethod.Method.ReturnType != null && 
            _testMethod.Method.ReturnType == typeof(System.Threading.Tasks.Task) ||
            _testMethod.Method.ReturnType.IsSubclassOf(typeof(System.Threading.Tasks.Task)))
            return true; //Task Support
        else
            return ReflectionUtility.HasAttribute(_testMethod, typeof(AsynchronousAttribute));
    }
    else if (MethodInfo != null)
    {
        return ReflectionUtility.HasAttribute(MethodInfo, typeof(AsynchronousAttribute));
    }
    else
    {
        return false;
    }
}

Next is modifying the Invoke method that executes your test, which is located in ‘Microsoft.Silverlight.Testing\Metadata\VisualStudio\TestMethod.cs’. This is where the main work is done to enable tasks to work:

public virtual void Invoke(object instance)
{
    _methodInfo.Invoke(instance, None);
}

This now changes to:

public virtual void Invoke(object instance, CompositeWorkItem workItem)
{
    var t = _methodInfo.Invoke(instance, None) as System.Threading.Tasks.Task;
    if (t != null)
    {
        if (t.IsFaulted)
        {
            throw t.Exception;
        }
        else if (!t.IsCompleted)
        {
            var context = System.Threading.SynchronizationContext.Current;
            t.ContinueWith(result =>
            {
                context.Post((d) =>
                {
                    if (result.IsFaulted)
                    {
                        Exception ex = result.Exception;
                        if (ex is AggregateException)
                            ex = ex.GetBaseException();
                        workItem.WorkItemException(ex);
                    }
                    else
                        workItem.WorkItemCompleteInternal();
                }, null);
            });
        }
    }
}

Basically it grabs the task that is returned and calls the code that TestComplete would have called or what a raised exception would have called in case the test raises an exception. Also note that we changed the signature of the method to give us the CompositeWorkItem we need to raise these events on. This change does affect quite a lot of other code, but it’s merely a matter of adding the same parameter there as well, and the only place that calls this method (which is the CompositeWorkItem) to set this parameter to ‘this’.

Now you can also write tests that tests for exceptions thrown. Often you don’t even need to await the result in those cases:

[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public Task TestOutOfRange()
{
    return Utility.Compute(0);  //no need to await
}

[TestMethod]
public Task TestOutOfRange_Failure() //This test will fail
{
    return Utility.Compute(0);
}

And here’s what that looks like for the entire test run:

image

To make it easy on you, you can download the modified unit test framework source here.

…But EVEN better: Go vote for this to be part of the official toolkit here:  http://silverlight.codeplex.com/workitem/11457

A Simple Way To Use App Simulator For App Purchases

When you are testing In-App purchasing in your Windows 8 app, you need to use “Windows.ApplicationModel.Store.CurrentAppSimulator” static class instead of “Windows.ApplicationModel.Store.CurrentApp”. This means you’ll end up writing a lot of code like this:

     var licenseInformation =
#if DEBUG
           Windows.ApplicationModel.Store.CurrentAppSimulator.LicenseInformation;
#else
           Windows.ApplicationModel.Store.CurrentApp.LicenseInformation;
#endif
//...
     string receipt = await
#if DEBUG
            Windows.ApplicationModel.Store.CurrentAppSimulator.
#else
            Windows.ApplicationModel.Store.CurrentApp.
#endif
            RequestProductPurchaseAsync(featureName, true)

Yes this gets ugly really quick. Here’s a neat little trick to get around that. Right below your using statements at the top of your file, simply add this alias mapping:

#if DEBUG 
    using Store = Windows.ApplicationModel.Store.CurrentAppSimulator; 
#else 
    using Store = Windows.ApplicationModel.Store.CurrentApp; 
#endif

And in your code you can now simply write:

     var licenseInformation = Store.LicenseInformation; 
     string receipt = await Store.RequestProductPurchaseAsync(featureName, true);

Shortcut Key Handling in Windows Store Apps

I wanted to create a simple ALT+S shortcut in my app to jump to a TextBox in my Windows Store App (no this is not the Win+Q search charm shortcut). However, this is not that obvious to get working app-wide, so I’ll share it here:

The obvious way to do this is assign a KeyDown event to your page using the typical “this.KeyDown += MyKeyDownHandler”, or override OnKeyDown. However this has one problem: If any control that has focus currently handles key down events (like TextBox), this event won’t be raised due to how event bubbling works. However there is another way to create an event handler that overrides the bubbling: UIElement.AddHandler. In the 3rd parameter of that, you can specify that you want to be notified even if the event has been handled. Here’s what that looks like for listening to the event app-wide:

Window.Current.Content.AddHandler(UIElement.KeyDownEvent, new KeyEventHandler(App_KeyDown), true);
//...
private void App_KeyDown(object sender, KeyRoutedEventArgs e)
{
     //Handle Key Down
}

If you attach to Window.Current.Content, be sure to detach again in OnNavigatingFrom, or you’ll risk having a memory leak, and also still get events firing in that page when it’s not loaded any longer. If you just want this within the page, use myPage.AddHandler of Window.Current.Content.AddHandler, but beware that if anything outside this page has focus the event won’t be raised. – at least in that case you don’t need to worry about unhooking again though.

Now second is to handle the key combination. You can check the menu/alt key status using the following line of code:

bool isMenuKeyDown = CoreWindow.GetForCurrentThread().GetAsyncKeyState(VirtualKey.Menu) == CoreVirtualKeyStates.Down;

So the obvious handler code would look like this:

private void App_KeyDown(object sender, KeyRoutedEventArgs e)
{
    if (e.Key == Windows.System.VirtualKey.S)
    {
        if(CoreWindow.GetForCurrentThread().GetAsyncKeyState(VirtualKey.Menu) == CoreVirtualKeyStates.Down)
        {
            //Handle key combination… 
} } }

Turns out the above code only works every other time. When reading the documentation on GetAsyncKeyState it states that “Determines whether a key is up or down at the time the function is called, and whether the key was pressed after a previous call to GetAsyncKeyState.

So basically this method changes its result based on whether it was called before, and not solely whether the key is down or not. This makes no sense to me why it was designed like this (but do feel free to explain in the comments if you know).

Anyway, if we just make sure this method is always called in the handler it now starts working predictably. So here’s my snippet that ckecks if ALT+S is pressed, sets focus on my textbox and selects all the text:

private void App_KeyDown(object sender, KeyRoutedEventArgs e)
{
    bool isMenuKey = CoreWindow.GetForCurrentThread().GetAsyncKeyState(Windows.System.VirtualKey.Menu) == CoreVirtualKeyStates.Down;
    if (isMenuKey && e.Key == VirtualKey.S)
    {
        queryTextBox.Focus(FocusState.Keyboard);
        queryTextBox.SelectAll();
    }
}

Using User-Provided Images for Secondary Tiles

Often when you are creating a secondary tile in Windows 8, it will be based on images coming from the internet.  However a requirement of secondary tile images are that they need to be stored locally. I initially had some problems getting this working right and the streams closed correctly for this to work, so here’s the code for other to use and save the hazzle:

public async static Task CreateSecondaryTileFromWebImage(
    string tileId, Uri imageUri, string shortName, string displayName,
    string arguments, Rect selection)
{
    //Download image to LocalFolder and use the tileId as the identifier
    string filename = string.Format("{0}.png", tileId);
    HttpClient httpClient = new HttpClient();
    var response = await httpClient.GetAsync(imageUri);
    var imageFile = await ApplicationData.Current.LocalFolder.CreateFileAsync(filename, CreationCollisionOption.ReplaceExisting);
    using (var fs = await imageFile.OpenAsync(FileAccessMode.ReadWrite))
    {
        using (var outStream = fs.GetOutputStreamAt(0))
        {
            DataWriter writer = new DataWriter(outStream);
            writer.WriteBytes(await response.Content.ReadAsByteArrayAsync());
            await writer.StoreAsync();
            writer.DetachStream();
            await outStream.FlushAsync();
        }
    }
    //Create tile
    Uri image = new Uri(string.Format("ms-appdata:///local/{0}", filename));
    SecondaryTile secondaryTile = new SecondaryTile(tileId, shortName, displayName, arguments, TileOptions.ShowNameOnLogo, image);
    secondaryTile.ForegroundText = ForegroundText.Light;
    await secondaryTile.RequestCreateForSelectionAsync(selection, Windows.UI.Popups.Placement.Above);
}

Often this is enough, but there is still a small problem. What if the image is very light, and the text you display on top of it is white, thus drowning in the background image? You could set the tile text to black, but then what happens if the image is dark? And if it has a lot of texture in it, the text still gets very unreadable. Since you might not know up front what the image looks like, whether it’s dark or bright, or textures, we will need a way to ensure the text will still look good on top of the image.

image

Unfortunately full WriteableBitmap support in WinRT isn’t there to help us out modifying the image, but we do get low-level access to the pixel buffer of the image, so we could fairly simple darken or brighten the bottom a bit to ensure the image looks good as a backdrop for the text.

I wrote a little utility that loads the image, gradually darkens the bottom 40% of the image before saving it back. I’ve found that doing a slight graduated darkening on photos isn’t too noticeably, while making the text in front of it much more readable. So with out further ado, here’s my simple image pixelbuffer modifier:

private async static Task DarkenImageBottom(string filename, string outfilename)
{
    var file = await ApplicationData.Current.LocalFolder.GetFileAsync(filename);
    BitmapDecoder decoder = null;
    byte[] sourcePixels = null;
    using (IRandomAccessStream fileStream = await file.OpenReadAsync())
    {
        decoder = await BitmapDecoder.CreateAsync(fileStream);
        // Scale image to appropriate size 
        BitmapTransform transform = new BitmapTransform();
        PixelDataProvider pixelData = await decoder.GetPixelDataAsync(
            BitmapPixelFormat.Bgra8,
            BitmapAlphaMode.Straight,
            transform,
            ExifOrientationMode.IgnoreExifOrientation, // This sample ignores Exif orientation 
            ColorManagementMode.DoNotColorManage
        );
        // An array containing the decoded image data, which could be modified before being displayed 
        sourcePixels = pixelData.DetachPixelData();
        fileStream.Dispose();
    }
    if (decoder != null && sourcePixels != null)
    {
        for (uint col = 0; col < decoder.PixelWidth; col++)
        {
            for (uint row = (uint)(decoder.PixelHeight * .6); row < decoder.PixelHeight; row++)
            {
                uint idx = (row * decoder.PixelWidth + col) * 4;
                if (decoder.BitmapPixelFormat == BitmapPixelFormat.Bgra8 ||
                    decoder.BitmapPixelFormat == BitmapPixelFormat.Rgba8)
                {
                    var frac = 1 - Math.Sin(((row / (double)decoder.PixelHeight) - .6) * (1 / .4));
                    byte b = sourcePixels[idx];
                    byte g = sourcePixels[idx + 1];
                    byte r = sourcePixels[idx + 2];
                    sourcePixels[idx] = (byte)(b * frac);
                    sourcePixels[idx + 1] = (byte)(g * frac);
                    sourcePixels[idx + 2] = (byte)(r * frac);
                }
            }
        }

        var file2 = await ApplicationData.Current.LocalFolder.CreateFileAsync(outfilename, CreationCollisionOption.ReplaceExisting);

        var str = await file2.OpenAsync(Windows.Storage.FileAccessMode.ReadWrite);
        BitmapEncoder enc = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, str);
        enc.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore, decoder.PixelWidth, decoder.PixelHeight,
            decoder.DpiX, decoder.DpiY, sourcePixels);
        await enc.FlushAsync();
        str.Dispose();
    }
}

So we can call this utility prior to creating the secondary tile request. Compare the before(left) and after (right) here:

imageimage

The image difference is barely noticeable, but the text is much more readable.

You can download the tile utility class and sample app here.