Custom Cursors in Silverlight

There has been quite a few blog posts on how to create custom cursors in Silverlight, but I felt some of them were very limited or not very re-useable, or required you to put the cursor in the layout and put restrictions on your layout (like for instance requiring you to use a Canvas as parent for your cursor). So I ended up writing my own class for it that doesn’t have any of these restrictions.

The basic idea is the same as other approaches out there: You “disable” the cursor on your element by setting myElement.Cursor = Cursors.None. Next you move a custom UIElement around on top of your layout which represents your custom cursor. My approach uses a Popup, and therefore never requires you to modify your layout, and you can easily change the cursor on the fly. The cursor look is defined using a DataTemplate and can contain animations and so on.

All you need to do is define a DataTemplate and use the Attached Property to define the template. Below is an example of this working on a border element:

<UserControl x:Class="CustomCursor.MainPage"
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   xmlns:local="clr-namespace:SharpGIS.CustomCursor">
   <UserControl.Resources>
        <DataTemplate x:Key="CircleCursor">
            <Ellipse Fill="Red" Width="20" Height="20" />
        </DataTemplate>
   </UserControl.Resources>
   <Grid>
       <Border Background="Blue" 
           local:CustomCursor.CursorTemplate="{StaticResource CircleCursor}" />
   </Grid>
</UserControl>

And that’s all there is to it!

If you need to customize where the centering of the element is, simply apply a TranslateTranform to the element. Ie. if you want the above circle to center on the mouse location, add a TranslateTransform of –10,-10 to the ellipse.

Get Microsoft Silverlight

Downloads:

Update: Someone awesome ported this to Windows Store apps as well! Go download here: http://customcursor.codeplex.com/

Windows Phone 7 Copy/Paste on TextBlocks

With the recent update of Windows Phone 7, AKA the "NoDo" update, Windows Phone got support for Copy/Paste. Any existing app out there instantly gets this support automatically, without the developer has to do anything.

However, copy/paste only works on TextBoxes, but what if you want to allow the user to copy text from a TextBlock? Today this is not possible. The trick to make this work is to simply use a readonly TextBox, and restyle it to look like a TextBlock. Here's the TextBox style that accomplishes this:

<Style x:Key="ReadOnlyTextBox" TargetType="TextBox">
	<Setter Property="Foreground" Value="{StaticResource PhoneForegroundBrush}" />
	<Setter Property="IsReadOnly" Value="True" />
	<Setter Property="TextWrapping" Value="Wrap" />
	<Setter Property="Template">
		<Setter.Value>
			<ControlTemplate>
				<Grid>
					<ContentPresenter x:Name="ContentElement" />
				</Grid>
			</ControlTemplate>
		</Setter.Value>
	</Setter>
</Style>

Note: This approach actually also applies to "normal" browser-Silverlight as well.

RefreshBox for Windows Phone 7

I always liked the simplicity in iPhone for refreshing a list of various feeds, for instance email, tweet, RSS etc. Basically when you get to the top of the list, you simply keep pulling and the app will check for the latest items:

image

I wanted something similar in my WinPhone application, and ventured out to make this. You could probably go and argue now that Windows Phone 7 shouldn’t be copying ideas from other phones but be its own, which is a somewhat fair argument, but when something just works, it just works. So yes this post is about totally ripping of an iPhone idea. I’ll admit that Smile

If you don’t care how this was done, just jump to the bottom to see the result in action, and/or download the bits.

Anyway, luckily creating a control like this wasn’t too hard. It starts with inheriting from the ListBox control which has all the list features needed. It also supports pulling beyond the contents of the list and when you let go, it “pops back” to the top item. So all I need to do, is :
1. Find a way to place an item in this “outside area”.
2. Detect that the user is pulling the list out in this area.

Placing the element in the outer area proved to be really simple. The ListBox template is basically just a ScrollViewer with an ItemsPresenter inside it. So all we need to do is place a grid with the contents on top of the ItemsPresenter with a negative vertical margin:

<ControlTemplate TargetType="local:RefreshBox"> 
    <ScrollViewer x:Name="ScrollViewer" ...> 
        <Grid> 
            <Grid Margin="0,-90,0,30" Height="60" VerticalAlignment="Top" x:Name="ReleaseElement"> 
                <!-- content goes here --> 
            </Grid> 
        </Grid> 
        <ItemsPresenter/> 
    </ScrollViewer> 
</ControlTemplate>

Sprinkle a little visual states in that content, and we would be able to change the message when we detect the user has been pulling beyond the threshold.

The first thing we do is get a reference to the ScrollViewer and the ReleaseElement during OnApplyTemplate(), and listen to MouseMove (the event that causes the scrolling) and ManipulationCompleted (triggered when the user lets go of the screen).

public override void OnApplyTemplate() 
{ 
    base.OnApplyTemplate(); 
    ElementScrollViewer = GetTemplateChild("ScrollViewer") as ScrollViewer; 
    if (ElementScrollViewer != null) 
    {            
        ElementScrollViewer.MouseMove += viewer_MouseMove; 
        ElementScrollViewer.ManipulationCompleted += viewer_ManipulationCompleted; 
    } 
    ElementRelease = GetTemplateChild("ReleaseElement") as UIElement;        
    ChangeVisualState(false); 
}

In MouseMove we can then check the VerticalOffset of the ScrollViewer. You would think this offset is negative, but it gets clamped at 0, so we can’t use that. Instead by calculating where the ReleaseElement is physically located gives us an idea of how far we pulled:

private void viewer_MouseMove(object sender, MouseEventArgs e) 
{ 
    if (VerticalOffset == 0) 
    { 
        var p = this.TransformToVisual(ElementRelease).Transform(new Point()); 
        if (p.Y < -VerticalPullToRefreshDistance) //Passed thresdhold : In pulling state area 
        {             
            //TODO: Update layout//visual states 
        } 
        else //Is not pulling 
        { 
            //TODO: Update layout/visual states 
        } 
    } 
}

Similarly, when you let go of the screen we make the same check and raise an event if necessary:

private void viewer_ManipulationCompleted(object sender, ManipulationCompletedEventArgs e) 
{ 
    var p = this.TransformToVisual(ElementRelease).Transform(new Point()); 
    if (p.Y < -VerticalPullToRefreshDistance) 
    { 
        //TODO: Raise Polled to refresh event 
    } 
}

So here’s that this looks like….

Scrolled to the top of the list:

image

Pulling slightly down on the listbox beyond the top item:

image

Pulling all the way down: Message tells you to let go to refresh:

image

And here’s what that look like in action:

You can download the code and binary for a fully skinnable control/reusable below here:

Source : SharpGIS.Controls.RefreshBox_source.zip (32.31 kb)

Binary : SharpGIS.Controls.RefreshBox_binary.zip (6.48 kb)

UPDATE: If you are using v7.1 (Mango), the above approach will only work if you set ScrollViewer.ManipulationMode ="Control" on the RefreshBox. See herefor details.

Using the accelerometer to control planar transforms on Windows Phone 7

Lately I’ve been looking into some Augmented Reality uses on Windows Phone 7, and one of the first things you need to do for this, is to use the sensors to control what’s displayed on the phone screen.

So the first step for me was to better understand how the accelerometer interacted with the phone’s orientation. I wanted to create a simple “3D Plane” that was always looked like it was “level”. This could be accomplished with a Rectangle and a PlaneTransform. The only question is: How do I transform the accelerometer values to the RotationX/Y/Z rotation angles on the PlaneTransform?

To make matters worse, depending on whether your phone is currently in Portrait, LandscapeLeft or LandscapeRight mode, the screen coordinate system, and thus the transforms changes, even though the accelerometer report the values independent of screen orientation. So to get a property transform, we also need to know what way the screen coordinate system is oriented relative to the accelerometer coordinate system.

With a little trigonometry it wasn’t too hard to figure out, so here’s the code to perform this little app.

First the border we used as the “level” plane:

<Border x:Name="plane" BorderBrush="Green" BorderThickness="2"
        Width="350" Height="350" >
    <Border.Projection>
        <PlaneProjection x:Name="proj"  />
    </Border.Projection>
</Border>

Next, create an accelerometer instance, and call “Update” every time we get a reading:

var meter = new Accelerometer();
meter.ReadingChanged += meter_ReadingChanged;
...
private void meter_ReadingChanged(object sender, AccelerometerReadingEventArgs e)
{
    Dispatcher.BeginInvoke(() =>
    {
        UpdateRotation(e.X,e.Y.e.Z);
    });
}

Lastly, update the PlaneProjection parameters, based on the rotation (this is where the “real meat” is):

private void UpdateRotation(double x, double y, double z)
{
    var offset = 0d;
    //Adjust for screen orientation when in landscape mode
    //PortraitUp mode doesn't need an offset
    if (Orientation == PageOrientation.LandscapeLeft) offset = .5;
    else if (Orientation == PageOrientation.LandscapeRight) offset = 1.5;                

    var rx = (Math.Acos(z) / Math.PI + 1) * 180;
    var rz = (offset - Math.Atan2(x, y) / Math.PI + 1) * 180;
    if(!double.IsNaN(rz)) proj.RotationZ = rz;
    if(!double.IsNaN(rx)) proj.RotationX = rx;
}

And lastly, here’s what this looks like (note how the screen orientation flips when the phone is rotated, but the plane stays in the same place):

Note that this sample doesn’t deal with calibration. I suggest you look at this blogpost on how to do that. It also includes a lot of information on the accelerometer in general. That blog also discusses filtering the noisy data which is important to give a smooth looking result. The sample code you can download below, includes a simple low-pass filter to make it feel a lot smoother.

Download source code

Lastly here’s an example/preview of a little Augmented Reality app I’ve been working on that is using this approach to overlay the camera feed with azimuth values:

ESRI DevSummit 2011–over'n'out

The ESRI Developer Summit is over, and I had quite a blast. We got tons of great feedback and tough questions from all the developers attending. Thank you for that! We also had some well-attended sessions on Windows Phone, Silverlight and WPF. When these recordings goes live, I’ll update this blog-post.

For the first time, I even got to go on stage during plenary and make a fool of myself. You might already have read about my ventures into programming with a Kinect, but I got to show a far more polished version during the plenary session, and luckily everything worked out great. This version (although not demoed due to time constraints) actually supports multiple users interacting with the map at the same time (at one point we had 4 different people drawing on the map all simultaneously). For those interested, this was based on OpenNI drivers and the ArcGIS API for WPF, which is a different approach than I originally blogged about. Here’s a recording of my 2 minutes of “fame” Smile:

Me on stage making a fool of myself

I have to admit I was a little nervous this would break down during the demo, since this was all based on stuff that wasn’t even in beta software yet. We later had it running at our showcase area 24/7 and I didn’t have to restart the app for almost 3 days running with lots of users playing with it, so perhaps it was running more solid than I had feared Smile - So go play with those OpenNI drivers! They might be called “unstable” but I found them to be very stable.

My little A-to-B app also go a nice mention during the plenary. I wrote this to learn some more phone application practices, exercise the phone API we’ve been working on for a while and simply to have some coding fun. If you have a Windows Phone 7, go download it now, and if not, here’s Dave giving a quick tour of the app.

A-to-B WinPhone7 Application demoed

Thank you to every who attended and especially those who gave us feedback.

Fun times with Kinect and WPF

I recently bought a Kinect controller for my Xbox and is really loving it. Of course I had to see if I couldn’t use that for something, and after some other people were nice enough to create and release free Windows drivers and libraries, I had to try it!

I started out with Code Laboratories “CL NUI Platform” which is a simple installer that installs the drivers and some libraries. It also comes with some samples showing how to connect to the sensor.

I quickly got that going, with a small app that displays the RGB and Depth image from the sensor.

223324934[2]

“Unfortunately” this is also all you get out of the box. If you expect to automatically get an event when you wave your hand or doing some other gesture, this is not for you. However if have a little love for some simple Image Analysis algorithms, you should be all over this too!

I wanted to make a simple gesture API that could tell me when a user drags, pinch/stretch or flicks something, very similar to what you do with a Touch Screen. So basically an API that mimics an invisible touch screen hanging suspended in the air. Luckily the algorithms for this turned out to be pretty simple.

The first thing you do is go through the depth image pixel by pixel. If you find a pixel that is too close or too far away, replace it with Color.Empty so we can ignore those. That we we will only “see” things in the picture that are within a certain distance from the sensor. Like for instance two hands stretched out. This leaves us with a set of pixels that could potential be some stretched out hands.

223411798[2]

We need to associate these pixels with each other. Let’s say all pixels that are next to each other that are not transparent, belong to the same hand. We call these sets of grouped pixels for “Blobs” and there’s a nice simple algorithm for detecting all the blobs in an image, known as “Connected Component Labelling”. Instead of explaining how it works here, all you need to know is here on wikipedia. If you register their center and size, you can easily use these as “touch input” and discard too large or small blobs. This is what this looks like, using a circle for center and size of each blob:

30 times a second you will be calculating a new set of blobs. Compare these to the previous blobs, and use the center and size to see which blobs are the same and has moved, which are new, and which disappeared. If I only have one blob and it moves, I consider this a “drag”. If I have more than 1, I calculate the bounding box of all the blobs, measure its diagonal and compare it to the size of the previous diagonal - The fraction between these two diagonals is the scale in a pinch or stretch.

Now all we need to do is take these “gesture events” and hook them up to something. Like for instance a map. So here’s the end-result: (note : It looks a little jerky because of the screen capture software, but it runs a lot smoother when not recording and flicks works consistently).

Minority MapReport

It worked surprising well for a first go.

Download source: Sorry no source available at this time, due to some licensing restrictions.

Simulating GPS on Windows Phone 7

If you are building Windows Phones apps that require the use of GPS, you cannot use the Emulator, and should preferably be outside.

While building some of my latest WinPhone apps, I needed to be able to able test moving around on a map, while being able to debug. However I preferred doing this at the comfort of my desk. So I had to “fake” a GPS signal. Luckily the GeoCoordinateWatcher class that is used for this, implements the interface “IGeoPositionWatcher<GeoCoordinate>”.  If I were to create my own class that implements this interface, I had an option for using a separate class for receiving data from the location service. I ended up needing two types of location simulators for my routing app: One that starts at a given location and direction, and randomizes speed and direction over time. The other follows a predefined route. You can download the source code for these below.

The next step is to automatically use the simulator when I’m running on the emulator, but use the “real” location service when running on the device. You can accomplish this by checking the DeviceName first. Example:

   1:  IGeoPositionWatcher<GeoCoordinate> geowatcher; 
   2:  string deviceName = Microsoft.Phone.Info.DeviceExtendedProperties.GetValue("DeviceName").ToString(); 
   3:  if (deviceName == "XDeviceEmulator") //Use simulator for emulator 
   4:      geowatcher = new GeoCoordinateSimulator(34.0568, -117.195, 270) 
   5:  else 
   6:      geowatcher = new GeoCoordinateWatcher(GeoPositionAccuracy.High); 
   7:  geowatcher.PositionChanged += watcher_PositionChanged; 
   8:  geowatcher.Start();

The above uses the “random” simulator when running on the emulator.

The other simulator takes a set of GeoCoordinates and interpolates between them at the specified speed. Example:

   1:  var pnts = new List<GeoCoordinate>(); 
   2:  pnts.Add(new GeoCoordinate() { Longitude = -117.195670639, Latitude = 34.0568044102 }); 
   3:  pnts.Add(new GeoCoordinate() { Longitude = -117.195697272, Latitude = 34.0585127272 }); 
   4:  pnts.Add(new GeoCoordinate() { Longitude = -117.204302784, Latitude = 34.0515719501 }); 
   5:  geowatcher = new SharpGIS.WinPhone.Gps.GpsDriveSimulator(pnts, 20);

Below is an example of this simulator in action on the emulator in a little app I’m working on:

screenshot_12-7-2010_22.7.41.326photo[1]

It was fun putting this app on the real device, go for a drive along roughly the same route and it worked exactly same. The simulator helped me weed out a series of bugs that would have taken me several (dangerous) “test drives” if I hadn’t had this simulator.

Download Simulator code here:

Submitted my first WinPhone 7 Application

imageI’ve just submitted my first application to the Windows Phone Marketplace. It’s a very simple app that creates a “Guide Post” with signs on a pole showing distances and direction to any point in the world. You have probably seen these at various sites all over the world, or in the TV Show “M*A*S*H”.

It’s all built in Silverlight, and uses PlaneProjection to give give the signs a 3D effect. You use your finger to slider over the screen to rotate it.

In addition to that there’s a Map where you can view your current location and the great circle line (ie. shortest path) between you and the points of interest. This is of course using the ESRI ArcGIS API for Windows Phone that we just released.

Below is a few screenshots and a video clip of the app in action.

screen_12-1-2010_8.51.30.996screen_12-1-2010_8.50.48.336
screenshot_12-2-2010_12.26.46.864screenshot_12-2-2010_12.27.59.830

GuidePost Screen Capture

It was a great fun little app to build and only took a few evenings to get done. You can download the app to your phone from this link: http://bit.ly/WP7GP

Displaying HTML in Silverlight

In my second post in the series of ripping off other peoples work :-) here’s an updated version of David Anson’s HtmlTextBlock control back from 2007.

This was originally written based on the TextBlock control, but since the TextBlock control was sealed and couldn’t be inherited from, it required a lot of copying properties around. Textblocks also only supports text, so the HyperLink method was not really a hyperlink, and there was no image support, but from what we had available back them, it was a really good solution considering the simplicity (I like simple small implementations).

Since then Silverlight 4 has arrived, and we now have the RichTextBlock control. This supports hyperlinks, images etc, and it is not sealed, so we can pretty easily get around the limitations that the original implementation had. Most of the logic for generating the HTML is the same, but now with proper support for images and real hyperlinks. Again all the credit goes to the original author. I just did some tweaks.

You can download the source with a sample app here: HtmlTextBlock.zip (34.15 kb)

UPDATE: Rewrote the parser approach quite a lot. Now it also supports tables (!) as well as setting fontsize using the H1..H4 tags. With the new parser approach it should be easier to add support for other more complex tags as well.

You can download the source with a sample app here: HtmlTextBlock_v2.zip (34.15 kb)

REALLY small unzip utility for Silverlight – Part 2

I earlier blogged about a very small unzip utility for Silverlight.

However there was one problem with the method: It relied on some Silverlight framework libraries that didn’t support all zip file formats (a known bug I’m told). Some file zippers places the file size after the zip data chunk, and Silverlight does not like that. So I added a little bit to the utility to detect this case and if so, will re-arrange the bytes in the zipfile so Silverlight will read it.

So I present to you: The really small unzip utility v.2.0. It’s slightly larger than before, but still under 200 lines of code (plus comments).

It works the same way as before, so read the original blogpost to see how to use it.

Although I only tested this on Silverlight 4 and WinPhone 7, it should work with earlier Silverlight releases as well.

Download here: SharpGIS.UnZipper_v2.zip (2.53 kb)

Also available at NuGet