SharpGIS

#GIS from a .NET developer's perspective

Adding SharpMap geometry to a PostGIS database

...or how to upload an ESRI Shapefile to PostGIS.

I've often been asked how to copy data from a shapefile to a PostGIS database. PostGIS comes with a commandline-tool for this (shp2pgsql.exe), but all it does is generate a huge SQL-textfile that you will need to run afterwards - Not very efficient I think - especially with the ASCII-representation of the geometry. Furthermore I've had several problems with it regarding many international letters in the attributes.

So why not try to let Npgsql and SharpMap do the job?

I've been working a bit with a small tool that makes it easy to upload an entire shapefile to a PostGreSQL/PostGIS database using SharpMap.

Below are some of the PostGIS/SharpMap related code explained:


 

 

First we create a Npgsql connection

NpgsqlConnection conn = new NpgsqlConnection("Server=localhost;Port=5432;User Id=username;Password=password;Database=myGisDB;")
NpgsqlCommand command = new NpgsqlCommand
();
command.Connection = conn;

The next step is to add a geometry column (see in the full source on how you create the table with all the attributes). In this case we set the spatial reference ID to '-1' and name the geometry column 'geom'.

command.CommandText = "SELECT AddGeometryColumn('','myTable','geom','-1','GEOMETRY',2);";
command.ExecuteNonQuery();

Now we are ready to upload to the database, so lets get hold of that shapefile! First we set up a datasource:

SharpMap.Data.Providers.ShapeFile shp = new SharpMap.Data.Providers.ShapeFile(@"C:\data\MyShape.shp", false);

We can now query all the feature object IDs, by using an extents-query on the full extents:

conn.Open();
List<uint> indexes = shp.GetObjectIDsInView(shp.GetExtents());

...and then loop through all the features:

foreach (uint idx in indexes)
{
   SharpMap.Data.FeatureDataRow
feature = shp.GetFeature(idx);
   
command.CommandText = "INSERT INTO \"myTable\" (\"geom\") VALUES (GeomFromWKB(:geom,:srid));"
;
   command.Parameters.Add(":geom", NpgsqlTypes.NpgsqlDbType
.Bytea);
   command.Parameters[":geom"
].Value = feature.Geometry.AsBinary(); //Add the geometry as Well-Known Binary
   command.Parameters.Add(":srid", NpgsqlTypes.NpgsqlDbType
.Integer);
   //Set the SRID of the geometry - this must match the SRID we used when the column was created
   command.Parameters[":srid"].Value = -1;

   
//TODO: Add parameters for remaining columns if nessesary (in that case alter the INSERT commandtext accordingly) 
 
   command.ExecuteNonQuery();
}
//Clean up
conn.Close();
shp.Close();

...and that is all there is to it !

The great thing about this, is that it is easy to change this to take any other SharpMap datasource and upload as well. And with Christians OGR extension you can suddenly upload a bunch of datasource directly to PostGIS.

Download the full source and compiled binaries here: Shape2Pgsql.zip (624,3 KB) (updated April 26, 2006)

Working with 8bit images in .NET

For some reason Microsoft made it pretty hard to work with anything but 24bit images. This is even though they provide you with several pixel formats, but no way of setting and getting the values of a pixel. If you use the SetPixel(x,y) or GetPixel(x,y) methods, your application will fail. There are plenty of articles and blogs on the Internet on how to do direct access on 1bit and 24bit, but I wasn't able to find anything on 8bit.

This article will cover some of the basics on how to access 8 bit greyscale or indexed images, by accessing the bitmapdata directly in memory. This also has the benefit of being much faster than the Set/GetPixel methods provided by the .NET Framework.

Before we can access the memory directly, we must lock its place in memory. We can do this by calling the Bitmap.LockBits() method:

BitmapData bmd  = myBitmap.LockBits(new Rectangle(0, 0, myBitmap.Width, myBitmap.Height),
ImageLockMode.ReadWrite, myBitmap.PixelFormat);

Likewise when we are done using the BitmapData, remember to unlock the data:

myBitmap.UnlockBits(bmd);

Now we need a method that can access the BitmapData. Lets make our own SetPixel and GetPixel method. Here we assume that we are dealing with 8bit pixels. We also add the 'unsafe' keyword since direct memory access isn't thread safe. I won't cover the Stride and Scan0 values. Bob Powell has a nice article on this.

public unsafe void SetPixel(int x, int y, byte c)
{
	byte* p = (byte *)bmd.Scan0.ToPointer();
	int offset=y*bmd.Stride+(x);
	p[offset] = c;
}

public unsafe Byte GetPixel(int x, int y)
{
	byte* p = (byte *)bmd.Scan0.ToPointer();
	int offset=y*bmd.Stride+x;
	return p[offset];
}

It is worth noting that GetPixel only returns a byte and not a color. The byte represents a number between 0 and 255. Each of the values is actually an index to a color palette. The palette could specify that for instance index 0 is black, index 1 is red, index 3 is blue etc. If you want a greyscale image, we can override the color palette. Let's set index 0 to black, index 255 to white, and linearly distribute the grayscale in between.

public static void SetGrayscalePalette(Bitmap b)
{
	ColorPalette pal = b.Palette;
	for(int i = 0; i < 256; i++)
	pal.Entries[i] = Color.FromArgb( 255, i, i, i );
	b.Palette = pal;
}

You can easily override this palette to specify other than grayscale images.

We can likewise create a function that can convert an index to a System.Drawing.Color. If you are working with a grayscale image, there is probably no need for this.

public System.Drawing.Color GetColorFromIndex(byte c)
{
	return = myBitmap.Palette.Entries[c];
}

Now let's put it all together into an easy-to-use 8bit image access class. Remember to allow unsafe code blocks before compiling.

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
namespace ImageProc
{
   /// <summary>
   /// Class used for direct memory access to 8bit grayscale images
   /// </summary>
   public class Image8Bit : IDisposable
   {
      private BitmapData bmd;
      private Bitmap b;
      /// <summary>
      /// Locks an 8bit image in memory for fast get/set pixel functions.
      /// Remember to Dispose object to release memory.
      /// </summary>
/// Bitmap reference public Image8Bit (Bitmap bitmap) { if(bitmap.PixelFormat!=System.Drawing.Imaging.PixelFormat.Format8bppIndexed) throw(new System.Exception("Invalid PixelFormat. 8 bit indexed required")); b = bitmap; //Store a private reference to the bitmap bmd = b.LockBits(new Rectangle(0, 0, b.Width, b.Height),
ImageLockMode.ReadWrite, b.PixelFormat); } /// <summary> /// Releases memory /// </summary> public void Dispose() { b.UnlockBits(bmd); } /// <summary> /// Gets color of an 8bit-pixel /// </summary> /// <param name="x">Row</param> /// <param name="y">Column</param> /// <returns>Color of pixel</returns> public unsafe System.Drawing.Color GetPixel(int x, int y) { byte* p = (byte *)bmd.Scan0.ToPointer(); //always assumes 8 bit per pixels int offset=y*bmd.Stride+x; return GetColorFromIndex(p[offset]); } /// <summary> /// Sets color of an 8bit-pixel /// </summary> /// <param name="x">Row</param> /// <param name="y">Column</param> /// <param name="c">Color index</param> public unsafe void SetPixel(int x, int y, byte c) { byte* p = (byte *)bmd.Scan0.ToPointer(); //always assumes 8 bit per pixels int offset=y*bmd.Stride+(x); p[offset] = c; } /// <summary> /// Sets the palette for the referenced image to Grayscale /// </summary> public void MakeGrayscale() { SetGrayscalePalette(this.b); } /// <summary> /// Sets the palette of an image to grayscales (0=black, 255=white) /// </summary> /// <param name="b">Bitmap to set palette on</param> public static void SetGrayscalePalette(Bitmap b) { ColorPalette pal = b.Palette; for(int i = 0; i < 256; i++) pal.Entries[i] = Color.FromArgb( 255, i, i, i ); b.Palette = pal; } private System.Drawing.Color GetColorFromIndex(byte c) { return = b.Palette.Entries[c]; } } }

Blog moved !

For a long time I wanted to start using a better blog-engine, and have now finally upgraded to dasBlog, a great ASP.NET-based blogging engine. At the same time I moved it to a new URL, so from now on, you can find my blog at www.sharpgis.net

Atlas released with Go-Live license

Atlas has just been released with a Go-Live license, so we now can use it on our websites. Woohoo !
“Atlas” makes it possible to easily take advantage of AJAX techniques on the Web and enables you to create ASP.NET pages with a rich, responsive UI and server communication.

I see a lot of potential when combining this with a map viewer, like Microsofts Live Local.

I have tried implementing Atlas in my Ajax demo on the SharpMap website, and it works great ! It's almost too easy :-)
I used Atlas for autocompletion when searching for cities, and for updating the cities-in-view list below the map.

Delaunay Triangulation in .NET 2.0

Based on an article by Paul Bourke, I've created a small light-weight .NET 2.0 library to triangulate point data. The library includes a small Windows Forms example showing how the library works.

Using the generic point type 'Point<T>' you can triangulate points with any attribute added to it. For instance to triangulate points with a Z-height, create a list with double-types for Z like this:

  List<Point<double>> Vertices = new List<Point<double>>();
//Add vertices ...
//Do triangulation
List<Triangulator.Geometry.Triangle> tris = Triangulator.Delauney.Triangulate(Vertices);


Otherwise you can just derive from and extend the 'Geometry.Point' class to triangulate points with more methods and properties.

 

The Delauney triangulation doesn't handle duplicate points very well (that is multiple points whose X and Y properties are equal). Luckily we now have anonymous methods to set up a simple predicate to check if we already have a point in my list before I add it:

  Triangulator.Geometry.Point pNew = new Triangulator.Geometry.Point(234.4,782.1); //New point to add
if(!Vertices.Exists(delegate(Triangulator.Geometry.Point p) { return pNew.Equals2D(p); }))
Vertices.Add(pNew);

 

The library can be downloaded here including a small demo-app. Feel free to use it as you like.

SharpMap v0.9 release moving closer

A lot have been happening to SharpMap lately. Several new features are already available in the alpha release. Here are some of the features that now readily are available:

  • Gradient Theme now works on not only fill-color, but all numeric style properties, giving even greater flexibility for controlling font-sizes, pen and outline color and widths etc. based on a numeric attribute.
  • Custom Theme: Define your own method that determines the style of a feature based on geometry and feature attribute. Ex. "Make all polygons with an area>100 and whose name starts with "S" green with a blue outline.
  • Symbols can now be scaled.
  • Create a WMS server with only very few lines of code.
  • Labels can now follow lines, or be rotated based on an attribute.
  • Lots of bug-fixes and optimizations.
  • Use any OleDb datasource that holds X and Y columns for creating a point layer.

Diego Guidi - who is the developer of NetTopologySuite has also created a link between SharpMap and NTS. This means that you instantly get all the great functionality of NTS inside SharpMap, including on-the-fly transformation, buffering etc. You can apply any custom method that performs NTS operations on any SharpMap datasource and render the result with SharpMap. Get it at the download section.

Download at http://sharpmap.iter.dk

Using GDAL from C#

Recently Frank Warmerdam and Michael Paul posted some examples on calling GDAL from .NET. This is pretty cool, since this enables you to access a great set of tools that I for one have been accessing by generating batch-files in a .NET application and then running them using the shell-commands (not pretty but it worked :-)

Their posts used several links to pointers and didn't have automatical garbage collection. Since disposing objects is very important when calling unmanaged code, and .NET developers isn't that used to pointers, I decided to wrap some of the GDAL functionaly into a few classes with automatic disposer etc. It still needs some brushing off and to include some more GDAL functionality, but let me know what you think. You can download it here.