Showing posts with label esri rest specification. Show all posts
Showing posts with label esri rest specification. Show all posts

Creating a node.js GeometryService : part 3

Its been a while since I posted my first and second post on a GeometryService based on node.js and PostGIS.

Today I want to announce that I open sourced my work in progress. It is in a pre-alpha state and it can be found here. The only thing it can do at the moment is convert ESRI Json geometries from one coordinate system to another. This is done by converting the geometry to WKT, creating a sql statement for the re-projection and then converting the GeoJSON result back to ESRI Json.

A sample input url when you started the geometryservice is : http://127.0.0.1:3000/rest/services/Geometry/GeometryServer/project?inSR=4326&outSR=3819&geometries={%22geometryType%22:%22esriGeometryPoint%22,%22geometries%22:[{%22x%22:-117,%22y%22:34},%20{%22x%22:-115,%22y%22:25}]}. This returns

{"geometries":[{"x":-115.00495449693217,"y":24.99533030379053},{"x":-117.00524224953409,"y":33.99507554185914}]}
as a result. The intermediate sql statement looks like this :
SELECT st_asgeojson(st_transform(st_geomfromtext(geometries.geometry, '4326'), '3819')) g FROM  (SELECT ST_ASTEXT('POINT(-117 34)') geometry UNION SELECT ST_ASTEXT('POINT(-115 25)')) geometries
.

Most of the logic of the code happens in geometryService.js. The conversion from and to ESRI Json happens in datatransformer.js. As stated in my previous post I also created some tests for the functions in the datatransformer.js. There is still a lot to do, especially everything related to error handling and logging and off course the implementation of the other geometry operations.

The main dependency is a runnning instance of PostgreSQL with PostGIS. As the connection string is not configurable yet you'll have to change it in the executeSQL function in geometryService.js.

The code can be found on following url https://bitbucket.org/gissolved/node_geometryservice and it is licensed under the MIT license.

If you like this then consider buying me a book or a license for Sublime Text 2.

Visualizing Geometries

In this post I'm going to give some more information about the GeometryVisualizer that I created recently.

The goal of the GeometryVisualizer is to be able to quickly visualize a GeoJson, WKT or ESRI JSON geometry. I created this to be able to quickly see how some ESRI JSON and WKT geometries looked like and visually verify some geometry conversion code I'm writing for my GeometryService in node.js. You can also use this to visualize the WKT or GeoJSON output of a PostGIS geometry query.

The ESRI JSON geometries are rendered with the ArcGIS JavaScript API. The WKT and GeoJSON geometries are rendered with OpenLayers.

Some code

In this section I'll show some parts of the code that made this web application possible and compare the ArcGIS JavaScript API with the OpenLayers API. First we'll take a look at the initialization of the map. The most notable facts about the initialization are that we need to add a layer before we are able to add graphics and that disabling the zoomwheel is less straight forward with the OpenLayers API.

ESRI JavaScript API map initialization:

var map = new esri.Map("esriJsonMap");
map.disableScrollWheelZoom();

// add a layer and directly hide it because otherwise we can't create graphics
var basemapURL= "http://server.arcgisonline.com/ArcGIS/rest/services/World_Street_Map/MapServer"

var basemap = new esri.layers.ArcGISDynamicMapServiceLayer(basemapURL);
map.addLayer(basemap);
basemap.visible = false;

OpenLayers map initialization:

var options = {
  maxExtent: new OpenLayers.Bounds(-1000000000, -1000000000, 1000000000, 1000000000),
}
var map = new OpenLayers.Map('openLayersMap', options);

var controls = map.getControlsByClass('OpenLayers.Control.Navigation'); 
for(var i = 0; i<controls.length; ++i){ 
    controls[i].disableZoomWheel(); 
}

var layer = new OpenLayers.Layer.ArcGIS93Rest( "ArcGIS World Street Map", 
                "http://server.arcgisonline.com/ArcGIS/rest/services/World_Street_Map/MapServer",
                {layers: '0'} );
map.addLayer(layer);
layer.setVisibility(false);

To be able to render an ESRI JSON geometry string with the ArcGIS JavaScript API you have to do some specific steps :

  • Clear the graphics
  • Parse the geometry string representation
  • Create a graphic and set its symbol according to the geometry type.
  • Update the extent of the map
Not that for updating the extent, I had to access an undocumented property of the graphics object and for point graphics the extent had to be enlarged.

function addGeometry(geometryText) {
    
    if(map.graphics === null) return; // map not loaded yet or something else is wrong
            
    map.graphics.clear();

    var geometryJson = JSON.parse(geometryText);
    var graphic = new esri.Graphic(geometryJson);

    if (!graphic.symbol){
      switch(graphic.geometry.type){
        case "point":
        case "multipoint":
          graphic.setSymbol(new esri.symbol.SimpleMarkerSymbol());
          break;
        case "polyline":
          graphic.setSymbol(new esri.symbol.SimpleLineSymbol());
          break;
        default:
          graphic.setSymbol(new esri.symbol.SimpleFillSymbol());
          break;
      }
    }

    map.graphics.add(graphic);
    // use undocumented object
    var ext = graphic._extent;
    // if point expand the extent
    if(ext.getWidth() < 0.00000001 || ext.getHeight() < 0.00000001){
      var factor = 1;
      ext.update(ext.xmin - factor, ext.ymin - factor, ext.xmax + factor, ext.ymax + factor, ext.spatialReference);
    }

    map.setExtent(ext.expand(2));
}

The process of adding WKT and GeoJSON geometries was very similar.

  • Remove all features
  • Parse the geometry string representation with a predefined formatter (OpenLayers.Format.WKT or OpenLayers.Format.GeoJSON)
  • Update the extent of the map.
Note that I didn't had to specify the symbology and zooming to the extent of the feature was easier. The WKT parsing was very straight forward. For the GeoJSON parsing I had to add two things. First I had to ensure that the parsed JSON had a property "type" with as value "Feature" and the parsed object returned a feature collection with one feature instead of directly returning the feature.

function addGeometry(geometryInput, parser, type){
  vectorLayer.removeAllFeatures();
  var feature = parser.read(geometryInput);
  vectorLayer.addFeatures(feature);
  if(dojo.isArray(feature)){
    feature = feature[0];
  }
  var bounds = feature.geometry.getBounds();
  bounds = bounds.scale(1.1);
  map.zoomToExtent(bounds);  
}

function addWkt(geometryText) {
  var wktFormat = new OpenLayers.Format.WKT();
  addGeometry(geometryText, wktFormat);
}

function addGeoJson(geometryText) {
  var geoJsonFormat = new OpenLayers.Format.GeoJSON();
  var geometry = JSON.parse(geometryText);
  geometry["type"] = "Feature";
  addGeometry(geometry, geoJsonFormat);
}

This are the most interesting parts of the code. The full source code is in the GeometryVisualizer.

Any questions/remarks/improvements ? Let me know !

Creating a node.js GeometryService : part 2

This is my second post on what I learned from creating an implementation of the geometryservice specification in node.js. The first post can be found here.

By default node.js doesn't reload your files when they have changed but while developing this can be very handy. The most easy to use tool for node.js on windows that monitors your files for changes, with an easy way to set the node.js debugging flag, is nodemonw. You can download nodemonw at https://github.com/cenanozen/nodemonw. Once you've downloaded the executable I suggest to copy it to your %Appdata%\npm directory or another directory thats in your path. To start nodemonw I now run the following command nodemonw --debug index.js. In the next section it will become clear why I added the --debug flag.

To be able to debug my node.js application I installed node-inspector (npm install -g node-inspector). As you can read in the readme of node-inspector it is really easy to get started with node-inspector. You just have to start node-inspector and then open http://127.0.0.1:8080/debug?port=5858 in your favorite WebKit based browser. On Wikipedia I found this list of WebKit based browsers. The most well known ones for Windows are Google Chrome and Safari. A screencast on node-inspector can be found here. There is also a node-inspector playlist on YouTube.

To be able to test parts of my GeometryService I used vows (npm install -g vows). Vows is an asynchronous behavior driven development framework. More info about it can be found on http://vowsjs.org/. I am now going to show a small part of the code from my GeometryService and some tests I wrote for this code. My directory structure for the code I'll show looks like this: lib/
-- datatransformer.js
test/
-- datatransformers.test.js
In datatransformer.js I started a function to convert an ESRI geometry JSON object to wkt based on its geometry type.

exports.esriGeoJsonToWKT = function esriGeoJsonToWKT (geometryType, geometry) {
  if(geometryType === "esriGeometryPoint") {
    return "POINT(" + geometry.x + " " + geometry.y + ")";
  }
}

And the content of datatransformer.test.js is:

var dt = require("../lib/datatransformer");
var vows = require('vows');
var assert = require('assert');

vows.describe('Esri geometry JSON to WKT').addBatch({
  'when converting esriGeometryPoint {"x":-117,"y":34}':{
    topic: function(){ 
      var geomType = 'esriGeometryPoint';
      var p = JSON.parse('{"x":-117,"y":34}');
      return dt.esriGeoJsonToWKT(geomType, p);
    },
    'we get "POINT(-117 34)"': function(topic){
      assert.equal(topic, "POINT(-117 34)");
    }
  },
  'but when converting esriGeometryPoint {"x":-117.01,"y":34.02}':{
    topic: function(){ 
      var geomType = 'esriGeometryPoint';
      var p = JSON.parse('{"x":-117.01,"y":34.02}');
      return dt.esriGeoJsonToWKT(geomType, p);
    },
    'we get "POINT(-117.01 34.02)"': function(topic){
      assert.equal(topic, "POINT(-117.01 34.02)");
    }
  }
}).exportTo(module);

As you can see I created two tests for converting point geometries from ESRI JSON to wkt. One for integer coordinates and one for decimal coordinates. The easiest way to run this tests with vows on windows was opening a commandline in the root directory of my node.js project and type vows --spec. This will run the tests it finds in the test and spec directories of your project. My output looks like this :

? Esri geometry JSON to WKT

when converting esriGeometryPoint {"x":-117,"y":34}
V we get "POINT(-117 "4)"
but when converting esriGeometryPoint {"x":-117.01,"y":34.02}
V we get "POINT(-117.01 34.02)"

V OK » 2 honored (0.007s)

On a side note if you ever encounter that node can't find any of your globally installed packages then it might help to add a new User Variable called NODE_PATH with as value %AppData%\npm\node_modules. That was it for this post more posts on this project will follow.

As I announced in this post, I open sourced the code of this project. It can be found in this bitbucket repository.

Creating a node.js GeometryService : part 1

Ever since I encountered the geoservices-rest-specification (pdf) I've been thinking about creating my own implementation. I also wanted wanted to try out node.js so I combined both ideas and started implementing a GeometryService in node.js. If you've never heard of node.js, this is the short introduction from the homepage of node.js:
Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices. The GeometryService is a web service that contains GIS related utility methods like project, intersect, buffer,... A sample server can be found here http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Geometry/GeometryServer.

I first downloaded and installed the windows installer of node.js version 0.65. After a reboot all locations where added to my path so that I can now start a new cmd window and type node. Note that when you install packages globally with the node package manager called npm (npm install <package> -g) npm (the node package manager) then the packages are installed in Windows 7 under %AppData%\npm (C:\Users\<username>\AppData\Roaming\npm). The full list of packages is located here search.npmjs.org.

To get started learning node I first read the node.js tutorial at nodebeginner.com. By following this tutorial I got a great insight in how to create your own webserver. At this moment I'm continuing to build my webservice based on this code but I plan to use some framework like express or journey in a later stage. This is not very urgent because I'll first focus on the JSON output of the GeometryService and thus won't need to output any html.

For my first version of the GeometryService I decided to use PostGIS as my geometry processor. I know this introduces an extra overhead. But I think this is the easiest way to get something up and running in a short period of time. If you don't want to bother with installing PostgreSQL and PostGIS I suggest you to download the Community edition of the Open Geo Suite. To be able to connect to the database I installed node-postgres with following command line: npm install pg -g. If you're on windows and get build errors you might try to add a file called true.cmd to the directory where node.exe resides (e.g. C:\Program Files (x86)\nodejs) with as content exit 0. Below is a short snippet on how I connect to the database. Make sure to replace all placeholders in the connection string.

var pg = require("pg");

function executeSQL(sql, parameters, resultCallback){
  var connectionString = "pg://username:password@host:port/databasename";
  pg.connect(connectionString, function(err, client) {
    client.query(sql,parameters, function(err, result) {
      // TODO add error handling
      console.log(result);
      resultCallback(result);
    });
  });
}

That was it for today. In the next part of the series I'll write about debugging node.js and about my progress on the GeometryService.

As I announced in this post, I open sourced the code of this project. It and can be found in this bitbucket repository.