Posts Tagged ‘shp’

The Geotaggers’ World Atlas

Tuesday, May 25th, 2010

4622376356_d0e78bd1c1

[Editor’s note: Similar to Flickr’s SHP api, but discrete to groups of photos rather than places (cities). Data visualization sorts all the geotagged pictures by photographer and date/time, and for each two adjacent pictures that are reasonably close together in place and time, drawing a line between them. The different colors represent different modes of transportation: Black is walking (less than 7mph), Red is bicycling or equivalent speed (less than 19mph), Blue is motor vehicles on normal roads (less than 43mph); Green is freeways or rapid transit. SF version.]

Republished from Eric Fischer’s Flickr site.

(above) Paris. The cluster to the northwest of the central city is La Defense. The cluster further to the southwest is Versailles.

Continue to Flickr for all 50 cities . . .

Review: Geocart 3 (Kelso)

Friday, May 14th, 2010

geocart3icon

Once a required computer application in many cartography shops in the 1990s, Geocart has come back with a vengeance with Mapthematic’s 3.0 release (Mac and now Windows).

“If map projections are your problem, Geocart is your solution”

While most GIS and remote sensing map software support a couple dozen obligatory projections, Geocart supports over 175 general case projections. Map projections are mathematical formulas for converting the earth’s round shape to a flat surface and their “parameters” can be adjusted to form thousands of specific projections. For comparison, ArcGIS, the popular commercial geographic information system software from E.S.R.I. supports 1/3 as many projections; MaPublisher from Avenza supports 1/2 as many as Geocart.

The program’s author, daan Strebe, is a leading authority in this specialized subject and the new version incorporates corrections to many standard formula resulting in near loss-less projections. Unlike other software packages, Geocart can transform any projection to another projection (full forward and inverse transformation support for all projections). Other map applications can damage data when it is transformed. Furthermore, Geocart 3 introduces a new rendering mode using PixSlice technology to create a sharper, more detailed raster images (examples after the jump). This works both for resizing images and when transforming from one projection to another (reprojecting).

The application manual includes a handy decision tree to assist in what projection to use depending on the map’s topic and geographic coverage. The application includes innovative advanced tools  to visualize the distortion inherent in each projection (sample image).

Pricing: For lapsed users, upgrade pricing is available for $500 with new professional licenses running $860, discount for multiple purchases. Steeply discounted non-commercial and student licenses are available. Price includes map databases (36GB with the pro version!) and, importantly, the new version imports shapefiles, the defacto geodata format.

Full review continued below . . .

screen-shot-2010-05-12-at-111830-pm

Installation

I tested Geocart using the free, month-long trial (note the watermarks in the screenshots). Download and installation (once for the application, again for the default databases) went quickly but you will need an administrator account to accomplish the install. When the package downloads, it is labeled with your operating system type rather than “Geocart” so in my case I looked for “Mac OS 10.5/10.6″ in my downloads.

The app and included databases each weigh in about 150 mb for 300 Mb of disk space. Rather than collecting associated database files in the Applications folder (Program Files on Windows), they are installed in Library > Application Support > Mapthematics > Databases. If you want quick “template” access to frequently used data, it should be added in that location. The “add recent databases” command partly makes up for this.

Setting up a map document

To start mapping, go to File > New. Then go to Map > New. Multiple maps can be stored in a single Geocart document, each having their own projection parameters and database content.

When making a map, the first step is to determine how large the map dimensions will be and how much geography it will show. The relationship between the two is called map scale. Some databases, like Natural Earth, are set up based on map scales. Using the right database will result in prettier maps that are generalized appropriately (the linework doesn’t look too detailed or too coarse) and smaller files that are easier to work with.

Geocart also includes a useful linework simplification routine when your data is complex and needs to be simplified. This toggle is on by default and is accessed under Map > Generalize vectors. Toggle it on and off to compare the resulting resulting lines, your mileage will vary by map scale, even with the same source database.

Tip: The application takes map scale seriously and includes a tool to calibrate your system under Preferences > Display. This calibration functionality is absent to most other mapping packages.

To add data to the map

Each new map starts with “Stylized World Topo 5400×2700″ raster image in layered with a vector grid (Map > Graticule) in sinusoidal projection. With the map selected, go to Map > Databases. I was able to easily add in shapefiles from Natural Earth, some of which are included in the default databases. If you have existing Geocart 2 format databases, those will import directly, including typesetting databases.

Tip: To modify which databases load for each new map, go to Preferences > New Map Databases. I set mine to use Natural Earth country boundaries but removed the default image database.

Have a scanned map without a projection?

Geocart will help you figure it out. Add the map with File > Place image. (Vectors are not supported at this time). Then align with a map with a vector map database. Adjust the settings of the map until it matches. Then choose File > Export Database. Load the database back into a Geocart map and start projecting.

I was also able to add several map images and quickly georeference them and then deproject to geographic (platte carrée) or into another projection. One was a simple map of the ash plume in Europe in Mercator. The other was a complicated world wall map from National Geographic in Winkle Tripel (examples below).

Tip: When georeferencing an image, maximize both the map and the placed image to fit the window (Map > Scale to Window). Then adjust your Geocart map to use the same boundaries as the placed map image (make an educated guess). Then cycle thru the projections until the vector lines (graticule and country boundaries, etc) begin to match. Mercator and Robinson are common for world maps, a conic like Albers or Lambert is common for country and state maps. Then adjust the projection parameters and fine tune the boundaries and nominal scale and map resolution till everything fits exactly. Finally, export the placed image to database format.

Note: For raster maps that are georeferenced, the exported database file remains in the native projection of the image (it it not transformed to geographic). This does not affect your ability to reproject the image, however

Choose a projection

The familiar icons by projection class are still found in the main menu bar (see screenshot above). With a map selected on the document, choose a different projection (some are even listed in cyrilic and arabic!) and watch the map update in real time.

If you want assistance in choosing a projection (who can remember all their quirks!?), check out Projection > Change Projection. A dialog with the same listing comes up but with descriptions, history, preview maps, and distortion information. Gain insight with the programmer’s unique and comprehensive expert knowledge will help guide your projection choice. While the map is projecting, a progress wheel with a rough remaining time will show in the upper left corner. Advanced datum support and transformation are provided.

Tip: The manual includes a full decision tree for choosing a projection. This is one of the best features of Geocart.

I love interrupted projections like the Goode homolosine and making one in Geocart is a cinch. Simply choose the Goode from the Pseudocylindric menu (oval icon on left) and then chose Projection > Interruptions > Goode Continental. While you’re getting the projection parameters, map size and resolution right, keep the rendering quality at draft (Map > Draft). When the settings are right, change that to Map > Final Quality for more precise results.

All databases in Geocart are geographic with live, on-the-fly transformations into your map’s specified projecting (see exception above for georeferenced images). I added in coastlines, rivers, lakes, country boundaries, US state boundaries into my test vector map. Even on my slowest, older laptop, rendering was responsive for basic usage creating vector world, regional, and country maps.

Tip: If you somehow end up with a strange looking map (off center, etc), choose Projection > Reset Projection and the current projection parameters will revert to defaults

Tip: When using a conic projection like Albers or Lambert, make sure the Projection > Projection Center is set to Latitudinal 0°N.

Geocart 3.0 is a world unto itself, however. While it does import raw data in shapefile format (YES!), it does not currently import or export PRJ files, part of the SHP file specification, the defacto geo data storage and exchange format. Imported SHP files must be in geographic projection. This makes sense in part as Geocart supports many more projections and parameters than most other mapping software packages (3 times as many as ArcMap, 6 times as many as Natural Scene Designer, 2 times as many as MaPublisher and Geographic Imager). Geocart also sometimes uses slightly different formulas for the same projections as the other applications (the author claims Geocart’s implementations fix errors in common formulas, which is probably the case based on my experience with the literature and web source code snippits).

But for the projections that are shared in common, it would be useful to offer PRJ support (including transformations out of the error prone versions), and shapefile export of databases after their coordinates have been transformed (and GeoTIFF for raster).

More importantly, PRJ files offer a quick load of common projection parameters. So if I’m in California I can load up the Albers with the standardized parameters so my data will interoperate with other cartographers working in that area, and they take some of the guess work out of choosing a map projection. Both ArcMap and MaPublisher are better then Geocart in this regard. MapTiler thru Proj4 is the worst. Azimuth (r.i.p.) is the best at setting appropriate projection and parameter for the visible, selected geography.

Tip: If you do have a PRJ file, open it in a text editor and manually copy over the parameters to Geocart. They use a “well known text” structure that is human readable.

Legend editor (stylizing your map)

Geocart includes basic legend editor for setting line and fill styles, appropriate for general reference mapping. Geocart is a general projection tool, not for making thematic maps. The layer sorting of individual databases is adjustable in the Map > Databases dialog.

Tip: Consistent styles can be shared between map projects by going to Preferences > New Map Line Styles.

Testing the limits

Don’t want to plot the entire world? Use Map > Boundaries to set a crop (and speed up map rendering). This window is quite amazing and has both 2d and 3d views with actual spherical trapezoids! Boundaries can be set relative to the projection center and can be a circular diameter, spherical trapezoid, or irregularly shaped “custom” boundary. To remove the boundaries, change the setting back to “Unconstrained”.

Quibble: When adjusting boundaries in most conic projection, your standard parallels should also change. A prompt should be provided in this use case to automatically adjust those to your new view. In the special case of setting standard parallels in Projection > Parameters, it would be helpful if Geocart showed these on a map like in the Projection Center dialog.

Quibble: The draw on map interface in Boundaries needs a little more work for modifying the existing settings. Other apps, like Geographic Imager, allow me to drag the edges of a drawn boundary while in Geocart I have to start over (or use the number fields). It’s also a little wonky when dragging exactly horizontal or vertical (a full latitude or longitude strip). There are also no ticker buttons to increment the parameter values, either. Once you have this set, though, you’re golden so it’s a minor inconvenience.

Next: Rendering quality and speed . . .

nsd5pro_geocart3_rendering_b

Above: Brand X on the left. Geocart at right. Examine the letter forms (U in United Kingdom, N in London, all in Paris, the Ca in Cariff). The Geocart render results in sharper, crisper letter forms with less “pixel burrs”. The demo water mark not with standing.

Rendering quality

The key concept is Geocart creates an optimized map on each render. The original data resolution is stored in the document, but what draws on the screen is determined by the map size and resolution. Set that in Map > Set size and resolution. Once adjusted, the map will fill that space in the window. You can zoom in and out with the normal Cmd-+ and – keyboard shortcuts and the zoom with update in the window title.

When Geocart is set to render in Final mode, its output results in better output than applications that use only nearest neighbor or bicubic interpolation. In the example above, looking at the letter edges on London, the Geocart version is crisper and smoother. This also comes into play at the edges of a world map where the projection distortion is more extreme and is especially important with projecting raster data.

For my heavy-use scenario, I put Geocart up against the latest National Geographic world map

The map is in Winkel Tripel projection. I rasterized the PDF (took about 1 hour with Photoshop on my old laptop) and then loaded the image into Geocart and georeferenced it and saved it out as a database (78 mb, seems small), see section on Adding map data above. I then reprojected it Goode homolosine in Geocart. I also ripped out a platte carrée from Geocart and projected that into Goode in Geographic Imager, Natural Scene Designer, MapTiler (Proj4), and ArcMap.

The final projected Goode image dimensions was 22,700 pixels by 9,910 at 675 mb in TIFF image format. Enough detail to print back out as a wall map or tile for a web map service.

Geocart is built for speed and will utilize all processors, including multicore

Paul Messmer’s under the hood improvements allow the application to make 100% use of all processor cores. I was still able to use other applications while Geocart processed data, however. One side effect of supporting multiple cores is rendering occurs per core in real time, see screenshot below. Geocart also plays nice on idle.

sixteencores

I tested Geocart on 3 different machines, all Intel Macs running 10.5 or 10.6 from an older laptop to a new desktop towers. Application task completion speed increased directly proportional to the number of cores available.

Fun fact: Geocart uses a Hilbert curve to render the map when utilizing multiple cores to keep memory accesses as local as possible in order to make the best use of the processor caches. This results in seperate render traces on the screen, see image below.

quad_rendering_b

At best “final” settings, the huge map in Goode homolosine projection described above took 20 min on the 16 core Mac Pro (2 x 2.93 quad core GHz quad-core Intel Xeon with 8 gb of RAM) but 1 hour 20 minutes on an older 4 core Mac Pro with the same RAM configuration. The draft render took significantly less time and was comparable in time and quality to Natural Scene Designer, Geographic Imager, ArcMap, and MapTiler (Proj4).

Because Geocart is always planning for the most general case with the most advanced options, this can slow down it’s rendering compared to other applications (most noticeable when in Final rendering mode). Future versions might speed up if special functions were added for the standard parameter cases. But by the time the programmer did that, the speed difference might be equivalent to increases in hardware speed and cores, so this doesn’t worry me much.

Compared the competition

Geographic Imager ($699 for Adobe Photoshop plugin, add $699 if you don’t already own Photoshop) did not support the interrupted form of the projection and produced confetti until I tweeked the settings. To project vectors, you’d need MaPublisher, a vector plugin from Avenza for Adobe Illustrator, will set you back $1399 plus cost for Illustrator. ArcMap (thousands of dollars) required a RGB (not indexed) version of the geographic TIFF version but insisted on reprojecting into grayscale. Natural Scene Designer ($160) produced the most comparable raster results and ease of use, but at less quality (though faster). It should be noted the Pro version of Natural Scene Designer 5 also supports multiple cores and limited vector shapefile support (raster rendering only), plus better handling of GeoTIFF with TFW export. MapTiler, Mapnik, and other open source GIS options are free but you’ll spend time setting them up and learning their make-by-and-for-programmer quirks.

Visualizing Distortion

Geocart is a good teaching tool as well when using the distortion visualizations and mouseOver readouts (available under Window > Information). The pertinent readouts are Angular deformation, Areal inflation, Scaler distortion, and Scale factor range.

Note: Geocart quit on me once when I tried to use Map > Copy Attributes while visualizing distortion with a very large selected map, but I was not able to replicate the error or any crash in subsequent testing sessions. In general I’ve found the program to be very responsive and to not hang up, even when rendering extremely large maps with multiple databases.

Quibble: The Information panel should display how long it took to render the selected map.

Exporting

On exporting out your final map, vector (PDF) and raster (TIFF, PSB “Photoshop”, and JPG) formats are available. On opening the map in Illustrator, each database layer is conveniently grouped, with clipped content. Geocart could take a page out of IndieMapper’s layered SVG approach where the file format would still be PDF but the groups would be named and even better yet actual PDF layers.

Quibbles: Geocart suffers from the same zealous masking and embedding as other apps. If no boundaries have been defined in Geocart, the clipping masks should not be included. Saving out as PDF will embed the raster databases into the file, like all other programs. On export of the raster formats, an option should be provided to NOT export the vector database layers. Another option should be provided to export each raster database layer to a separate file (or layered TIFF / PSB). Needs to export out a PRJ file for the raster and GeoTIFF with embedded registration, pixel size, and projection tags.

Note: If you’re looking for SHP export, you’ll be disappointed. Though that’s kind of missing the point of Geocart. See “Choose a projection” section above.

Final word

Geocart 3 is a solid release that will satisfy most of your reference mapping needs, especially if projection matters to you. If you liked Geocart 2, you’ll definitely enjoy working with version 3, and on the latest computer hardware it simply screams. The addition of direct shapefile import removes a barrier to geodata access, though more could be made of the PRJ files and DBF attributes. There are still some missing features when compared to version 2 and daan (the programmer) is interested in hearing from the cartography community which should added back. They also seem responsive to fixing some of the usability issues I’ve noted above.

But where are those Kelso Corners, I ask? Besides being a personal soapbox, my blog is named for the “corners” that form when a pseudocylindric or lenticular projection is extended to fill out it’s rectangular bounding box by repeating content that would otherwise only be found on the opposite edge of the map. They are righteously awesome, plus they satisfy non-carto designers  proclivity to design to a boxy grid. However, you can only find these “corners” on a few old print maps; I don’t know of a single digital app that creates them. I’ve staked naming rights ;)

Pros: Over 175 projections (best in industry), support for advanced projection parameters, loss-less reprojection, PixSlice technology for sharper, more detailed raster images. Runs on both Windows and Mac, with support for multiple core processors. Now imports shapefile vector map data. Large document support. Easy to use. Software programmer responsive to emails and forum posts.

Cons: No PRJ support. Does not export GeoTIFF, or world file created after georeferencing images. Does not include a SHP filter in file dialogs, and file dialogs do not remember last browsed directory. Should start with blank new document on launch. Linework generalization engine filters just by Douglas-Peucker in this version, not the smooth bezier curves found in Geocart 2 or the amazing generalization found at MapShaper.org. Rendering in PixSlice can significantly increase render times. No support for scripting/automation. No export back to SHP format (especially with DBF attributes), useful for thematic mapping in a secondary GIS application.

Simple shapefile drawing in ActionScript 3 (Cartogrammar)

Wednesday, July 29th, 2009

Shapefile + magic = map in Flash!

[Editor's note: Andy Woodruff explains how to use his quick and easy implementation of Edwin van Rijkom's AS3 classes for loading SHP files and their DBF attributes into Flash/Flex. This library DOES NOT PROJECT your shp files, you might consider doing that first.]

Recently I’ve heard two friends independently inquire about some sort of basic guide for loading and drawing a shapefile in Flash. The only real tutorial/example I can recall is here, dealing with Google Maps. But these guys are looking for something more bare-bones. Being a regular user of Edwin van Rijkom’s invaluable code libraries for reading shapefiles, and usually forgetting the process myself, I thought it would be a good idea to put together a very simple set of AS3 classes that load a shapefile and throw a map on screen.

So to get those jerks off my back, I wrote a little thing called ShpMap, which supplements van Rijkom’s classes by loading and drawing a shapefile. It’s nothing fancier than that. Sometimes all you need is to get your base map on screen. (Update: just to round it out a little more, I’ve added basic loading and parsing of a shapefile’s accompanying DBF file, which contains attribute data. This also uses classes by van Rijkom.)

I hope that this class (and the several associated classes) can both be directly usable for some projects and serve as a basic guide to using van Rijkom’s classes to load shapefiles.

Dig it:

  • An example that loads and displays a US states shapefile (and then puts a square on my house and colors the state of Wisconsin green). View the source code here.
  • Download the source code. (My classes plus van Rijkom’s, as well as a demo US States shapefile.)

Shapefiles, Actionscript 3.0, and Google Maps (Box Shaped World)

Monday, June 8th, 2009

[Editor's note: Tutorial for using the vanrikom classes to read in shapefiles in Flash and Flex AS3 and display them on a Google Maps mashup.]

Republished from the Box Shaped World blog. 28 April 2009.

I’m working an Adobe AIR application and I wanted to be able to have the user select a shapefile, and then parse it to create a KML file.  I didn’t want to have the user be responsible for creating a KML file.  I thought I might try and crack the shapefile enigma since it is a well documented format, but that would have taken time and I suddenly realized I’m not actually a developer :).  Instead, I found this set of Actionscript Classes to parse a shapefile in Flash.  Unfortunately, I didn’t find a very good tutorial on how to work with the classes.  The example is a little confusing (at least for me) and also uses a far file.  I’d never heard of far compressed files.  So I took the classes and created my own parser.  I thought I would post a tutorial on how to use these shapefile classes in conjunction with AIR and the Google Maps API for flash.  This technique would work with flex as well, I just didn’t want to have to write the code to upload a file.  I presume a few things with this.  The shapefile you are using for this should already have a geographic projection (e.g. latitude and longitude Geographic NAD 83).  In order to use the Google Maps API with AIR, you need a URL with a key associated with it.  Below are two zipfiles available for download.  The testfile.zip is the shapefile I was using.  The vanrikom.zip is the downloaded actionscript classes from the Google Code repository.  I had trouble downloading the using an svn so I did it manually.  I’ll save you the time by making it available here…unless the original author asks me to remove them.  There are parts that I find confusing with the way the reader was set up.  For some reason polyline inherits from polygon.  Intuitively to me it should be the other way around…but like I said, I’m not a developer.

This was all done using FlashDevelop and the Flex SDK 3.  There are 4 custom classes in addition to the mxml file.  Each are shown here.

Continue reading at Box Shaped World . . .

GISLook & GISMeta – Preview GIS Data Before Opening It

Thursday, June 26th, 2008

 gis quick look plugin bernie

Using Quick Look in Mac OS X 10.5, you can view the contents of a file without even opening it.

I am proud to announce today that Mr. Genius Bernhard Jenny of the Swiss Institute of Cartography at ETH Zurich has created Quick Look and Spotlight plugins for GIS data for Leopard. Download here.

Use GISLook to browse and preview GIS data in Finder window thumbnails, including Cover Flow and Quick Look windows. Use GISMeta to view the size of GIS raster grid files.

If you haven’t upgraded to 10.5 yet, this is good reason. If you don’t own a Mac, get one and run ArcMap  via Parallels Desktop.

This software is donationware. You can freely use them at no charge. If you use them regularly, it is suggested that you pay a donation of €5 or a more suitable amount.

Supported file types include: 

Vector data

  • ESRI Shape (.shp)
  • E00 ArcInfo Interchange (.e00)
  • ArcInfo Coverage (.adf)

Raster grids, such as digital elevation models or land cover data with a single band

  • BIL (.bil), BIP (.bip) and BSQ (.bsq) with .hdr file
  • ESRI ASCII Grid (.asc)
  • ESRI Binary Grid (.flt with .hdr file)
  • PGM (.pgm)
  • SRTM (.hgt and .dem)
  • Surfer Grid (.grd)
  • USGS DEM (.dem)