Posts Tagged ‘topology’

Review: Geocart 3 (Kelso)

Friday, May 14th, 2010

geocart3icon

Once a required computer application in many cartography shops in the 1990s, Geocart has come back with a vengeance with Mapthematic’s 3.0 release (Mac and now Windows).

“If map projections are your problem, Geocart is your solution”

While most GIS and remote sensing map software support a couple dozen obligatory projections, Geocart supports over 175 general case projections. Map projections are mathematical formulas for converting the earth’s round shape to a flat surface and their “parameters” can be adjusted to form thousands of specific projections. For comparison, ArcGIS, the popular commercial geographic information system software from E.S.R.I. supports 1/3 as many projections; MaPublisher from Avenza supports 1/2 as many as Geocart.

The program’s author, daan Strebe, is a leading authority in this specialized subject and the new version incorporates corrections to many standard formula resulting in near loss-less projections. Unlike other software packages, Geocart can transform any projection to another projection (full forward and inverse transformation support for all projections). Other map applications can damage data when it is transformed. Furthermore, Geocart 3 introduces a new rendering mode using PixSlice technology to create a sharper, more detailed raster images (examples after the jump). This works both for resizing images and when transforming from one projection to another (reprojecting).

The application manual includes a handy decision tree to assist in what projection to use depending on the map’s topic and geographic coverage. The application includes innovative advanced tools  to visualize the distortion inherent in each projection (sample image).

Pricing: For lapsed users, upgrade pricing is available for $500 with new professional licenses running $860, discount for multiple purchases. Steeply discounted non-commercial and student licenses are available. Price includes map databases (36GB with the pro version!) and, importantly, the new version imports shapefiles, the defacto geodata format.

Full review continued below . . .

screen-shot-2010-05-12-at-111830-pm

Installation

I tested Geocart using the free, month-long trial (note the watermarks in the screenshots). Download and installation (once for the application, again for the default databases) went quickly but you will need an administrator account to accomplish the install. When the package downloads, it is labeled with your operating system type rather than “Geocart” so in my case I looked for “Mac OS 10.5/10.6″ in my downloads.

The app and included databases each weigh in about 150 mb for 300 Mb of disk space. Rather than collecting associated database files in the Applications folder (Program Files on Windows), they are installed in Library > Application Support > Mapthematics > Databases. If you want quick “template” access to frequently used data, it should be added in that location. The “add recent databases” command partly makes up for this.

Setting up a map document

To start mapping, go to File > New. Then go to Map > New. Multiple maps can be stored in a single Geocart document, each having their own projection parameters and database content.

When making a map, the first step is to determine how large the map dimensions will be and how much geography it will show. The relationship between the two is called map scale. Some databases, like Natural Earth, are set up based on map scales. Using the right database will result in prettier maps that are generalized appropriately (the linework doesn’t look too detailed or too coarse) and smaller files that are easier to work with.

Geocart also includes a useful linework simplification routine when your data is complex and needs to be simplified. This toggle is on by default and is accessed under Map > Generalize vectors. Toggle it on and off to compare the resulting resulting lines, your mileage will vary by map scale, even with the same source database.

Tip: The application takes map scale seriously and includes a tool to calibrate your system under Preferences > Display. This calibration functionality is absent to most other mapping packages.

To add data to the map

Each new map starts with “Stylized World Topo 5400×2700″ raster image in layered with a vector grid (Map > Graticule) in sinusoidal projection. With the map selected, go to Map > Databases. I was able to easily add in shapefiles from Natural Earth, some of which are included in the default databases. If you have existing Geocart 2 format databases, those will import directly, including typesetting databases.

Tip: To modify which databases load for each new map, go to Preferences > New Map Databases. I set mine to use Natural Earth country boundaries but removed the default image database.

Have a scanned map without a projection?

Geocart will help you figure it out. Add the map with File > Place image. (Vectors are not supported at this time). Then align with a map with a vector map database. Adjust the settings of the map until it matches. Then choose File > Export Database. Load the database back into a Geocart map and start projecting.

I was also able to add several map images and quickly georeference them and then deproject to geographic (platte carrée) or into another projection. One was a simple map of the ash plume in Europe in Mercator. The other was a complicated world wall map from National Geographic in Winkle Tripel (examples below).

Tip: When georeferencing an image, maximize both the map and the placed image to fit the window (Map > Scale to Window). Then adjust your Geocart map to use the same boundaries as the placed map image (make an educated guess). Then cycle thru the projections until the vector lines (graticule and country boundaries, etc) begin to match. Mercator and Robinson are common for world maps, a conic like Albers or Lambert is common for country and state maps. Then adjust the projection parameters and fine tune the boundaries and nominal scale and map resolution till everything fits exactly. Finally, export the placed image to database format.

Note: For raster maps that are georeferenced, the exported database file remains in the native projection of the image (it it not transformed to geographic). This does not affect your ability to reproject the image, however

Choose a projection

The familiar icons by projection class are still found in the main menu bar (see screenshot above). With a map selected on the document, choose a different projection (some are even listed in cyrilic and arabic!) and watch the map update in real time.

If you want assistance in choosing a projection (who can remember all their quirks!?), check out Projection > Change Projection. A dialog with the same listing comes up but with descriptions, history, preview maps, and distortion information. Gain insight with the programmer’s unique and comprehensive expert knowledge will help guide your projection choice. While the map is projecting, a progress wheel with a rough remaining time will show in the upper left corner. Advanced datum support and transformation are provided.

Tip: The manual includes a full decision tree for choosing a projection. This is one of the best features of Geocart.

I love interrupted projections like the Goode homolosine and making one in Geocart is a cinch. Simply choose the Goode from the Pseudocylindric menu (oval icon on left) and then chose Projection > Interruptions > Goode Continental. While you’re getting the projection parameters, map size and resolution right, keep the rendering quality at draft (Map > Draft). When the settings are right, change that to Map > Final Quality for more precise results.

All databases in Geocart are geographic with live, on-the-fly transformations into your map’s specified projecting (see exception above for georeferenced images). I added in coastlines, rivers, lakes, country boundaries, US state boundaries into my test vector map. Even on my slowest, older laptop, rendering was responsive for basic usage creating vector world, regional, and country maps.

Tip: If you somehow end up with a strange looking map (off center, etc), choose Projection > Reset Projection and the current projection parameters will revert to defaults

Tip: When using a conic projection like Albers or Lambert, make sure the Projection > Projection Center is set to Latitudinal 0°N.

Geocart 3.0 is a world unto itself, however. While it does import raw data in shapefile format (YES!), it does not currently import or export PRJ files, part of the SHP file specification, the defacto geo data storage and exchange format. Imported SHP files must be in geographic projection. This makes sense in part as Geocart supports many more projections and parameters than most other mapping software packages (3 times as many as ArcMap, 6 times as many as Natural Scene Designer, 2 times as many as MaPublisher and Geographic Imager). Geocart also sometimes uses slightly different formulas for the same projections as the other applications (the author claims Geocart’s implementations fix errors in common formulas, which is probably the case based on my experience with the literature and web source code snippits).

But for the projections that are shared in common, it would be useful to offer PRJ support (including transformations out of the error prone versions), and shapefile export of databases after their coordinates have been transformed (and GeoTIFF for raster).

More importantly, PRJ files offer a quick load of common projection parameters. So if I’m in California I can load up the Albers with the standardized parameters so my data will interoperate with other cartographers working in that area, and they take some of the guess work out of choosing a map projection. Both ArcMap and MaPublisher are better then Geocart in this regard. MapTiler thru Proj4 is the worst. Azimuth (r.i.p.) is the best at setting appropriate projection and parameter for the visible, selected geography.

Tip: If you do have a PRJ file, open it in a text editor and manually copy over the parameters to Geocart. They use a “well known text” structure that is human readable.

Legend editor (stylizing your map)

Geocart includes basic legend editor for setting line and fill styles, appropriate for general reference mapping. Geocart is a general projection tool, not for making thematic maps. The layer sorting of individual databases is adjustable in the Map > Databases dialog.

Tip: Consistent styles can be shared between map projects by going to Preferences > New Map Line Styles.

Testing the limits

Don’t want to plot the entire world? Use Map > Boundaries to set a crop (and speed up map rendering). This window is quite amazing and has both 2d and 3d views with actual spherical trapezoids! Boundaries can be set relative to the projection center and can be a circular diameter, spherical trapezoid, or irregularly shaped “custom” boundary. To remove the boundaries, change the setting back to “Unconstrained”.

Quibble: When adjusting boundaries in most conic projection, your standard parallels should also change. A prompt should be provided in this use case to automatically adjust those to your new view. In the special case of setting standard parallels in Projection > Parameters, it would be helpful if Geocart showed these on a map like in the Projection Center dialog.

Quibble: The draw on map interface in Boundaries needs a little more work for modifying the existing settings. Other apps, like Geographic Imager, allow me to drag the edges of a drawn boundary while in Geocart I have to start over (or use the number fields). It’s also a little wonky when dragging exactly horizontal or vertical (a full latitude or longitude strip). There are also no ticker buttons to increment the parameter values, either. Once you have this set, though, you’re golden so it’s a minor inconvenience.

Next: Rendering quality and speed . . .

nsd5pro_geocart3_rendering_b

Above: Brand X on the left. Geocart at right. Examine the letter forms (U in United Kingdom, N in London, all in Paris, the Ca in Cariff). The Geocart render results in sharper, crisper letter forms with less “pixel burrs”. The demo water mark not with standing.

Rendering quality

The key concept is Geocart creates an optimized map on each render. The original data resolution is stored in the document, but what draws on the screen is determined by the map size and resolution. Set that in Map > Set size and resolution. Once adjusted, the map will fill that space in the window. You can zoom in and out with the normal Cmd-+ and – keyboard shortcuts and the zoom with update in the window title.

When Geocart is set to render in Final mode, its output results in better output than applications that use only nearest neighbor or bicubic interpolation. In the example above, looking at the letter edges on London, the Geocart version is crisper and smoother. This also comes into play at the edges of a world map where the projection distortion is more extreme and is especially important with projecting raster data.

For my heavy-use scenario, I put Geocart up against the latest National Geographic world map

The map is in Winkel Tripel projection. I rasterized the PDF (took about 1 hour with Photoshop on my old laptop) and then loaded the image into Geocart and georeferenced it and saved it out as a database (78 mb, seems small), see section on Adding map data above. I then reprojected it Goode homolosine in Geocart. I also ripped out a platte carrée from Geocart and projected that into Goode in Geographic Imager, Natural Scene Designer, MapTiler (Proj4), and ArcMap.

The final projected Goode image dimensions was 22,700 pixels by 9,910 at 675 mb in TIFF image format. Enough detail to print back out as a wall map or tile for a web map service.

Geocart is built for speed and will utilize all processors, including multicore

Paul Messmer’s under the hood improvements allow the application to make 100% use of all processor cores. I was still able to use other applications while Geocart processed data, however. One side effect of supporting multiple cores is rendering occurs per core in real time, see screenshot below. Geocart also plays nice on idle.

sixteencores

I tested Geocart on 3 different machines, all Intel Macs running 10.5 or 10.6 from an older laptop to a new desktop towers. Application task completion speed increased directly proportional to the number of cores available.

Fun fact: Geocart uses a Hilbert curve to render the map when utilizing multiple cores to keep memory accesses as local as possible in order to make the best use of the processor caches. This results in seperate render traces on the screen, see image below.

quad_rendering_b

At best “final” settings, the huge map in Goode homolosine projection described above took 20 min on the 16 core Mac Pro (2 x 2.93 quad core GHz quad-core Intel Xeon with 8 gb of RAM) but 1 hour 20 minutes on an older 4 core Mac Pro with the same RAM configuration. The draft render took significantly less time and was comparable in time and quality to Natural Scene Designer, Geographic Imager, ArcMap, and MapTiler (Proj4).

Because Geocart is always planning for the most general case with the most advanced options, this can slow down it’s rendering compared to other applications (most noticeable when in Final rendering mode). Future versions might speed up if special functions were added for the standard parameter cases. But by the time the programmer did that, the speed difference might be equivalent to increases in hardware speed and cores, so this doesn’t worry me much.

Compared the competition

Geographic Imager ($699 for Adobe Photoshop plugin, add $699 if you don’t already own Photoshop) did not support the interrupted form of the projection and produced confetti until I tweeked the settings. To project vectors, you’d need MaPublisher, a vector plugin from Avenza for Adobe Illustrator, will set you back $1399 plus cost for Illustrator. ArcMap (thousands of dollars) required a RGB (not indexed) version of the geographic TIFF version but insisted on reprojecting into grayscale. Natural Scene Designer ($160) produced the most comparable raster results and ease of use, but at less quality (though faster). It should be noted the Pro version of Natural Scene Designer 5 also supports multiple cores and limited vector shapefile support (raster rendering only), plus better handling of GeoTIFF with TFW export. MapTiler, Mapnik, and other open source GIS options are free but you’ll spend time setting them up and learning their make-by-and-for-programmer quirks.

Visualizing Distortion

Geocart is a good teaching tool as well when using the distortion visualizations and mouseOver readouts (available under Window > Information). The pertinent readouts are Angular deformation, Areal inflation, Scaler distortion, and Scale factor range.

Note: Geocart quit on me once when I tried to use Map > Copy Attributes while visualizing distortion with a very large selected map, but I was not able to replicate the error or any crash in subsequent testing sessions. In general I’ve found the program to be very responsive and to not hang up, even when rendering extremely large maps with multiple databases.

Quibble: The Information panel should display how long it took to render the selected map.

Exporting

On exporting out your final map, vector (PDF) and raster (TIFF, PSB “Photoshop”, and JPG) formats are available. On opening the map in Illustrator, each database layer is conveniently grouped, with clipped content. Geocart could take a page out of IndieMapper’s layered SVG approach where the file format would still be PDF but the groups would be named and even better yet actual PDF layers.

Quibbles: Geocart suffers from the same zealous masking and embedding as other apps. If no boundaries have been defined in Geocart, the clipping masks should not be included. Saving out as PDF will embed the raster databases into the file, like all other programs. On export of the raster formats, an option should be provided to NOT export the vector database layers. Another option should be provided to export each raster database layer to a separate file (or layered TIFF / PSB). Needs to export out a PRJ file for the raster and GeoTIFF with embedded registration, pixel size, and projection tags.

Note: If you’re looking for SHP export, you’ll be disappointed. Though that’s kind of missing the point of Geocart. See “Choose a projection” section above.

Final word

Geocart 3 is a solid release that will satisfy most of your reference mapping needs, especially if projection matters to you. If you liked Geocart 2, you’ll definitely enjoy working with version 3, and on the latest computer hardware it simply screams. The addition of direct shapefile import removes a barrier to geodata access, though more could be made of the PRJ files and DBF attributes. There are still some missing features when compared to version 2 and daan (the programmer) is interested in hearing from the cartography community which should added back. They also seem responsive to fixing some of the usability issues I’ve noted above.

But where are those Kelso Corners, I ask? Besides being a personal soapbox, my blog is named for the “corners” that form when a pseudocylindric or lenticular projection is extended to fill out it’s rectangular bounding box by repeating content that would otherwise only be found on the opposite edge of the map. They are righteously awesome, plus they satisfy non-carto designers  proclivity to design to a boxy grid. However, you can only find these “corners” on a few old print maps; I don’t know of a single digital app that creates them. I’ve staked naming rights ;)

Pros: Over 175 projections (best in industry), support for advanced projection parameters, loss-less reprojection, PixSlice technology for sharper, more detailed raster images. Runs on both Windows and Mac, with support for multiple core processors. Now imports shapefile vector map data. Large document support. Easy to use. Software programmer responsive to emails and forum posts.

Cons: No PRJ support. Does not export GeoTIFF, or world file created after georeferencing images. Does not include a SHP filter in file dialogs, and file dialogs do not remember last browsed directory. Should start with blank new document on launch. Linework generalization engine filters just by Douglas-Peucker in this version, not the smooth bezier curves found in Geocart 2 or the amazing generalization found at MapShaper.org. Rendering in PixSlice can significantly increase render times. No support for scripting/automation. No export back to SHP format (especially with DBF attributes), useful for thematic mapping in a secondary GIS application.

What is GeoDesign and why is it important (ESRI + GeoInformatics)

Monday, March 15th, 2010

[Editor's note: Like mashups, but in ArcGIS and analytical without programming skills. Sounds like CommunityViz but is more generally the "pairing of design and GIS. It unites the art and creativity of design (planning) with the power and science of geospatial technology. As one, GeoDesign can produce more informed, data-based design options and decisions." This drive will introduce modeling, sketching, and feedback capabilities in ESRI's ArcGIS Desktop 10, set for release in the second quarter of 2010. Looks like it will rely more on GIS services (web apps and 2) and more validating of resulting feature topology by GIS techs. Recently concluded mini-conference on GeoDesign has streaming video clips. This article is also good. Thanks @geoparadigm and @gisuser.]

Republished from ESRI and GEO Informatics.

What is GeoDesign?
GeoDesign is a set of techniques and enabling technologies for planning built and natural environments in an integrated process, including project conceptualization, analysis, design specification, stakeholder participation and collaboration, design creation, simulation, and evaluation (among other stages). “GeoDesign is a design and planning method which tightly couples the creation of design proposals with impact simulations informed by geographic contexts.” [1] Nascent geodesign technology extends geographic information systems so that in addition to analyzing existing environments and geodata, users can synthesize new environments and modify geodata. Learn more about GeoDesign on Wikipedia.

Read more at ESRI ArcWatch . . .

Jack Dangermond on GeoDesign:
“In January [ESRI hosted] the first GeoDesign Summit. It will bring people from both the GIS and design fields together and have them share their work and get a conversation going. I’m not totally sure what the outcome is going to be, but I’m hoping a new profession or direction will emerge. I think we need this kind of mixing at this point to bring these two fields together; people who design the world with people who design the future. Today, geography lives very well in its world and designers live very well in their world, but there’s not this cross-mixing. I believe the outcome will be much enlightened ways to do development; ways that bring science into how we design things: cities, the environment, highways, everything that we do. Today we certainly see the need for this all the way from global warming to designing more livable and sustainable cities. We need more geographic thinking in the way we make decisions. GeoDesign is an attempt to try to do something about that.”

Read more at GEO Informatics . . .

What does it mean for GIS discipline:
“It is not so much that geodesign is new, but rather that technology has reached a point that allows artists to participate in the geodesign process – without becoming technologists.” (Kirk at GeoThought) It still requires good (accurate, precise) base maps and themes in GIS to enable smart decision making (geodesign) on the desktop and in the cloud (web apps). Instead GIS techs being puck jockeys, the planning folks will be able to use the GIS directly, or it’ll seem that way to them ;) I used to work somewhere where the boss had desktop design apps installed and he could comp out designs, but they still had to be rebuilt to production specs. My guess is the same will be true with GeoDesign for a good bit yet. Meanwhile, focus on core competencies.

Learn more at the ESRI Developers User Conference later this month . . .

NodeXL: Network Visualizations in Excel (Visual Business Intelligence)

Thursday, February 25th, 2010

[Editor's note: Visualizing complex connection topologies is made easier with a new plugin for Microsoft Excel. Now someone needs to port it to Flash ActionScript 3!]

Republished from Visual Business Intelligence.

This blog entry was written by Bryan Pierce of Perceptual Edge.

The chances are good that you’ve seen network visualizations before, such as the one below in which the circles and octagons represent large U.S. companies and each connecting line represents a person who sits on the board of both companies.

(This image was created by Toby Segaran: http://blog.kiwitobes.com/?p=57)

While these types of graphs have become more common in recent years, there’s still a good chance that you’ve never created one yourself. This is because, traditionally, to create network visualizations, you’ve either needed specialized (and often unwieldy) network visualization software or a full-featured (and usually expensive) visualization suite. That’s no longer the case. A team of contributors from several universities and research groups, including the University of Maryland and Microsoft Research, recently released NodeXL, a free add-in for Excel that allows you to create and analyze network visualizations.

Using NodeXL you can import data from a variety of file formats and it will automatically lay out the visualization for you, using one of twelve built-in layout algorithms.

Continue reading at Visual Business Intelligence . . .

How to split up the US (Pete Search)

Tuesday, February 16th, 2010

6a00d83454428269e20120a86baaf6970b-800wi

[Editor's note: Topology analysis of the Facebook social network (how many people in one town are connected to another) overlayed on a curious map base in geographic and regrouped into regions like Greater Texas, Socalistan, and Mormonia. Not quite sure of how the author define's Pacfiica and the map suffers from poor red-green contrast but cool concept.]

Republished from Pete Search.

As I’ve been digging deeper into the data I’ve gathered on 210 million public Facebook profiles, I’ve been fascinated by some of the patterns that have emerged. My latest visualization shows the information by location, with connections drawn between places that share friends. For example, a lot of people in LA have friends in San Francisco, so there’s a line between them.

Looking at the network of US cities, it’s been remarkable to see how groups of them form clusters, with strong connections locally but few contacts outside the cluster. For example Columbus, OH and Charleston WV are nearby as the crow flies, but share few connections, with Columbus clearly part of the North, and Charleston tied to the South:

Columbus Charleston

Some of these clusters are intuitive, like the old south, but there’s some surprises too, like Missouri, Louisiana and Arkansas having closer ties  to Texas than Georgia. To make sense of the patterns I’m seeing, I’ve marked and labeled the clusters, and added some notes about the properties they have in common.

Continue reading at Pete Search . . .

OpenStreetMap leveraged for bikes: Ride the City – DC Metro

Monday, February 8th, 2010

screen-shot-2010-02-08-at-21357-am

[Editor's note: This routing tool considers bike paths and trails and supports drag and drop start and stop icons (rather than just address entree). It's available for several major metro areas across the US and just came to Washington, DC. How can you get it in your town? Yet another reason to contribute to OpenStreetMap.org, the backend behind the tool. Thanks Jaime!]

Republished from Ride the City.

Washington D.C. is a great city for bicycling: its greenway network is extensive and it’s relatively flat. D.C. is also home to Smartbike DC, a public bike rental program.

We’re happy to announce that today bicycling in the nation’s capital just got easier: Welcome Ride the City – DC Metro! This newest addition includes Washington D.C., Arlington, Alexandria, all of Fairfax, and the Maryland suburbs within the Capital Beltway. We’re hopeful that by making it easier to ride bikes around the epicenter of U.S. political power that we may inspire more action to bring about improved bicycle facilities everywhere, especially in cities where biking is a sensible alternative to driving.

Ride the City – DC Metro was probably our biggest challenge to date. It was tricky because of the many jurisdictions (six counties) and various data sources that had to be organized, not to mention the 1,148 square miles of area and over 450 miles of separated (i.e. Class 1) bike ways that had to be manually edited. We’re happy to have had help from many good people in the bicycling world. Among those who helped, we’d like to thank Chantal Buchser (Washington Area Bicyclist Association), Bruce Wright (Fairfax Advocates for Better Bicycling), and Jeff Hermann (Fairfax County DOT) for helping us with data, troubleshooting, and leveraging volunteers to test routes early on.

(For those of you who are new to Ride the City, keep in mind that the Cloudmade basemap that we use is based on Open Street Map, the volunteer effort to map the world. If you notice discrepancies on the map, you can edit Open Street Map yourself or tell us about it and we’ll edit Open Street Map for you. To learn more about Open Street Map, click here.)

Try it out at Ride the City . . .

Political prerogative in the embassies (Kelso via Wash Post)

Monday, October 26th, 2009

gr2009102300322

[Editor's note: I created this bivariate Dorling cartogram for Al's column on Friday. The bubbles are grouped by geographic region show number of total ambassadors and the subset (in red) who have been political appointees the last 49 years. The subset is aligned bottom middle instead of sharing the same center point as the total bubble. If you haven't seen our Head Count interactive database tracking all Obama's federal appointments, check it out!]

Republished from The Washington Post. Reported by Al Kamen.

Just after the election in November, we wrote that an Obama administration was likely to eschew “the traditional sale of most ambassadorships, so aptly carried on during the Bush administration.” The chatter was that the new team would pick political types, but with some foreign policy cred — as the Clinton administration tended to do — and maybe reduce the percentage of politicals in favor of more career Foreign Service officers.

Yeah, well, we must have been eschewing something. The fat-cat contributors naturally got the plum postings, as usual.

But judging from data compiled by the American Foreign Service Association, the career employees union, it appears that Obama is on track to reduce, at least marginally, the percentage of jobs going to contributors and cronies. While there are still a lot of vacancies, AFSA officials project that Obama is likely to end the year appointing fewer political folks than either Bush or Clinton to the 181 ambassadorial postings — but still too many, as far as the career diplomats are concerned.

About 30.1 percent of Bush’s ambassadors during his eight years were political folks, AFSA found. Clinton’s average, 33 percent politicals, was higher, but Clinton’s folks were a mix of non-career people who actually knew a lot about the countries or regions to which they were named and pure cash types — our favorite was hotelier Larry Lawrence for Switzerland, the guy whose body was exhumed from Arlingon National Ceremony when it turned out he lied about being in the Merchant Marine.

If Obama’s first-year total ends up slightly lower than Bush’s, then Obama’s eventual four-year — or eight-year — percentages will probably be clearly lower than his immediate predecessors’, we’re told, because the first round of appointments tends to skew more to paying off politicals than do the later rounds.

Of course, the politically connected still get the finer spots in the Caribbean and Western Europe. As the accompanying chart shows, the career diplomats head to somewhat less delightful (even nasty) postings in Central Asia (100 percent career since 1960), the Middle East, Africa and South America.

Since 1960, no Foreign Service officer has ever run the embassy in Dublin and only one, Ray Seitz, has gone to the Court of St. James’s in London. On the other hand, no political appointee has ever gone to Chad and only one has gone to Bulgaria.

See AFSA’s full data at http://www.afsa.org/ambassadors.cfm.

SOURCE: American Foreign Service Association, data 1960 through today. | Nathaniel Vaughn Kelso and Karen Yourish/The Washington Post – October 23, 2009

Continue reading at The Washington Post . . .

News Dots (Slate)

Wednesday, September 9th, 2009

slatenewsdots

[Editor's note: This is one of the first tools I've seen that links topics, people & places into a network of graduated circles based on their ranking in the news. The circles are arranged based on their edge connections within the overall topology using the Flare visualization package in Flash AS3. As seen in the above screenshot, Germany is linked to Afghanistan, NATO, the Taliban, The Washington Post, and 20 other nodes. This project is one step forward in the vision I outlined in Topology and Projections: 21st Century Cartography. Disclosure: Slate is owned by the Washington Post Company, my employer, but I was not involved in this project.]

Republished from Slate.

Introducing News DotsAn interactive map of how every story in the news is related, updated daily.

Like Kevin Bacon’s co-stars, topics in the news are all connected by degrees of separation. To examine how every story fits together, News Dots visualizes the most recent topics in the news as a giant social network. Subjects—represented by the circles below—are connected to one another if they appear together in at least two stories, and the size of the dot is proportional to the total number of times the subject is mentioned.

Like a human social network, the news tends to cluster around popular topics. One clump of dots might relate to a flavor-of-the-week tabloid story (the Jaycee Dugard kidnapping) while another might center on Afghanistan, Iraq, and the military. Most stories are more closely related that you think. The Dugard kidnapping, for example, connects to California Gov. Arnold Schwarzenegger, who connects to the White House, which connects to Afghanistan.

To use this interactive tool, just click on a circle to see which stories mention that topic and which other topics it connects to in the network. You can use the magnifying glass icons to zoom in and out. You can also drag the dots around if they overlap. A more detailed description of how News Dots works is available below the graphic.

Interact with the original and learn more at Slate . . .

MarkerClusterer in an All New Flavor – ActionScript! (GoogleGeoDev)

Friday, August 14th, 2009

[Editor's note: An AS3 Flash / Flex library for auto-clustering near map location markers into groups symbols. This speeds map rendering and groups points all within the same neighborhood, avoiding "red dot fever" marker overload. An AS3 implementation of existing JavaScript extension. Still needs to account for geographic region clustering (not just within a grid). Thanks Laris!]

Republished from Google Geo Developer Blog.
Monday, July 27, 200. 

My name is XIAO Juguang – just call me Juguang. I am a freelance software developer based in Beijing, China. Technically speaking, I’m double sided. On one side, I specialize in knowledge management and business modeling, traditionally using LAMP and now experimenting with offerings like Google App Engine. On the other side, I love visualization in time and space, with charts, trees, graphs, and maps, always using the power of ActionScript/Flex, with the help of open-source projects like Degrafa, Axiis, and Birdeye, and of course, APIs like the Google Maps API for Flash.

A few month ago, Xiaoxi Wu (also from China!) created the MarkerClusterer library for the Google Maps JavaScript API v2 and released it in their open source utility library. This library did automatic clustering of markers placed on a map, so that a large amount of markers wouldn’t overcrowd the map or overwhelm the user. This is a great technique for having a better performing map (see this talk for more tips on improving map performance), and the Flash map community immediately rushed to port the code to ActionScript. Developer Sean Toru posted the first port, a version that was only Flash-compatible, Ian Watkins modified that port to use Flex packages, and then I refactored the code to be more ActionScript-friendly and released it into the open-source library. It’s great when random strangers can collaborate together on a common code goal. :)

To see how the AS3 MarkerClusterer works, try out the demo (shown above). As you zoom and pan the map, you can witness how the markers are clustered and re-clustered. To learn how to use MarkerClusterer on your own map, view the source code of the demo. To use the library, check out the source code and import it into your project.

The current algorithm is quite simple, just clustering markers in a grid and using static images for the cluster markers. Future extensions could include support for regional clustering or using arbitrary DisplayObjects for the cluster markers. If you’re interested in extending the library, join the project.

Travellr: Behind the Scenes of our Region-Based Clusters (Google GeoDev)

Monday, July 6th, 2009

[Editor's note: The age-old rule for cloropleth mapping that suggests aggregation by multi-scale areal units based on the map's zoom level is slowly seeping into "clustering" for the point-based mashup geo community. This overview from Travellr published on the Google GeoDevelopers blog includes two illustrations that show the power of this technique. I used such a technique (different implementation) on The Washington Post's recent swine flu mapping.]

Republished from Google GeoDevelopers Blog.
Monday, June 22, 2009

Recently, there has been a lot of interest in clustering algorithms. The client-side grid-based MarkerClusterer was released in the open source library this year, and various server-side algorithms were discussed in the Performance Tips I/O talk. We’ve invited the Travellr development team to give us insight on their unique regional clustering technique.

Travellr is a location aware answers service where people can ask travel-related questions about anywhere in the world. One of its features is a map-based interface to questions on the site using Google Maps.

dxjmnbf_4t5qkqpfw_b1
Figure 1. An example of the Travellr Map, showing question markers for Australia.

Clustering for usability
We learned that the best way to display markers without cluttering our map was to cluster our questions depending on how far you zoom in. If the user was looking at a map of the continents, we would cluster our questions into a marker for each continent. If the user zoomed-in to France we would then cluster our questions into a marker for each region or city that had questions. By clustering our data into cities, regions/states, countries, and continents, we could display relevant markers on the map depending on what zoom level the user was looking at.

Optimizing for Clustering
Our next challenge was how to extract clustered data from our database without causing excessive server load. Every time the user pans and zooms on the map, we need to query and fetch new clustered data in order to display the markers on the map. We also might have to limit the data if the user has selected a tag, as we’re only interested in a questions related to a topic (ie: “surfing”). To execute this in real-time would be painstakingly slow, as you would need to to cluster thousands of questions in thousands of locations with hundreds of tags on the fly. The answer? Pre-cluster your data of course!

Step 1. Structure your location data
When a question is asked about a city on Travellr, we also know its region/state, country and continent. We store more than 55,000 location points as a hierarchy, with each location “owning” its descendent nodes (and all of their data). Our locations are stored in a Modified Preorder Tree (also called Nested Sets). Modified Preorder Trees are a popular method of storing hierarchical data in a flat database table, having a focus on efficient data retrieval, and easy handling of sub trees. For each location we also keep a record of its depth within the tree, its location type (continent, country, region/state, or city), and its co-ordinates (retrieved using the Google Maps geocoder).

Step 2. Aggregate your data
We calculate aggregate data for every branch of our locations tree ahead of time. By storing aggregate data for cities, regions/states, countries, and continents, we provide an extremely fast and inexpensive method to query our locations database for any information regarding questions asked about a particular location. This data is updated every few minutes by a server-side task.

Our aggregations include:

  • Total question count for a location
  • Most popular tags for that location
  • Number of questions associated with each of those tags.

How we query our structured, aggregate data on the map
Whenever the user zooms or pans the map we fire off a query to our (unpublished ;) API with the tags they are searching for, the current zoom level, and the edge co-ordinates of the map’s bounding box. Based on the zoom level (Figure 2) we work out whether we want to display markers for continents, countries, states, or cities. We then send back the data for these markers and display them on the map.

dc287ncr_29cb84v7ct_b
Figure 2. Clustering at different zoom levels (blue = continents, countries, pink = states, cities)

Everyone Wins
So what is the result of structuring and aggregating our data in such a way? It means that we have nicely organized, pre-clustered data that can be read from cheaply and easily. This allows us to provide a super-fast map interface for Travellr that puts minimal load on our infrastructure. Everyone is happy!

Comments or Questions?
We’d love to hear from you if you have any questions on how we did things, or suggestions or comments about Travellr’s map. This article was written by Travellr’s performance and scalability expert Michael Shaw (from Insight4) and our client-side scripting aficionado Jaidev Soin.

You can visit Travellr at www.travellr.com, or follow us on Twitter at twitter.com/travellr.

The Internet’s Undersea World (Guardian)

Wednesday, June 24th, 2009

[Editor's note: Wireless is all the buzz but good old copper and fiber-optic cables link most of us together, as this graphic from last year's Guardian newspaper shows.]

Republished from the Guardian newspaper (UK).

The vast majority of the world’s communications are not carried by satellites but an altogether older technology: cables under the earth’s oceans. As a ship accidently wipes out Asia’s net access [in 2008-ed], this map showss how we rely on collections of wires of less than 10 cm diameter to link us all together.

View larger JPG version (hi-res pdf).

seacablehi