Archive for the ‘Data source’ Category

Natural Earth v2.0.0 is live

Thursday, November 8th, 2012

I am proud to announce the immediate availability of Natural Earth 2.0.0!

The 2.0.0 release focuses on 7 major areas and is available to download today à la carte at NaturalEarthData.
ZIP combo downloads of all vectors: SHP (279 mb) or SQLite (222 mb) or QuickStart kit for ArcMap and QGIS (165 mb).
  1. Economic geography: adds global roads, railroads, ports, airports, and time zones to show how people are interconnected and goods route (read Richard Florida on airports, full legal document about time zones and international date line shifts, and background on the E-Road network).
  2. Remastered geometries: fixes topological errors at 1:10 to 1:1,000 scales in the basic coastline, ocean, land, admin-0, and admin-1 related themes for files in the the 1:10m scaleset. By removing self-intersections, sliver polygons, and adjusting offset polygons, Natural Earth imports into more GIS software (like PostGIS) and will be easier to maintain. The coastline is adjusted to better conform to ~1:3,000,000 satellite imagery. Because of all these changes, some raster themes are also updated. Land, ocean, and minor islands all build topologically by scripting ingredients, as do the admin-0 and admin-1 cultural themes.
  3. Introduce Gray Earth rasters. Worldwide terrain depicted monochromatically in shades of gray. It combines shaded relief and regionally adjusted hypsography that emphasizes both high mountains and the micro terrain found in lowlands. View new raster »
  4. New file name and field name schemas. Full adoption of ne_10m_theme_name.shp file names with `ne_` prefix to allow better import into GeoDB and PostGIS storage, lowercase field (column) names instead of MiXeD and UPPER cased names, and use of consistent `name` field (versus name1).
  5. Address user submitted bug reports, ~25 since the 1.4 release, and earlier.
  6. Moved to Github for the backend versioned file management and coordination. Includes scripts to package updates and auto-create derived themes. View Natural Earth Vector on Github »
  7. Adopt semantic versioning. Know, by theme, the level of effort needed to update your maps when Natural Earth data updates are released. Read more about Natural Earth versioning »

Other notable changes:

All themes now include README and VERSION files. The admin-0 attributes have more veracity and now includes nested disputed areas (was a sidecar). Adds continent, region, subregion codes. Adds versions of country and admin-1 without boundary lakes.  All places and parts of places have population and GDP estimates. The populated places pop_max and pop_min attributes are now fully built out for all records (pop max is for the metropolitan area, pop_min is for the incorporated city of the same name). populated places now include rank_max and rank_min for simple town size grading. All instances of name1 have been changed to name, name to name, name2 to name_alt. Vertexes were added to many themes to allow them to project into conics smoothly (they’re back!). All field (column) names are now generally in the order of: scalerank, featurecla, name, name_alt, natscale, labelrank, *.

Contributors

Many thanks to the individuals who contributed over the last year of development: Tom, Nathaniel, Alex Tait, Hans van der Maarel, Scott Zillmer, Mike Migurski, Daniel Huffman, Xan Gregg, Peter Bispham, Drew Noakes, Miguel Angel Vilela, Matthew Toro, Kevin Pickell, Shawn Allen, Robert Coup, Iain, Leo, and more! Thanks also to Stamen thru the Knight Foundation Citytracking grant for sponsoring a portion of this work including remastering geometries for better PostGIS import, the move to Github, and adopting semantic versioning.

Changelog

Over 225 files have been updated in Natural Earth 2.0.0. Abbreviated listing below.
  • UPDATED: NE_ADMIN_0 - Updated for South Sudan, map colors (now with 7, 8, 9 and 13 options), population figures, removed () from notes, and more. note: diffs between sov, adm0, map units, map subunits, and new breakaways are all calculated on the a3 codes now, no longer mix of names and a3 codes. Added and split note_adm0 and note_brk to note which countries are parts of which sovereignties and who’s breaking away or disputing. One spurious “county” feature code fixed to “country” (finland). Added labelrank on all. Added new mapcolors (7, 8, 9 and old 13). Includes new detail on Caribbean Netherlands map unit. Adds more detail to Bhutan disputed areas. Now includes continent codes, and future region code placeholder columns. Added name_len to know when to abbreviate labels. Added label ranks.
  • UPDATED: NE_10M_ADMIN_0_BOUNDARY_LINES_LAND - Minor updates to alignment of boundary lines (and topology fixes), additional coding to allow official US gov’t view of same. better disputed coding, including Kosovo. Densified vertex along lines to allow smooth projection into conics. Moved Omani exclave Madha to correct location. Adds left and right labels and codes. Fixes: N96NSYPAPV, ZQNTN5VGDD, Z8ZYYUQZVS.
  • UPDATED: NE_50M_ADMIN_1_STATES_PROVINCES_SHP – Added some new ISO coding, other minor changes. Fixes topology errors. Adds admin-1 for brazil and australia. Uses same coding as 10m files. Derived from new scale rank version.
  • **NEW**: NE_10M_ADMIN_0_ANTARCTICA_CLAIMS – Although countries have paused their claims to the southernmost continent, they haven’t suspended them. Thanks, Hans!
  • **NEW**: NE_10M_ADMIN_0_ANTARCTICA_CLAIM_LIMIT_LINES – Although countries have paused their claims to the southernmost continent, they haven’t suspended them. Thanks, Hans!
  • UPDATED: NE_10M_POPULATED_PLACES – A couple name corrections (Morelia, Mexico spelling fixed. Mazatlan, Mexico spelling fixed. Clarified confusion around Tabatinga / Leticia on the Colombian / Brazilian border. On the Brazil / Bolivia border, clarified Brasileia / Cobija. Fixed spelling of Shuozhou, China), many population max values, mostly in China, India, rift valley (Africa), Nigeria, and other countries in east Asia, but some elsewhere. Made sure cities in Switzerland are coded admin-0 of CH and China are CN. Moved Amundsen Base to 176° so it’s in the -12 timezone. Also moved Peter I Island. Vatican City is also moved to be contained by it’s admin-0 polygon. Same for San Marino. Added a poprank column with 0 to 14 numerical classes. Deleted spurious Extra Eureka town in Canada near Greenland. Delete duplicate town Urengoy in RUS, rename the real one Novy Urengoy. Fixes: 4SUAZ7BB49, D459XT1Z6Y.
  • UPDATED: NE_10M_COASTLINE – Better matches modern satellite imagery to zoom 8-ish. The earlier coastline could have been several kilometers off (like in Gibraltar). Several large new islands added. Includes densified vertex along lines to allow smooth projection into conics.
  • UPDATED: NE_10M_RIVERS_LAKE_CENTERLINES – See changelog for ne_10m_rivers_lake_centerlines_scale_ranks for details.
  • UPDATED: NE_10M_RIVERS_LAKE_CENTERLINES_SCALE_RANKS – Updated river names, few new rivers, splits. added river connector in Sweden between lake near Stockholm and Baltic Sea. Fixes in France and Netherlands. Fixes Mackenzie river at it’s confluence with Dawson river in Australia. Names the Mahakam in Borneo (Rivernum 544). Changes scalerank on Nelson river in Canada. Fixes: SHAWNZQJ3B, 5J47B13PJ7, W9X539LBUT, 35YLBL2W9Z.
  • UPDATED: NE_10M_RIVERS_LAKE_CENTERLINES_NORTH_AMERICA_SUPPLEMENT – Updated river names, few new rivers, splits. Fixes: SHAWNZQJ3B, 5J47B13PJ7.
  • UPDATED: NE_10M_RIVERS_LAKE_CENTERLINES_EUROPE_SUPPLEMENT – Updated river names, few new rivers, splits. fixed topology errors. Fixes: SHAWNZQJ3B.
  • UPDATED: NE_10M_LAKES – Removed major lake groupings (Great Lakes, Finger Lakes, etc) to geography label areas instead. Title cased the feature class values. Added Swedish lake near Stockholm (had been extension of Baltic Sea in ocean theme). Fixed topology errors. Fixed a few reservoir and salt lake codes (thanks Craig!).
  • UPDATED: NE_10M_LAKES_NORTH_AMERICA_SUPPLEMENT – Name1 have been changed to name, name to name, name2 to name_alt. Fixes 4VA9P9UGQE.
  • UPDATED: NE_10M_GEOGRAPHIC_LINES – New int’l date line, thanks Alex! Also densified linework for smoother projection into conics.
  • UPDATED: NE_10M_LAND – A dissolved version of the original 1.x file, now renamed “ne_10m_land_scale_rank”, see that changelog for full details. Fixes XAWXTN54GT.
  • **NEW**: NE_10M_LAND_SCALE_RANKS – Renamed our original land file to this. Incorporates new coastline. Includes densified vertex along lines to allow smooth projection into conics. Fixes XAWXTN54GT.
  • UPDATED: NE_10M_OCEAN – A dissolved version of the original 1.x file, now renamed “ne_10m_ocean_scale_rank”, see that changelog for full details. Fixes XAWXTN54GT.
  • **NEW**: NE_10M_OCEAN_SCALE_RANKS – Renamed our original ocean file to this. Incorporates new coastline. Removed Swedish lake near Stockholm (had been extension of Baltic Sea in ocean theme) to lakes layer. Incorporates new coastline. Includes densified vertex along lines to allow smooth projection into conics. Fixes XAWXTN54GT.

Announcing Natural Earth v2.0.0, release candidate 3

Tuesday, October 16th, 2012

Just in time for NACIS 2012 in Portland, Oregon!

This is a followup to the earlier version 2.0.0 release candidate 2. Read more about RC2 »

Downloads:

Please submit bugs or other oddness found release candidate 3 to myself nathaniel@kelsocartography.com ASAP. The plan is to cut a final 2.0.0 release by the end of October. View full combo change log for 2.0.0 »

What’s new in 2.0.0 release candidate 3:
  1. Added explicit labelrank in the admin-0 XLS doc. Also added homepart (map units, map subunits, brkaway) and tiny (country) to better deal with labeling awkwardness and adds consistency between themes and scalesets.
  2. Updated sr_label_i, sr_label_o  on tiny_countries for better labeling.
  3. Added 3 new “mapcolors” for 7-up, 8-up, and 9-up into details doc and onto the admin-0 and admin-1 themes. This adds to the 13-up original. The 9-up is optimal, the others are useful in their own ways. 
  4. Renamed the “note” in the admin-0 files to “note_adm0” and added details for Korea and Japan parts (just the exterior bits). Added AUZ note for CSI, Cayman Islands, Turks, etc.
  5. Added the disputed areas note directly to the admin-0 details, named “note_brk“.
  6. Fixed admin-0 coding with Georgia to fix map units bug. Added new Caribbean Netherlands and metropolitan Netherlands codes to reflect rejigger there.
  7. Added continent codes for admin-0 countries and their various parts. Stubbed out region and subregion fields, values to come later release. Thanks SmugMug!
  8. Removed “County” admin-0 feature class, remapping it to “Country” (Finland only).
  9. Removed parenthetical () from notes, see Faulkland Is. etc. Removed all claim notes except from the BRK bits in admin-0 details. Add that while labeling now (consistent with other app logic now).
  10. Add admin-0 diffs for sov, adm0, map units, map subunits, and disputed areas are now calculated on the a3 code, not the name. This can be used for progressive labeling, fixing problems in Tanzania and elsewhere. You still need logic in your app to determine if the labeling name is different, but now it’s consistent.
  11. Added Taiwan to disputed areas. Renamed Congos (eg Democratic Republic of Congo instead of Congo (Kinshasa)).
  12. Modified the scalerank of Venezuala claim into Guyana boundary line fix 10m theme. Similar for the Saichen Glacier.
  13. Changed the scaleranks on all 10m claim polys. Most now appear at scalerank 6 and in.
  14. For labeling, added name_len to the admin 1 and admin 0 details and themes.
  15. Changed several scaleranks of marine label areas in 10m and 50m themes.
  16. Added continent code for roads (partial) so North America and Europe can be separated out easier. Might build topology for roads in final release to make future cleanup easier.
  17. Updated, added new Airports by rerunning export from Mile High Club, with better field names.
  18. Updated, added new Ports by rerunning export from High Seas, with better field names. This added inland ports especially along the North American Great Lakes.
  19. Promoted a few populated places up to scalerank 1 and 2 to give better balance to compilation. Many cities have more accurate pop_max values (partial, more due in final release). Thanks for the nudge Craig!
  20. National park (protected areas) points were missing “name”. Some reranking for park points in urban centers (to scaleranks 7, 8, 9). A couple park areas reranked around scalerank 3. Added Cesar Chavez NM and added missing unit_codes for Clinton Birthplace NHS and River Raisin NBP. Thanks Tom!
  21. Add admin_0 alpha3 codes to boundary lines to allow better selection (grab the boundaries of USA only, for instance). Also added left and right names to allow better admin-0 line labeling on zoom in.
  22. Parts of Chile and Cyprus were missing in derived admin-0 themes (were present in scale rank masters) so modified some label points so their scalerank is now <= 6 for admin-0 selection. These had been 7 and 8.
  23. Removed old versions of admin-1 lines from the ZIPs.
  24. Added left and right names for admin_1 boundary lines for better labeling once zoomed in. Because native names are in UTF8 (from the polygons), also added a “master” lines version in GeoDB format along with the SHP version, sidestepping character encoding problems. Thanks Mamata and Mike!
  25. The admin-1 boundary between Tasmania and mainland Australia is now straight (thanks Craig!). Adjusted admin-1 boundaries in Bhutan to match new admin-0 disputed areas and boundaries.
  26. Added Australia and Brazil admin-1 lines and polys to 50m. Redid the 50m and 110m admin-1 attributes (lines, polys) to match the 10m. Added scale_rank version of the polys at 50m and 110m to allow for easier updates. Recut a lakes erase version of the 50m admin-1, but now using lakes with scalerank <= 0 instead of 1 (North America’s Great Lakes, but not Florida or Utah lakes) at Mike Bostock’s suggestion.
  27. Modified scaleranks on the 10m urban areas and copied those over to the 50m urban areas. Deleted a bit of 10m urban on Mt. Hood, Oregon snowfield.
  28. Redid marine label area scale ranks.
  29. Removed duplicate Laccadive Is. from 10m-geography-regions.
  30. Fixed river scalerank bug in 10m theme in Canada for the Nelson River leading out of Lake Winnipeg, thanks Gregor!

Still to do for final release:

  • Rebuild the 10m, 50m, and 110m admin 0 themes using the new details. Add area_sqkm to all themes after build.
  • Merge in modified 10m China, Taiwan admin-1 from a fork, thanks Chunshek!
  • Merge in modified cities in China, Taiwan from a fork, thanks Chunshek!
  • Merge in modified 10m lakes from a fork, thanks Craig!
  • Resolve ne_10m_railroads_beta1_north_america_original.shp
  • Finish adding continent codes on 10m roads, this might involve adding topology to crack geoms.
  • Include PRJ files with each GeoTIFF, for Jody and the uDig app on OSGeo disk.
  • Update quickstart MXD, QGIS docs with new file names, paths, field names / values.

Announcing Natural Earth v2.0.0, release candidate 2

Monday, August 20th, 2012

The 2.0.0 release focuses on 6 major areas and is available to download today as preview release candidate 2.

  1. Economic geography: added global roads, railroads, ports, and airports to show how people are interconnected and routes goods move.
  2. Remastered geometries: fixed topological errors at 1:10 to 1:1,000 scales in the basic coastline, ocean, land, admin-0, and admin-1 related themes. By removing self-intersections, sliver polygons, and adjusting offset polygons, Natural Earth will import into more GIS software (like PostGIS) and be easier to maintain. The coastline was also adjusted to better conform to ~1:3,000,000 satellite imagery. Because of all these changes, some raster themes were also updated.
  3. New file name and field name schemas. Full adoption of ne_10m_theme_name.shp file names with `ne_` prefix to allow better import into GeoDB and PostGIS storage, lowercase field (column) names instead of MiXeD and UPPER cased names, and use of consistent `name` field (versus name1).
  4. Address user submitted bug reports, ~20 since the 1.4 release.
  5. Moved to Github for the backend versioned file management and coordination.
  6. Adopt semantic versioning. Know, by theme, if new changes in Natural Earth affect your project. See below for more information.

Downloads:

Please submit bugs or other oddness found release candidate 2 to myself nathaniel@kelsocartography.com by September 10th. The plan is to cut a final 2.o.0 release by the end of September. A full change log will accompany the final release.

Sample images:

Global roads:

London, Paris, Amsterdam

Brazil

New scale ranking on North American roads. Better matches with global roads, and OpenStreetMap:

DC and NYC with extra roads

Administrative units 0 and 1 have been completely remastered:

New admin-1 (states, provinces) for Natural Earth 2.0

Urban areas are now scaleranked, and false urban areas removed (mostly Nevada desert errors):

New urban scale ranks, Tokyo

Semantic Versioning

The 2.0 release in 2012 marked the project’s shift from so-called marketing versions to semantic versioning.

Natural Earth is a big project with hundreds of files that depend on each other and the total weighs in at several gigabytes. SemVer is a simple set of rules and requirements around version numbers. For our project, the data layout is the API.

  • Version format of X.Y.Z (Major.Minor.Patch).
  • Backwards incompatible changes, increment the major version X.
  • Backwards compatible additions/changes, increment the minor version Y
  • Bug fixes not affecting the file and field names, patch version Z will be incremented.

Major version increments:

  • Changing existing data file names
  • Changing existing data column (field) names
  • Removing FeatureCla field attribute values
  • Additions, deletions to admin-0
  • Introduce significant new themes

Minor version increments:

  • Any shape or attribute change in admin-0
  • Additions, deletions, and any shape or attribute changes in admin-1
  • Additions, deletions to any theme
  • Major shape or attribute changes in any theme
  • Adding, changing FeatureCla field attribute values
  • Introduce minor new themes

Patch version increments:

  • Minor shape or attribute changes in any theme
  • Bug fixes to shape, attributes in any theme

Under this scheme, version numbers and the way they change convey meaning about the underlying code and what has been modified from one version to the next.

When we introduce a new version of Natural Earth, you can tell by the version number how much effort you will need to extend to integrate the data with your map implementation.

  • Bug fixes Z: can simply use the new data files, replacing your old files.
  • Minor version Y: limited integration challenges.
  • Major version X: significatnt integration challenges, either around changed file strucutre, field layout, field values likeFeatureCla used in symbolizing data, or significant new additions or significant changes to existing themes.

When open data is not open: World Bank double speak on Google Map Maker?

Sunday, February 19th, 2012

[Editor's note: Good news! The World Bank has listened and responded with a new blog post clarifying their stance on open data and backed away from Google Map Maker: "If the public helps to collect/create map data, the public should be able to access, use & reuse that data."]

By Nathaniel Vaughn KELSO

In their recent op-ed in the New York Times and on the official World Bank blog, Caroline Anstey* and Soren Gigler** made a compelling case for open data and open government that is fatally flawed. Unless fixed, it has dangerous implications for the future of open data globally. The World Bank’s new policy around map data is needlessly exclusionary: public data should remain publicly accessible.

Over the past two years, the World Bank has made great strides in making its processes more transparent and rethinking the international organization as a development platform and innovation bank.

However, on January 13th, Ms. Anstey and Mr. Gigler muddied that effort by partnering with Google on map data as an end game around many of those “open” goals. Let me be very clear:

The World Bank’s new agreement with Google is a neocolonial wolf handing out shiny blue Map Maker t-shirts.

Google Map Maker expressly prohibits citizen cartographers from using/sharing the very data they add to the map in ways that can help their own development efforts. Users are locked into the Google platform: they cannot export their data or create derivative work, especially commercial projects. Nor can they share that geo data with other online mapping efforts, especially critical during disaster relief.

The World Bank has successfully partnered on map data before, most notably in Haiti with OpenStreetMap (OSM) in response to the major earthquake and the following humanitarian crises.

The Haiti experience shows that crowd sourcing map data works. Most developing countries do not have basic local map data. Timely, accurate geo data showing roads, schools, fresh water sources, health facilities, and more help save lives during an emergency, and in the meantime we all enjoy with up-to-the-minute maps. Regardless if you are in Washington, D.C. or Nairobi, Kenya.

I agree the Bank’s core mission is advanced by improving access to geo data for humanitarian response and development planning. This serves to make development more effective and inclusive by expanding access to basic geo information.

It is appropriate to engage citizens in the Bank’s client countries by inviting them to participate (via mapping parties and online portals) and strengthen the capacity of civil society to “put them selves on the map”. That is an inalienable right and a noble effort by the Bank to facilitate.

But this new agreement falls down on close inspection of the Google Map Maker terms of service. The corporate legalese is contrary to another Bank core principle:

“the right to use that same open data to empower citizens in effective development.”

The Google partnership proposes a new digital serfdom. The Bank should instead embrace the OpenStreetMap model: a system of micro-data grants that empowers a self-sustaining wave of economic development as more data gets added to the map. The citizen map maker should have an ownership share.

I urge the Ms. Anstey and Mr. Gigler to emphasize to the World Bank’s local offices and partner organizations (including the United Nations) that this new agreement with Google is *non-exclusive*, meaning the Bank can and must open data by sharing local geo data with other organizations, like OpenStreetMap.

When the Bank partners to allow citizens to draw their own map, the resulting map data must be free and open. Indeed, open mapping tools and civil society organizations like OpenStreetMap (who innovated first with mapping parties and their online map editor) should be leveraged and grown as much as possible.

Instead, the current agreement allows Google to use local citizens to collect information for free and make exclusive profit. The agreement, specifically the general Map Maker terms or special ODbL terms for the World Bank project, should be rewritten as Patrick Meier*** suggests, to ”allow citizens themselves to use the data in whatever platform they so choose to improve citizen feedback in project planning, implementation and monitoring & evaluation.”

Terms of use like Open Database License (ODbL) promote circulation of geo data for the most good and Google has been receptive in the past to opening up parts of their Map Maker data for humanitarian relief. Let’s complete the circle so this type of license is a core part of a revised World Bank “open” data agreement and have it in place before the next disaster.

The World Bank must reiterate its commitment to truly open data with due speed.

—–

The author is chief cartographer for the Natural Earth, a public domain map database. His maps have been published in The Washington Post and National Geographic and he is a design technologist at Stamen Design.

—–

* Caroline Anstey is a managing director of the World Bank.
** Soren Gigler is Senior Governance Specialist, Innovation at the World Bank.
*** Patrick Meier is director of crises mapping at Ushahidi.

Related reading

PR: World Bank and Google Announce Map Maker Collaboration

Open Aid Partnership:

Patrick Meier, Ushahidi’s director of crisis mapping

RWW coverage

Directions Magazine coverage

Google’s Official LatLong Blog

Global Integrity

Does a Google-World Bank Deal On Crowdsourcing Ask Too Much of the Crowd? (TechPresident)

“What Gets Measured Can Be Changed”: World Bank Turns Its Data Catalog Public (TechPresident)

The World Bank Responds to the Google Map Maker Deal (Global Integrity)

 

Natural Earth version 1.4 release notes

Saturday, August 20th, 2011

Natural Earth has been downloaded more than 250,000 times and is closing in on 1 million page views since launching in the final month of 2009. Thanks!

Over 65 files have changed in version 1.4 of Natural Earth. The most significant edits reflect the July 2011 independence of South Sudan. That country’s administrative level-1 units have also been refreshed, and the disputed area of Abyei is retained.

Mea culpa: A major correction fixes a coding error introduced in version 1.3 that incorrectly merged Panama and Papua New Guinea into a super country in some admin-0 files. Version 1.4 reestablishes them as separate countries.

Because of South Sudan, the admin-0 (sovereign, country, map units, map subunits, scale ranks, scale ranks with minor islands), boundary lines land, disputed areas and boundaries, etc. have all been updated in the 10m, 50m, 110m scale sets.

50m admin-1 states and provinces now includes a version with the lakes punched out, like the 10m has enjoyed since version 1.3.

The 10m admin-1 file now includes a “scale rank” exploded version that will import into a wider range of GIS and CAD software packages that cannot import polygons over a certain vertex count.

Other changes and corrections are detailed below and credited to the correction request author.

Nathaniel Vaughn Kelso and Tom Patterson 19 August 2011
Read full release notes at NaturalEarthData.com »

Donate your iPhone location logs anonymously

Monday, May 2nd, 2011

[Editor's note: Apple and the other big smart phone players have been harvesting your location data to help your phone's GPS locate you faster. There was a big hubub a couple weeks ago about the insecure nature of the iPhone log database. A new website, CrowdFlow.net is crowd sourcing the federation of individual users location files so we get a complete picture. By pooling our logs, we can visualize patterns among these location beacons. Act now as the next update from Apple will remove this 'functionality'. Image above shows the locations around the world so far. Want to view your own tracks without sharing them? Check out Pete Warden's original iPhoneTracker.]

What’s the idea?

You probably know by now that your iPhone collects the position data of wifi and cell networks near by. Google Android devices as well. More background info herehere and here.

We would like to combine as many of these log files as possible, create an open database of wifi and cell networks and thus visualize how these networks are distributed all over the world.

So please contribute your iPhone log files and help us to create an open wifi und cell database.

You can find out more on this project on [their] blog.

How can I contribute?

We are starting with the iPhone.
If you would like to contribute your log file – and you can do that entirely anonymously -
follow these two simple steps:

Continue reading at CrowdFlow.net . . .

Natural Earth version 1.3 released

Monday, January 31st, 2011

When Natural Earth relaunched in December 2009 with updated raster and new vector data our aim was two fold: First, to give cartographers an off-the-shelf solution for creating small-scale world, regional and country maps from scratch. Second, we included a wealth of features both large and small in hopes of improving the overall geographic literacy of map readers. Since then, we’ve taken Natural Earth on an around-the-world road show and January 2011 saw our 150,000th direct download and 500,000th pageview. We even made it into Wikipedia, were featured in PrettyMaps, and power some of the goodness behind Google Fusion Tables. With today’s 1.3 release, we add a couple newly independent countries, better delineate the world’s states and provinces, and make a whole host of corrections and additions to the original dataset, detailed below. Please continue to use these fine map ingredients to make great web and print geo mashups. Bon appetit.

Nathaniel Vaughn Kelso and Tom Patterson

Continue reading and download the updated files at NaturalEarthData.com »

    New Flickr shapefile public dataset 2.0 (find the esri type .shp here)

    Tuesday, January 18th, 2011

    2971287541_27e6a06a21

    An updated version of the Flickr shapefile public dataset (2.0) was released last week. From nils official post:

    … We haven’t completely forgotten about shapefiles and have finally gotten around to generating a new batch (read about Alpha Shapes to find out how it’s done). When Aaron did the first run we had somewhere around ninety million (90M) geotagged photos. Today we have over one hundred and ninety million (190M) and that number is growing rapidly. Of course lots of those will fall within the boundaries of the existing shapes and won’t give us any new information, but some of them will improve the boundaries of old shapes, and others will help create new shapes where there weren’t any before. Version 1 of the dataset had shapes for around one hundred and eighty thousand (180K) WOE IDs, and now we have shapes for roughly two hundred and seventy thousand (270K) WOE IDs. Woo. The dataset is available for download today, available for use under the Creative Commons Zero Waiver.

    True to it’s claim, the version 2.0 release brings added fidelity on existing shapes (they are becoming more conformal to the features’ true geographic shape as more human sensors perambulate) and surveys some more cities and significantly more neighborhoods. From a data analytics perspective, I wish the new version had the summary photo count and centroid XY per feature of the 1.0 version. But very excited to see a new version released! Image above by Aaron Straup CopeMore coverage of things Flickr on Kelso’s Corner »

    While the dataset is distributed in GeoJSON format, that isn’t accessible to everyone so I’ve mirrored an ESRI Shapefile version of the Flickr Shapefile Public Dataset 2.0 with this blog post (~60 mb). Details on how I did the conversion after the jump.

    (more…)

    “Joy of Stats” from BBC4 featuring Hans Rosling

    Monday, January 3rd, 2011

    Now on YouTube! This hour long documentary “takes viewers on a rollercoaster ride through the wonderful world of statistics to explore the remarkable power they have to change our understanding of the world, presented by superstar boffin Professor Hans Rosling, whose eye-opening, mind-expanding and funny online lectures have made him an international internet legend.” BBC4

    Exploring Place through User-generated Content: Using Flickr to Describe City Cores (Spatial Info Sci)

    Monday, August 9th, 2010

    screen-shot-2010-08-09-at-20836-am

    [Editor's note: Good focus on vernacular geography, on how we name and describe space, with a particular focus on downtown city cores explored thru millions of photos on Flickr. "Importantly, it deals with regions which are typically not represented in formal administrative gazetteers and which are often considered to be vague." Never seen Flickr geography before? Check out Aaron's flickr shapetiles (map), shpfile browser, and geotagger world atlas.]

    Republished from the Journal of Spatial Information Science.
    By Livia Hollenstein and Ross Purves

    Terms used to describe city centers, such as Downtown, are key concepts in everyday or vernacular language. Here, we explore such language by harvesting georeferenced and tagged metadata associated with 8 million Flickr images and thus consider how large numbers of people name city core areas. The nature of errors and imprecision in tagging and georeferencing are quantified, and automatically generated precision measures appear to mirror errors in the positioning of images. Users seek to ascribe appropriate semantics to images, though bulk-uploading and bulk-tagging may introduce bias. Between 0.5–2% of tags associated with georeferenced images analyzed describe city core areas generically, while 70% of all georeferenced images analyzed include specific place name tags, with place names at the granularity of city names being by far the most common. Using Flickr metadata, it is possible not only to describe the use of the term Downtown across the USA, but also to explore the borders of city center neighborhoods at the level of individual cities, whilst accounting for bias by the use of tag profiles.

    Continue reading via their PDF . . .