Posts Tagged ‘geo’

New Flickr shapefile public dataset 2.0 (find the esri type .shp here)

Tuesday, January 18th, 2011


An updated version of the Flickr shapefile public dataset (2.0) was released last week. From nils official post:

… We haven’t completely forgotten about shapefiles and have finally gotten around to generating a new batch (read about Alpha Shapes to find out how it’s done). When Aaron did the first run we had somewhere around ninety million (90M) geotagged photos. Today we have over one hundred and ninety million (190M) and that number is growing rapidly. Of course lots of those will fall within the boundaries of the existing shapes and won’t give us any new information, but some of them will improve the boundaries of old shapes, and others will help create new shapes where there weren’t any before. Version 1 of the dataset had shapes for around one hundred and eighty thousand (180K) WOE IDs, and now we have shapes for roughly two hundred and seventy thousand (270K) WOE IDs. Woo. The dataset is available for download today, available for use under the Creative Commons Zero Waiver.

True to it’s claim, the version 2.0 release brings added fidelity on existing shapes (they are becoming more conformal to the features’ true geographic shape as more human sensors perambulate) and surveys some more cities and significantly more neighborhoods. From a data analytics perspective, I wish the new version had the summary photo count and centroid XY per feature of the 1.0 version. But very excited to see a new version released! Image above by Aaron Straup Cope. More coverage of things Flickr on Kelso’s Corner »

While the dataset is distributed in GeoJSON format, that isn’t accessible to everyone so I’ve mirrored an ESRI Shapefile version of the Flickr Shapefile Public Dataset 2.0 with this blog post (~60 mb). Details on how I did the conversion after the jump.


Maybe places are more about time than location: Retrofitting Geo for the 4th Dimension (Fekaylius)

Tuesday, July 27th, 2010


[Editor’s note: Thanks Sylvain!]

Republished from Fekaylius’s place.

We are in a period of mass-market place ambiguity.

Places drift, jump, and fade, physically. Some places have a much higher propensity towards noticeable drift than others, but location, in general, is not stable. The geo-web of the past few years has mostly ignored this as a low impact edge case. The era of the Google Maps API dramatically boosted developer productivity and interest within the geo space because it simplified and lowered the barriers to entry, while simultaneously reinforcing a few paradigms that find easy adoption within rapidly moving startups and business, ideas like “the perfect is the enemy of the good” and “solve for the 80% use case”. Startups are constantly faced with a to-do list that can never be 100% complete, but these catchy ideas formalize and automate the painful process of deeming some desires unworthy of your attention. Since 80% of the places that most people are searching for, or reviewing, or visiting feel relatively immune to change (at least in the “several years” lifespan much of today’s software is being designed for), we have very quickly built up a stiff and rigid framework around these places to facilitate the steep adoption of these now ubiquitous geo-services. The rigidity is manifest in the ways that place drift isn’t handled, places are assumed to be permanent.

Continue reading at Fekaylius’ place . . .

Yahoo GeoPlanet gains more concordance with other gazetters

Tuesday, June 8th, 2010

[Editor’s note: Yahoo has several nifty tools, including Placemaker, all powered off their GeoPlanet gazetteer, freshly updated in v 1.4 to include links back to other place name gazetteers and even Wikipedia articles, so we (and our machines) can know we’re all talking about the same places. Flickr, for instance, stores the geographic tags on photos using GeoPlanet WOEIDs. ]

Republished from Yahoo’s Geo Blog.
By Gary Vicchi.

Back in March at the annual geo-fest that is Where 2.0 in San Jose we released our concordance API as part of Yahoo! GeoPlanet. That initial release allowed conversion between the identifier namespaces of WOEIDs, ISO 3166, FIPS, INSEE, Geonames, JGCD, IATA and ICAO.

Lots of you liked this and asked us to add support for other identifier namespaces as well.

change sign

Robert C. Gallagher once said that change is inevitable – except from a vending machine and that works in the Geo world as well. So in version 1.4 of GeoPlanet which is live right now at you’ll find additional concordance namespace support for

  • OpenStreetMap place codes
  • Wikipedia place article ids
  • UN/Locode place codes
  • Country Code Top Level Domain codes
  • FAA airport codes
  • US and CA ZIP codes

Continue reading at Yahoo . . .

UK Addressing, The Non Golden Rules of Geo or Help! My County Doesn’t Exist (Yahoo!)

Monday, June 22nd, 2009


[Editor’s note: Amusing and practical example of geographic taxonomy, topology with the example of England versus United Kingdom.]

Republished from Yahoo! Geo.
By Gary Gale, Head of UK Engineering, Yahoo! Geo Technologies

George Bernard Shaw once said the golden rule is that there are no golden rules and at Geo Technologies we understand that there is no one golden rule for Geo and so we try to capture and express the world’s geography as it is used and called by the world’s people. Despite the pronouncement on golden rules, a significant proportion of the conversations we have with people about Geo lend themselves to the Six Non Golden Rules of Geo, namely that:

  1. Any attempt to codify a series of geo rules into a formal, one size fits all, taxonomy will fail due to Rule 2.
  2. Geo is bizarre, odd, eclectic and utterly human.
  3. People will in the main agree with Rule 1 with the exception of the rules governing their own region, area or country, which they will think are perfectly logical.
  4. People will, in the main, think that postal, administrative and colloquial hiearachies are one and the same thing and will overlap.
  5. Taking Rule 4 into account, they will then attempt to codify a one size fits all geo taxonomy.
  6. There is no Rule 6, see Rule 1.

I codified these rules after a conversation last week, via Twitter and Yahoo! Messenger, with Andrew Woods, a US based developer who was, understandably, confused by the vagaries of the how addresses work in the UK. Andrew’s blog contains the full context but can be distilled into three key questions:

  • If the country is The United Kingdom, how come the ISO 3166-2 code is GB?
  • If the country is The United Kingdom, is England a country?
  • If England is a country, do I use it in an address?

As a US developer, Andrew is naturally fluent with the US style of addressing, with all of its’ localised and regional exceptions. This is a good example of both Rules 3 and 4 in the real world; most people in the US will use number, street, city, State and ZIP for specifying an address. But how does this transfer to the UK? What’s the equivalent of a State … England, Scotland or Wales? Let’s try to answer some of these problems:

Continue reading at Yahoo! Geo . . .

Yahoo! Geo Technologies

Monday, June 22nd, 2009

[Editor’s note: Yahoo! provides advanced mapping capabilities including GeoPlanet, a Web 2.0 gazetteer of world placenames (see also GeoName’s post on the relational ontology / the semantic web).]

Republished from Yahoo!

Yahoo! wants to connect the Web to the World; here you can access our increasing portfolio of platforms to help you geo-enrich your applications and make the Internet more location-aware:

GeoPlanetâ„¢: Provides the geographic developer community with the vocabulary and grammar to describe the world’s geography in an unequivocal, permanent, and language-neutral manner. (Blog post)

GeoPlanet Data: Tab-delineated files containing WOEIDs and the corresponding place-names that underlie GeoPlanet.

Placemakerâ„¢: Identify, disambiguate, and ‘extract’ places from unstructured and structured textual content to help create local- and location-aware applications. (Blog post)

Fire Eagleâ„¢: Allows users to share their location with sites and services through the Web or a mobile device.

Maps: Embed rich and interactive maps into your web and desktop applications.

Yahoo Woe (Where On Earth, that is) IDs (GeoBloggers)

Tuesday, October 21st, 2008

[Editor’s note: Continuing to work thru a backlog of noteable but past news… I like how this API defines a placename’s location in a bounding box instead of a centroid. Wish it were both, but the bounding box seems more relevant (as a center point can be extracted, and you get the benefit of scale).]

Republished from Posted on by Reverend Dan Catt

Roll up, roll up, roll up, get your WoE IDs here …

… a jolly nice step in the right direction.

Yahoo have opened up their geo database (which is pretty good btw) which is far more awesome than I’m going to make it sound in this blog post. I already did a little bit back here. But a quick call out to these two functions that’ll try and find you WoE IDs based on string input…

… the first API call gives, 36240 as the value for Stoke on Trent in the UK, my old hometown. With that WoeID I can plug it back into these other API calls to get various useful information back …

We use WoE IDs over at Flickr, again you can read a little more about that near the end my terribly long previous blog posts about twitter, APIs and such like. A quick recap is that we have these WoE related APIs …

The ones that probably compliments the APIs over at are flickr.places.findByLatLon, which will turn a Lat/Long into what Flickr believes is there, more on this in a second.

And flickr.places.resolvePlaceId, if you plug the value of 36240 into the API explorer there you’ll get this xml back …

<rsp stat="ok">
	<location name="Stoke on Trent" woeid="36240" place_id="gEXCB1iaB571gw" place_url="/United+Kingdom/England/Stoke+on+Trent">
		<locality place_id="gEXCB1iaB571gw" woeid="36240">Stoke on Trent</locality>
		<county place_id="B_K1Z7iYA5qfCIiHaw" woeid="12602189">Staffordshire</county>
		<region place_id="pn4MsiGbBZlXeplyXg" woeid="24554868">England</region>
		<country place_id="DevLebebApj4RVbtaQ" woeid="23424975">United Kingdom</country>

Where you can see the same woeid, the URL to get to the Places page for Stoke on Trent, and the parent hierarchy flickr uses.

You can also plug the woeid into the to get photos back for just that area. A quick example of why that is useful is California, there’s no way you’d want to find photos for California using the bounding box, as it covers a couple of other states …

… but instead you can use the woeID for California (2347563) which deals with all the bends and kinks of CA. You can also throw that ID into the Places URL, like this …, although you’ll probably want to pass that through flickr.places.resolvePlaceId first if you want a pretty URL.

Differences between Yahoo geo API stuff and Flickr

Over at Flickr, we only use specific ‘levels’ of geo information, such as city, region, state, country, while the APIs over at Yahoo will spit out far more levels in-between the ones Flickr uses, as well as deeper down to neighborhood levels, which Flickr doesn’t do (yet).

Just because you get a WoeID back from Yahoo, doesn’t mean we’ll have photos for that specific area, we’ll probably bounce you up to the next largest places we deal with. So our parent hierarchy will have less steps in it that the ones you get back from Yahoo’s geo stuff. It also means our find by lat/long will only go down to town/city level and not precise neighborhood level.

We also don’t do photos or Places pages for Landmark WoeIDs, taking their example of Sydney Opera House which gives you an ID of 28717584 and throwing that at Places will give you no photos (maybe one day), although you can bounce up to Sydney.

Anyway, it goes beyond photos at Flickr, it’s just a really useful way that you, a developer can key off an ID for a place, that someone else, somewhere else can also key off.

If you use the ID 2347563 for California, and they use the ID 2347563 for California. And one, or both of you, publish your information with that ID, then you, they, or other people know that you’re both talking about the same place and match that data up.

Which is nice.

[Update 1: If you’re coming here from the CNET news article my talk isn’t actually about the location platform, its about Flickr photos and location :)]

[Update 2: For more official stuff about the Location Platform, you should probably see the Yahoo Local & Maps Blog post Abstracting Spatial Relationships with the Yahoo! Internet Location Platform where they use phrases such as “unambiguously”, “permanent” and “language-neutral”, and sum everything up far better than I]

[Update 3: and if you haven’t yet checked it out, I still think its worth reading my blog post “Twitter API updates, FireEagle and new Flickr API fun” as an example of what to do with woe IDs]

ALSO from Read Write Web.

Yahoo! today [in May] released a developer preview of its Yahoo! Internet Location Platform, a collection of in-depth geo-location based APIs. We expect to see location be more smartly used in many applications around the web thanks to this platform.

The gist of what’s being enabled is this: applications can provide the name of one location and then the Yahoo! APIs will report neighboring and “parent” locations. Flickr developer and map lover Dan Catt articulates the potential power of the API very well in a blog post today.

A lot of Ground Covered

Yahoo! explains the breadth and depth of location data it now offers thusly: “The [Platform] contains about six million places. Coverage varies from country-to-country but globally includes several hundred thousand unique administrative areas with half a million variant names; several thousand historical administrative areas; over two million unique settlements and suburbs, and two-and-a-half million unique postcode points covering about 150 countries, plus a significant number of points of interest, Colloquial Regions, Area Codes, Time Zones, and Islands.”

Geolocation is Hot Everywhere

Geolocation is hot, a number of new projects are underway to leverage increasingly sophisticated geographic knowledge to deliver value to end users. See our coverage of Brightkite and of Yahoo!’s own excellent FireEagle, for example.

Flickr developer Catt explains, for example, that Flickr could use the new APIs to offer images of nearby photos on several different levels, with accuracy as granular as Flickr is able to output.

There are a lot of interesting possibilities, not just for mapping but for services that are map aware. What would you like to see turned geo-smart? We’re excited to see what developers come up with. We probably won’t have to wait for long, either, since the Platform was released the day before O’Reilly’s Where 2.0 conference begins in Burlingame, California. Keep your eyes peeled for location savvy apps this week!