OS Locator validation mapping for UK OpenStreetMap data

ITO have produced a mapping layer which can be used within OpenStreetMap editors to check for differences between OpenStreetMap data and Ordnance Survey Locator (which has been released as Open Data).

Within the OpenStreetMap editor the new layer appears as a series of boxes with associated names for all streets that are in OS Locator but which are not in OSM within the same bounding box with a small buffer area added.



Normally a box in this layer indicates an error or omission in OSM, however there are also some errors in the OS data. The not:name can be added to the OSM data in these situations and the box will not appear in subsequent analyses and there is less risk that someone transcribing incorrect OS data into OSM at a later date.

An example of an error in OS Locator is the mispelling of Arthur's Terrace in Ipswich has been entered as 'Athurs Terrace'. I have added a tag 'not:name=Athurs Terrace' to the street and it now on longer shows on the analysis.

How many errors are there in OS Locator? I have been through data for Ipswich and have found 7 errors where OS Locator data does not match the relevant street sign. I have added not:name tags to all of these, follow the links to see how they work. It is also useful to add a 'note' field with a description of why the not:name tag has been added.

1. Berry Road (should be Belfry Road)
   
2. Gwendoline Road (should be Gwendoline Close)
   
3. Lagonda Drlve (the 'i' in Drive has been replaced with a '1')
   
4. The Garrods (should be Garrards)
   
5. Inglelow Gardens (should be Ingelow Gardens)
   
6. Athurs Terrace (should be Arthur's Terrance)
   
7. Fen Bright Circle (it is actually Fen Bight Circles)

In addition to the above I am investigating a number of other potential errors at present which will probably add another 5 or so errors to the list creating a total of 12 out of some 3000 named road names in the town meaning that OS Locator is 99.7% accurate which isn't bad. This test does not of course show the streets which are in OSM but missing from OS but that is a separate test for another day and OSM should perform pretty well on that one.

To access the above mapping from Potlatch add the following string of text into the 'custom' layer box.

http://tiles.itoworld.com/os_locator/!/!/!.png

As a technique I would recommend doing a first pass getting rid of the bigger and the less complicated boxes and then do a second pass on the remaining smaller ones as it can be difficult to make sense of multiple overlapping boxes in one pass.

The analysis is current run each night based on the Geofabrik GB data published earlier the same day so changes to OSM will show up in the map rendering not the next day but the day after that. You may therefor choose to do a pass across an area one day and then do a second pass a couple of days later.

Also, do note that the OS bounding boxes seem to have a small rounding error resulting in edges of the bounding box that do not always exactly match the end of the relevant street.

TED, Tim Berners-Lee & ITO - The Video

Its taken a little while for us to publish this, but I'm sure you will agree it was worth the wait. Here is Tim Berners-Lee showing off some of our visualisation magic at this year's TED conference in "The Year Open Data Went Worldwide":



Now that's story telling with data.

OSM Mapper Is Receiving Some TLC

As some of you have noticed, OSM Mapper isn't updating right now. Why you may ask? Well the OSM planet file was broken for a week, which is essential so we couldn't do much about that, and take a look at this:

Thanks to the growth of the OSM community, the data is growing exponentially. We are amazed and delighted by this growth, but it does give us some development challenges with our global OSM model. We've taken the down-time as an opportunity to take a look at how we process OSM data.

Fear not, Hal is working on our new and improved global OSM model. We'll let you know when the new model is in operation and OSM Mapper is updating again.

Posted via email from Christopher's posterous

ITO World at TED 2010 - Project Haiti

We are proud to announce that our work has been displayed at this year's TED conference, where the world's greatest thinkers and doers come together. We were asked to provide visualisations that helped tell the story of open data in 2009, for Sir Tim Berners-Lee to give an update on open data since his talk last year.

We chose to tell the story of 'OpenStreetMap - Project Haiti'.

We all followed the crisis that unfolded following the Haiti earthquake, many of us chose to donate money, a few were flown out and deployed as part of the relief effort. But what practical impact can many have without being there in Haiti itself? Well, during this crisis a remarkable story unfolded; of how people around the world could virtually collaborate and contribute to the on-the-ground operations.

With the little existing physical, political and social infrastructure  now destroyed or damaged, the situation was especially challenging for aid agencies arriving on the ground. Where are the areas most in need of assistance, how do we get there, where are people trapped under buildings, which roads are blocked? This information is important to the rescue agencies immediately after the event, and to the longer rebuilding process. In many developing countries, there is a lack of good mapping data and particularly after a crisis, when up-to-date information is critical to managing events as they evolve.

Enter OpenStreetMap, the wiki map of the world, CrisisMappers and an impromptu community of volunteers who collaborated to produce the most authoritative map of Haiti in existence. Within hours of the event people were adding detail to the map, but on January 14th high resolution sattelite imagery of Haiti was made freely available and the Crisis Mapping community were able to trace roads, damaged buildings, and enter camps of displaced people into OpenStreetMap. This is the story of OpenStreetMap - Project Haiti:


OpenStreetMap - Project Haiti from ItoWorld on Vimeo. Supported by Ideas In Transit

Each flash represents a new edit into OpenStreetMap, and this visualisation is a vivid picture of how much work was contributed by volunteers, following the quake. First the primary and secondary roads (green and red) are added and then smaller residential streets and many other features such as the blue glowing camps of displaced people that emerge.



Its not all about OpenStreetMap however, a variety of organisations provided technical solutions with vital services, such as Ushahidi - a service where anyone in Haiti can send an SMS message with their location and report an emergency or need. All this information is then aggregated and publicly displayed in near real-time, on a platform that almost anyone can use.

With all the talk of open data and the launch of Data.gov.uk, its very easy for us to get excited about the release of huge tranches of government data, but means little to anyone else. We now have the potential for many new services that will bring tangible benefits, in this case for the crisis management of the Haiti Earthquake. After the crisis, thousands of people from all over the world were able to contribute and build what is now the most complete map of Haiti - in use on the ground by the United Nations, Ushahidi, and the default map of choice for NGOs. This picture says it all:

OpenStreetMap being used on a GPS unit for search and rescue in Haiti by Fairfax County Urban Search & Rescue Team

What does all this have to do with ITO World and transport? Well, we want to visualise and tell the story of transport - how does the network function, what is the story of your commute, how will this planning decision affect transport in my city? We are working on powerful analytical tools that will visualise the current status, and future scenarios of transport networks enabling us to make better decisions. By creating a narrative around transport, people can understand how and why decisions are being made, and collaborate to build a better transport system that meets their needs.

Something's Brewing in the Ito Labs

Posted via email from Christopher's posterous

Video - Storytelling with Visualisation at the Data.gov.uk Launch

We whipped up a storm at the Data.gov.uk launch, demoing our Local Transport Posters and storytelling with visualisation.

See the video here:


Data.gov.uk Launch from ItoWorld on Vimeo.

I would like to point out that I don't usually scratch my nose so much, but sharing a stage with Sir Tim Berners-Lee is rather nerve-wracking!

Interested in the posters or our visualisation expertise? Contact sales@itoworld.com

Mapping the Crisis - OpenStreetMap Response to Haiti Earthquake

Since the Haiti Earthquake, there has been an incredible response from the OpenStreetMap community to map Haiti for use by aid agencies on the ground.

Before the quake, Haiti looked like this.



And now looks like this.


View Larger Map

During the response, it was clear that better coordination tools were needed. With regards to OpenStreetMap editing, a tool to show where edits were occurring and where further efforts should be focused was necessary.

Our own OSM Mapper, currently runs on a 48hour update cycle and was of little use for coordinating live, crowdsourced map edits. We hope to be running on a much shorter update cycle in the future, which should be of more use for coordination during future events. However, it does give us some useful insight after the event.



Looking at the OSM edits up to midnight 15th of January, red edits are the most recent. We can see a focus on Port-au-Prince and surrounding urban areas, which are presumably the worst affected. However, as relief efforts focused on the capital, access and awareness of the surrounding localities will be limited. These areas need to be mapped too and some mappers encouraged to make edits away from the main urban centres.

Looking through the tagging, 147 spontaneous camps of people displaced by the earthquake had been identified and tagged on the map. OSM Mapper has a handy KML export option, allowing you to focus on just one tag and export all the relevant features from OpenStreetMap in KML.


You can download the KML file of spontaneous camps here.

We are looking at running an update, and will get some images online to show progress after 15th January.

Official NaPTAN bus stops data available in OpenStreetMap

We help run a research project in partnership with the DfT, Ideas in Transit; applying user innovation to the transport challenges faced by individuals and society.

Ideas in Transit has been supporting the import of the UK’s official database of 360,000 bus stops and other public transport access points (NaPTAN) into OpenStreetMap over the past year. Data for Birmingham was imported in March 2009, followed by Greater London, Suffolk and Kingston-upon-Hull in August 2009 and subsequently a further 51 authorities.

We would like to highlight the great work done in the OSM community by Thomas Wood, Christoph Böhme and Chris Hill. Thomas Wood is responsible for importing the data into OpenStreetMap, Christoph created the NOVAM viewer which allows mappers to check if an imported bus stop has been surveyed:




Lastly, I hear word that Chris Hill has been surveying Hull and talking to Hull City Council's transport team about how to feedback the crowdsourced improvements from OSM back to the NaPTAN dataset.

This is a wonderful example of how Public Sector Bodies can open up their data and engage with communities, for mutual benefit. Local transport users get better personalised transport information from services using OpenStreetMap data, and Local Authorities can receive crowdsourced improvements to their data.

This initiative would not have been possible without the support of Transport Direct and National Traveline who offered the release of the NaPTAN dataset. Ideas in Transit will continue to support the project over the coming years with support from the Technology Strategy Board, the Department for Transport and the EPSRC.

We have upgraded OSM Mapper!

We have just upgraded OSM Mapper. There is a new user interface with some great new features. We have increased the maximum size of areas by a factor of three and have made it easier to manage longer lists of areas. You can now save your favourite views into the data and export data as KML.

Better handling of areas
Areas used to be listed across the screen in the order in which they were created and as the list grew the map would be pushed down the page. The areas are now available from a selection list and the processes of editing areas, creating new areas and deleting areas have been cleaned up. You can also set a default area.

Views
Using the new view feature you can save many different, er, views of an area. There are some standard views such as 'sessions', 'users' and 'tags'. You can also add your own views by selecting the information you are interested in and saving it with a suitable name. Check out this short tutorial to see how easy it is to create areas and views and then use them for real.


Using OSM Mapper - An Introduction from ItoWorld on Vimeo

Advanced views
Ok, so that covers the basic stuff about creating and using areas. The following tutorial shows how to create some more advanced views into the data, including looking at who has been editing particular features, taking RSS feeds for selected features and creating list of names for waterways and streets. It shows how to set your OSM username from the profile page in order to use the 'filter by me' and 'filter by not me' feature.


OSM Mapper - Using Advanced Views from ItoWorld on Vimeo.

KML Export
A powerful new feature is the ability to export selected data as KML. Select an area and a view and then select 'Export > Map as a KML file' from the menu above the image. Give it a name, save it to your computer and then check it out in Google Earth. The image below shows some schools in Bristol in Google Earth. One school is shown in yellow, and the rest in blue.



You can also check out a KML view of London railway network and the UK Railway network

Have fun and tell us what you think. If you spot any problems then definitely tell us what's up and we are ready to get them fixed.

OSM Mapper Now Available for the USA

OSM Mapper is now available for the United States (and everywhere else). To find out more about OSM Mapper read our initial blog entry. Take a look at where work is being done across the USA. In the image below the bright spots show areas with recent edits and the background colours show when the main TIGER data was imported.



Looking more closely you can see a lot of activity in New York:



It is also possible to see who has actually been doing the work (click on any image to access a more detailed version):



Across on the West Coast there has also been a lot of work in the Bay Area:



Around New Orleans:



And many other places. Why not subscribe for free and find out what is happening your area?

This product was developed with support from Ideas in Transit.

Expanded Coverage for OSM Mapper

OSM Mapper coverage has now been extended to include Africa, the Middle East, Asia, Australasia, South America and Canada. The only area still missing is the USA which will be added at the next stage.

OSM Mapper is a product (free to use after registration) that allows users to analyse and view data for areas of interest, but until now coverage has been limited to Europe. To find out more about the product's features, see the original announcement.

Here is an image showing nice team-work in Melbourne, Australia.



The following images show examples of recent activity around the world. Please note that currently only ITO staff can create such wide-area images.

Zooming out from Melbourne we can see where there has been recent activity across Australasia.



Head north and see what been happening in Japan, parts of China and North and South Korea.



Moving west we can see what is going on in India. The big bulk upload of trunk roads arrived in February 2008, courtesy of Automotive Navigation Data. Since then there have been some busy areas, possibly due to the series of mapping parties held in India.



Moving further west we can see activity across Cyprus, Lebanon, Israel, Egypt and Palestine with a lot of progress in the past few months.



Skipping existing coverage in Europe, we can see Canada - with lots of activity in the more densely populated South.



Heading down to South America, we can see that clearly some areas are doing a lot of work.



Finally, if the place you are really interested in is the USA then subscribe to this blog and we will post here as soon as we are set up to handle all the TIGER data.

If you want to find out more about the product and how to subscribe or use it then read our original announcement.

We would like to thank Geofabrik for providing free access to their daily version of the OSM datafile.

This product was developed with support from Ideas in Transit, a five-year research project following user-led information technology innovation in the transport domain funded by the UK Department for Transport, the Technology Strategy Board and the Engineering and Physical Sciences Research Council.

'OSM Mapper' for OpenStreetMap Contributors

I am just back from the amazing second State of the Map Conference in Limerick at the weekend where we showed our new free product, OSM Mapper, that allows contributors to OpenStreetMap to monitor and improve the quality of data in their areas. We got the conference delegates to try it out and they seemed to be pleased. The few initial snags are now resolved and the servers are holding up well so it is time to open it up more widely.

Since the conference we have expanded the area available from the UK to the whole of Europe, and have moved from a weekly to a daily update. The next challenge is worldwide coverage which we will do as soon as we are confident with Europe. Btw, the new version of Potlatch that went live today means that it is now possible to click on the 'Potlatch' link from ITO Mapper and enter Potlatch with the relevant feature already selected.

Using this product you can set up one or more areas to monitor and analyse in a variety of ways. You can even establish RSS feeds to get alerts when other people make changes within areas of interest. You can also generate beautiful images to download and post anywhere as (CC-by-SA).

You can show the main contributors to the map using a view colour-coded by contributor. This image shows the top contributors in the Paris area.



You can also see how long it has been since different parts of an area have been modified over various time frames. The next image shows changes in Berlin in the past month.



You can focus on a particular aspect of the data, eg railways, waterways, schools etc. This map shows the various railway systems in London.



You can subscribe to an RSS feed for recent changes to railways in London. As the feed contains GeoRSS data, you can also view the feed in an appropriate viewer like Google Maps.

With filters, you can select features that are missing data. For example, this image shows a breakdown of unnamed residential roads in North London.



You can check for changes in an area by other people. This image below shows edits made in Ipswich by people other than PeterIto.



An RSS feed is also available to monitor the area above for changes by people other than PeterIto.

This image shows where we currently offer coverage (and also shows recent edits in the area).



Why not subscribe to our product, check out a tutorial slide presentation, or look at various images on Flickr?

There are a number of current limitations that will removed in due course: The coverage is Europe only. There is an size limit when selecting areas; this is large enough to cover most European cities - however the ability to generate images of very large areas, such as whole countries, is restricted due to the high processing times involved. Updates are normally daily and are unlikely to become more frequent just yet. It doesn't pay any attention to point features, only ways, so it is good for analysing roads and parks etc but not letter boxes and bicycle parks. We intend to offer functionality to allow people to share areas; you can do that now by sharing URLs, but we will implement a more formal approach in due course. Finally, deleted features are not picked up in the RSS feeds so you won't be warned if someone has deleted a feature, however big it is! Stay tuned to this blog for information about future enhancements.

Please give us feedback by commenting on this post, using the comment link at the bottom of each page in the product, or emailing support 'at' itoworld.com.