Companies have been analyzing their internal data for years, but as free, publicly-available data sets open up, they are finding themselves able to focus the lens.
Business intelligence and data analytics is a common theme among smart companies that want to get a competitive advantage in their market. But proprietary data isn’t the only way to do it – increasingly, free public data is making analytics more effective, and giving commercial organizations the chance to find new insights. Welcome to the world of open data.
Open data is freely available data, available to people without restrictions on usage or publication. Typically, it is provided by governments, who are the biggest collectors of data. Increasingly, public sector institutions are publishing vast swathes of public data online, along with APIs that allow companies to do what they wish with it.
billions of dollars
It’s about time. Estimates suggest that billions of dollars-worth of data has been locked up inside public sector vaults, which could bring significant benefits to commercial organizations, if only they were allowed to use it. In the UK alone, Deloitte has estimated that public sector information could be worth between £1.8bn and £2.2bn,
“This open data provides an opportunity for businesses to blend with their owned data to gain more insights, and are using software and tools to understand the data available and therefore be able to use it in the best way for their business,” says Maxime Marboeuf, an analyst at Tableau Data, a data analytics tools company that also publishes a free tool for interpreting open data.
Recently, the Open Data 500 was published, which is a list of 500 US companies doing innovative things with that data to drive economic activity. The most common type of company to use this data specialized in technology, while finance and investment firms ran a close second. Business and legal firms, and companies specializing in governance, came third and fourth. Many of these firms found themselves using data from multiple agencies, the Open Data 500 survey found.
how open data helps
What kinds of applications are open data used for? The Outside View, a UK-based consulting firm that specializes in predictive analytics, was able to help one large property client to maximize its business by targeting those people most likely to sell their houses, explains CEO Rob Symes.
“The biggest problem in London is that they don't have enough houses to sell,” he explains. It is difficult to be targeted when sending out letters inquiring whether someone would be interested in selling their house.
The UK’s Land Registry recently opened up a tranche of new data for public access, including transaction data on house sales, payment price data, and a house price index. “We can look at a probability model on how likely it is for Joe Bloggs in Croydon to sell a house,” Symes says. “Then we build a persuasion score. How likely are they to be persuaded to sell by this property company?”
The availability of more free data from the Land Registry is just part of a broader movement, in which governments are gaining a better understanding of the need for open data, and are responding. In the UK, the government bought together Companies House, the Land Registry, the Met Office, and Ordnance Survey under the Public Data Group, an organization tasked with making high quality data available for both businesses and the public sector. This complemented an existing scheme, the data.gov.uk web site, launched in 2010, which now offers over 9,000 data sets for public use.
In the US, things are moving ahead in lockstep. The White House launched its own open data initiative in 2011, and is now on its second action report. It is reforming its Data.gov open data web site, which already has over 90,000 data sets.
healthcare possibilities
One of the biggest areas where open data has opened up opportunities is healthcare. The MedRed BT Health Cloud, a partnership between BT and US-based MedRed, a company that specializes in healthcare software and services. The cloud is a collection of aggregated data from more than 50 million people, spanning the UK and the US, going back up to five years. Pharmaceutical companies and universities are using the data to help develop new drugs and therapies, and establish best practices in healthcare.
Healthcare is one of the most promising areas where open data may be used, says Symes. “Government data is useful here because we can look at physical patterns of illness. We fuse that with proprietary data and we can start making predictions,” he says.
But for that to happen, governments must begin to unlock that information. “A lot of data has been wrapped up in the NHS's files. It's kind of like oil. If you don't have the tools to get it out, you can't do anything with it,” Symes continues.
The march towards broadly available public sector information continues, but it is a long, slow effort. As governments unlock it, companies will find new ways to marry their proprietary data with publicly available information, accessible in machine-readable format. Used intelligently, this promises to focus the lens on everything from customer demographics to environmental behavior. And as big data initiatives continue to gather traction, large, publicly available datasets can only accentuate their results.
Read our paper on big data to find out about how analytics can help your business.