The Open Data Institute states that “Open data is data that anyone can access, use or share”, a concise definition which perhaps over simplifies some of the challenges that we face with data and its restrictions be that tech or legal.
Technologically we have come a long way from the early days of data.gov and the metadata challenges of the European Inspire Directive
Our Tech Stack
Here at Stockport we have built a common Tech Stack enabling us to extract, anomalies and format data automatically and serve that data through both internally hosted web services and Amazon Web Services
This same Tech Stack drives the Extract Transform Load (ETL) processes that help deliver digital services such as Report a Faulty Street Light and builds the Council’s Business Intelligence Warehouse.
The creation of open data should not be an additional burden upon resources but should be a natural extension of what we already do and this is what our common Tech Stack allows.
Datasets that we feel can be pushed out as open data or pushed out to the wider community can be directed into the open data ETL pipeline.
We now need to look into how we inform developers and the community of what data we hold, we need a solution that we can connect to our existing Tech Stack, that we can populate automatically and which does not need a resource to manually publish metadata into.
The need for a consistent national data schema
This in turn leads us to one of the biggest challenges – the lack of a National Information Infrastructure. One of the key aims of Open Data is to drive service innovation and economic growth, but for this to happen we need a common data schema and a National Information Infrastructure that all organisations can conform to; that tech leads can tap into and be sure that the data from London is the same as the data from Stockport; so that when they develop an innovative solution they know its scalable at a national level.
As well as offering data to the developer, we need to enable and empower the community to use and analyse our data, we need to provide the data to support community groups in developing funding baseline and provide them with the tools to monitor the impact of their work within the community. This is a wider challenge, even tech companies such as Airbnb acknowledge that there is a data analytics skills gap which is why they are investing in a data skills university for its workforce.
What’s the latest?
On the 22nd November Philip Hammond in the Autumn Budget announced the creation of the “Geospatial Data Commission to develop a strategy for using the government’s location data to support economic growth.”
This simple statement will have a massive impact upon the quality and accuracy of data that we can provide. Data and derived data which is currently restricted under crown copyright will be unlocked and made available to the wider community.