Overview
Government generates tons of data — code, policies, meeting minutes, news releases, and more. Being able to access and share this data is essential for helping agencies evaluate and improve services to the public. It’s vital that government takes steps to organize data in clear, consistent ways.
Problem
Government data is often siloed, unstructured, not interoperable, nor easily shareable. This disorganization is often reflected back in public information and services — which are too often disjointed, inefficient, inaccessible, or broken.
The result: the collective work of public servants is wasted because users can’t find or access it. This issue is compounded as we increasingly rely on machines such as applications, search engines and artificial intelligence bots to access this information.
Solution
We need to make government data easy to read, share, and use. That means changing how many public sector agencies operate. For data to be open and accessible, we need to consider formats that machines can read easily — like application programming interfaces (APIs) — instead of locked in inaccessible or unstructured formats, like PDFs.
Context
The Open Data Charter, a collaboration of governments and organizations, rightfully sees the public sector as the bridge to our machine-readable future:
“We want a world in which governments collect, share and use well-governed data to respond effectively and accountably to our most pressing, social, economic, and environmental challenges.”
Making data machine-readable is a technical way of saying that information should be accessible to digital systems, such as web crawlers, artificial intelligence robots, or third-party applications. As the Charter’s “machine processable” principle states, data must be “reasonably structured to allow automated processing.”
Serve structured, standardized data
Government heavily relies on static documents, like PDF and Word files. Unfortunately, they are proprietary, non-standard, and unstructured. This makes important information like text, images, and charts inaccessible, especially for machines trying to read and syndicate across multiple channels.
An example of structured, standardized data is the Schema.org protocol. Search engines use this to display information in search results. AI chatbots use it to better deliver prompt responses.
Adopting a system of structured, standardized data allows for more seamless integration across internal and external systems.
Build single sources of truth
Government data is scattered across many departments and systems; some are self hosted while others are vendor hosted. There is often duplicate data that leads to confusion as to what’s most accurate and up-to-date. This is compounded when multiple data sources with the same information are delivered to different endpoints, such as a website or application or internal business intelligence systems.
The idea of a single source of truth means there should be one authoritative dataset per domain (meeting agendas/minutes, municipal code, financial transactions, news, budgets, etc.). This guarantees clear data ownership and governance rules.
Go bot gov
The entire organization must be on board with machine-readable public service.
Agencies need to appoint a primary data leader and train staff on data principles and best practices.
Contracts must require data standards and API-first or direct access to structured data in vendor systems, and that quality assurance systems are put in place to validate compliance. Public records policies should default to machine-readable responses and let requesters specify format preference.
Governance frameworks like Open Data Charter’s principles and data integration protocols like Data Management Association’s Data Management Body of Knowledge can help guide public sector data into the future.
Data integration practices must be adopted. Examples include:
- JSON and XML for data exchange
- SOAP/REST APIs for systems integrations
Data in open formats that move seamlessly between systems future proofs government against technology changes, but also opens it up to new opportunities.
Machine-readable government means public sector organizations adopt a culture of data. Staff and vendors are knowledgeable and aligned on principles and practices of managing and making government data standardized, structured, interoperable, and accessible.
Open access to government information — directly to the public and to the machines that we rely on — is essential. It will allow people — both in and outside of government — to connect dots in ways never before imagined.
Mantras
- Feed the machine
- Go bot gov
Checklist
- Inventory data systems.
- Evaluate/standardize data structures for each.
- Streamline data systems for internal and external sharing.
- Protect data ownership and privacy.
- Automate, establish repeatable builds and integrations.
- Build for interoperability by adopting an API-first approach.
Questions to ask
- Is one person responsible for and empowered to lead a data-driven culture?
- Are all staff trained with basic awareness on data principles and best practices?
- Do contracts address data standards and rights?
- Have you identified internal and external stakeholders who will use the data/API’s?
- Is there a strategy for keeping data sustainable and relevant over time?
