Conclusions of civil society and public sector policy discussions on data use in government

Download: Conclusions of civil society and public sector policy discussions on data use in government [.pdf]

Download: Conclusions of civil society and public sector policy discussions on data use in government [.docx]

Download: Annexes [.docx]



Since March 2014 civil society organisations, privacy groups, officials from a number of government departments, academics and representatives from parts of the wider public sector have been collectively discussing how government can be made to be more efficient and effective through its use of data. The core focus has been to enhance the availability of high quality research and statistics from administrative data; prevent fraud and help citizens manage the debt they have with government; and ensure the right services are offered to the right person at the right time. The findings set out in this document do not represent the views of HM Government (HMG) but summarise the conclusions of those individuals who participated in discussions. Given the large range of people and organisations involved, the findings and recommendations set out in this paper are important and will inform future work in the area, which may involve further consultation.

Work on this policy initially began using conventional Whitehall policy development approaches, with a focus and presumption that data sharing offered the solution to its objectives. However, it became apparent that determining where the balance between the proportionate use of data to deliver services and maintaining people’s privacy would be a sensitive and difficult exercise to do in isolation, particularly because of the variable levels of citizens’ trust of government on the subject of data. An open policy making (OPM) approach was adopted to ensure there was a shared understanding of both what government wanted to achieve and the concerns of those outside of government.

Involve, a not for profit organisation established to improve public engagement with government, were asked by the Cabinet Office to work collaboratively on an open policy making process on this issue. Involve agreed to help facilitate the process and external engagement. Since then over two hundred organisations have been invited to participate based on their particular interest in the issues considered. The OPM process has been open to any interested organisations to join. The number of non-government individuals and organisations willing and able to participate in the process was fewer than anticipated. Resourcing issues, particularly where the organisation is relatively small, has been a key factor. Groups engaged in the process included those with a specific interest in individual privacy and rights, academics, statisticians, researchers and their funders, charities, government officials and some private sector organisations. Representatives from these organisations and individuals participated in discussions and the development of proposals in each of the themes. The scope of this new joint work was expanded to include looking at alternative solutions, not just data sharing. The task set was to examine the evidence and use the analysis to inform the design of policy proposals from it. The process was designed to ensure that all voices were heard from the outset, increasing the likelihood of balanced, successful policy recommendations being delivered.

Transparency has underpinned the whole process, with all work as open as possible. Key information and updates have been posted on, a non-government website, to act as a repository and audit trail of the work.

The following key principles have underpinned the discussions:

    1. Proposals should not consider the building of new large and permanent databases, or collecting more data on citizens;
    2. Proposals should avoid the indiscriminate sharing of data within Government; and
    3. Proposals should not weaken the Data Protection Act.

The findings set out in this paper offer a balanced consideration of options by the individuals involved, including more data sharing as a potential solution to the specific challenges we looked at. It is the product of a truly open collaboration between a range of public sector officials and civil society organisations and privacy campaigners. In some instances, the consensus among the group was that legislative changes along with appropriate safeguards would be the best course of action to achieve the objectives. Whilst with other issues, the consensus was that data sharing was not the best solution. These balanced recommendations demonstrate the value of the open policy making approach and collaborating with a range of organisations with different perspectives.

There remain some areas, particularly around safeguards that will need to be in place where data is to be shared, where further and wider consultation is required, as consensus could not be achieved. Government would wish to consult further on the full range of policy proposals before deciding which to take forward and in what exact form.

Cabinet Office is engaging with officials from devolved administrations in Wales, Scotland and Northern Ireland to determine the cross-border implications of the proposals and to consider whether there would be scope or demand for some of the policies to apply in devolved nations either identically or with particular differences. Devolution issues are complex because of the differing areas of competence in the settlements and differing functions of bodies working in devolved areas. We will consider findings as they emerge and identify where there is scope for cooperation to achieve agreed outcomes across different parts of the UK.

Open policy making still represents a new way of working. Taking this approach to an issue as challenging as data sharing builds on the excellent work delivered by using this approach to develop the UK’s 2013-15 Open Government Partnership national action plan. This process provides further evidence of the value of working in partnership to look at some of our biggest challenges.


The OPM process has been extensive, with engagement taking place at individual and group levels, either for individual specific policy challenges or across the policy areas. Over twenty sessions were run with a large number of representatives between April and the end of 2014. Representatives from both within and outside government have listened and changed their position where the case has been compelling. To ensure transparency, all key discussion documents were posted on, a non-government website, and invited comments from the general public.

The following three specific policy challenges were explored as part of the open policy making process:

  • Research and statistics – improving the quality of statistics and enabling the availability of better evidence to inform the formulation of policy and delivery decisions;
  • Fraud, error and debt – saving taxpayer’s money wasted on fraud and error and provide those citizens with multiple debts to government greater support to help manage their debts; and
  • Tailored public services – improving the tailoring of public services so that the right services are offered and provided to the right person at the right time.

A brief summary of the OPM group’s key recommendations are:

Research and statistics

De-identified data

Representatives from both within and outside government recognised the need for public bodies to be able to link data for research purposes. Representatives were supportive of a proposal, provided data linking was carried out using a Trusted Third Party sharing system. This process uses significant procedures to de-identify data so that it can be linked in a secure access facility and made available to researchers under controlled conditions. Trusted Third Parties, researchers, and the subject of the research would all have to be accredited by an accreditation body under a system established through legislation. Any research intended to make use of this system under the legislation would have to satisfy the specified condition that the research ‘is in the public interest’. Extensive consultation via the open policy making process led to consensus on the aims and proposed powers. It was proposed that specifically defined health services and social care bodies would be excluded from these proposed new powers.

Identified data

Participants in the open policy making process considered a specific proposal from Government to enable public authorities to disclose data to the Office for National Statistics (ONS) in order to allow it to carry out the executive functions of the UK Statistics Authority (Authority) to provide statistics that serve the public good. The proposal would replace the current arrangements whereby the Minister for the Cabinet Office provides an information gateway through regulations, which are approved by Parliament through affirmative resolution process. This would reduce the burden on businesses and other respondents by reducing the cost of surveys as well as time taken by respondents to complete them; improve policy making decisions based on research and statistics by strengthening the evidence base for policies; improve the quality of statistics, while preserving the privacy of data subjects and ensuring that data are used appropriately, by ensuring that safeguards are embedded in the process. It was agreed that changes to legislation would be required to meet the identified objectives.

The group identified alternative options for the scrutiny of proposals for disclosure of administrative identified data to ONS, with advantages and disadvantages, but there was no consensus on any of the options. There have been strong representations from some elements of Civil Society that the Parliamentary process described above for opening new gateways should be sacrosanct.

HM Revenue and Customs (HMRC) general and aggregated and de-identified data

Participants in the open policy making process agreed with a Government proposal to reduce the restrictions around the disclosure of less sensitive general, aggregated and individual level de-identified HMRC data for public benefit. Legislation limits the circumstances in which HMRC may share information. Most other central government departments are not subject to equivalent restrictions, and thus this proposal helps achieve greater equality for HMRC, enabling it to contribute to a wider range of government initiatives and academic research projects than at present. Representatives from within and outside government agreed with the aims and the rationale for legislative change. HMRC has already consulted on this proposal.

Fraud, error and debt

Representatives from both within and outside government called for more robust evidence to be gathered on a range of fraud, error and debt issues (from the scale of the problem to the value of different types of intervention) from which assessments of potential options could be made.

It was agreed that a set of pilots be developed, some requiring new legislation, to explore the value of interventions through data sharing. Pilots would include feedback mechanisms to provide insight into the demand for data sharing, and citizen attitudes to data use to counter fraud.

Consensus between participants was that it may be difficult to determine a clear line between fraud and error, making any work that specifies error separate to fraud potentially uncertain and therefore less likely to be successful. It was also agreed that non-legislative avenues may be able to address error. Further, it is likely that an amount of error will be reduced through the increased data quality that work to reduce fraud would help to deliver.

OPM Group participants agreed that there are benefits to the debtor from a reduced number of approaches and a combined management of debt. The Debt Market Integrator (DMI) work will provide a single point of access to a wide range of debt management and collection services. Later developments could include a single view of debt and supports debtors through the development of a single payment plan. Whilst some preferred a legislative approach, the group were keen that a consent-based approach be tried first.  This consent-based approach would be reviewed and a decision would be made as to whether it continues or whether more formal intervention is required.  To manage the risk of wasted revenue due to delay should this process be necessary, the group have proposed that parallel preparatory work on legislation should be carried out whilst the consent-based work is assessed.

Tailored public services

Participants in the OPM process agreed that there was value in exploring data sharing to support the delivery of public services better tailored to individual need.

The initial proposal was for specific gateways to address specific issues. Through an iterative policy development process, which explored a range of social policy areas, the group have developed and reached agreement on recommending a broader but constrained power. This would be a permissive power for defined public bodies to share data with defined public bodies for the purposes of improving the delivery or targeting of public services, in specified areas of social policy, resulting in an offer of help to an individual. Examples of specific objectives which would meet the criteria for the power, or not, are detailed below.

This power would be intended to operate in accordance with a number of principles:

  • The intent of the power is to help individuals by endeavouring to ensure that they are offered the right intervention at the right time.
  • The purpose of the data sharing would be to benefit individuals, not to punish them or do anything to their detriment.
  • Data matched but not used for the purpose described in the business case would be disposed of according to relevant information governance processes and not used for any other purpose.
  • The business case (for the specific data share) includes – under the objective – details of the intervention that will enable the achievement of the objective.

Furthermore, discussions around the specified bodies that would be designated resulted in restricting these to public bodies. It was felt that this would result in significant improvement in the delivery of services to citizens while protecting citizen data.


Throughout this process we have considered and challenged whether data sharing and legislative solutions are required to achieve the desired outcomes or whether other approaches could be taken. We believe these recommendations reflect this approach. They adhere to our principles that we would not develop solutions which require the building of large permanent databases or collecting more data on citizens, or weaken the protections afforded by the Data Protection Act.

Where a solution, which requires changes to data sharing legislation, has been recommended, it is supported by measures that provide appropriate safeguards to protect the privacy of citizens. The need to be transparent has been at the core of the discussions and as a result options include making privacy impact assessments available for public scrutiny. In developing these proposals we have sought to balance a consistent approach across the different areas with the necessary tailoring to ensure that the unique features of each area are addressed appropriately, informed by the broader framework within which they fit.  This has led to a few key differences in approach being taken across the three strands.

4 thoughts on “Conclusions

  1. Sonia Ogier

    When was the “Conclusions of civil society and public sector policy discussions on data use in government” report published?

  2. Shelagh Lyth

    Dear Sirs,
    Please could I ask whether you considered for example having a specific statutory power to support the provision of the Troubled Families Service by local authorities?


    Shelagh Lyth
    Solicitor employed by Rochdale MBC

  3. Pingback: The Four Horsemen of our rights Apocalypse |

  4. Pingback: The Four Horsemen of our rights Apocalypse | infinite ideas machine

Leave a Reply

Your email address will not be published. Required fields are marked *