Freedom of Information requests to the Climatic Research Unit

From 1978 onwards, the Climatic Research Unit developed its gridded CRUTEM data set of land air temperature anomalies back to 1850, based on instrumental temperature records held by National Meteorological Organisations around the world, often under formal or informal confidentiality agreements that restricted use of this raw data to academic purposes. Beginning in 1991, Phil Jones of CRU discussed data with Warwick Hughes (later of the New Zealand Climate Science Coalition). From 2002 onwards, Jones received requests from Stephen McIntyre for instrumental raw data (McIntyre became prominent in contrarian disputes over the methodology of paleoclimate hockey stick graphs shown in the IPCC Third Assessment Report of 2001). Jones says that at first he tried to meet McIntyre's requests, but he soon became inundated with requests that he could not fulfill due to time or confidentiality constraints, and began refusing requests.[1]

The new UK Freedom of Information Act came into effect in 2005, and in February of that year Jones discussed with fellow climate researchers the potential implications of the Act for McIntyre's requests. In 2007 he told colleagues that, having seen what McIntyre's Climate Audit blog was doing, UEA had been turning down FOIA requests associated with the blog. The scientists concerned saw such requests as disrupting the time available for their work, and those making them as nitpicking to suit an agenda rather than trying to advance scientific knowledge. Late in 2008, the university's FOI managers took advice from the Information Commissioner's Office (ICO) on exceptions allowing refusal of requests.[1]

McIntyre complained that data denied to him had been sent to Peter Webster at the Georgia Institute of Technology, who was working on a joint publication with Jones, and between 24 and 29 July 2009 the university received 58 FOI requests for raw data or details of the confidentiality agreements from McIntyre and his readers at the Climate Audit blog.[1] On 24 July Jonathan A. Jones of the University of Oxford made a FOIA request for the data that Jones had sent to Webster, the UEA refused this request on 14 August. Don Keiller of Anglia Ruskin University in Cambridge then made a similar FOIA request, UEA refused this on 11 September 2009.[2][3]

On 12 August 2009, Nature News published a statement by Phil Jones that he was working to release the raw data in a systematic way, and was writing to all the National Meteorological Organisations requesting their agreement to waive confidentiality.[1] In mid October CRU issued a statement on data availability, describing how National Metereological Services (NMSs) and scientists had given or sold them data with written or verbal agreements that it must only used for academic purposes, and not passed onto third parties. There were difficulties in separating out raw data, some of which was subject to charges made by NMSs, and "These data are not ours to provide without the full permission of the relevant NMSs, organizations and scientists." They hoped to obtain consents and to publish all the data jointly with the Met Office.[4]

On 24 November 2009, four days after the start of the Climatic Research Unit email controversy, the university stated that over 95% of the CRU climate data set had already been available for several years, and the remainder would be released when permissions were given.[5] The university worked closely with the Met Office, which sent requests to National Meteorological Organisations for agreement to waive confidentiality on raw instrumental data,[6] as CRU had announced on 12 August 2009.[1] Some gave full or conditional agreement, others failed to respond, and the request was explicitly refused by Trinidad and Tobago and Poland.[3]

In discussions with the ICO about the FOIA requests which Jonathan Jones and Don Keiller had made before the email controversy had begun, the university argued that the data was publicly available from the Met organisations, and the lack of agreement exempted the remaining data. In its decision released on 23 June 2011, the ICO stated that the data was not easily available and there was insufficient evidence that disclosure would have an adverse effect on international relations. The ICO required the university to release the data covered by the FOIA request within 35 calendar days.[3] On 27 July 2011 CRU announced release of the raw instrumental data not already in the public domain, with the exception of Poland which was outside the area covered by the FOIA request. The data are available for download from Met Office website and from CRU. The university remained concerned "that the forced release of material from a source which has explicitly refused to give permission for release could have some damaging consequences for the UK in international research collaborations."[2][6]

In May 2008 David Holland, an electrical engineer from Northampton, made a FOI request for all emails to and from Keith Briffa about the IPCC Fourth Assessment Report (AR4); formal review exchanges had already been published. The University of East Anglia (UEA) information policy and compliance manager (IPCM) refused the request.

Holland appealed to the Information Commissioner's Office (ICO) about the UEA's refusals of FOI requests he had made for emails to and from Briffa about the IPCC AR4 report. On 23 November 2009, after the start of the Climatic Research Unit email controversy, he wrote to the Commissioner explaining in detail the relevance of the alleged CRU emails to his case.[7] In one of these sent in May 2008, Jones asked others to delete emails discussing AR4 with Briffa.[8] ICO decisions are enforced under Section 50 by notices defining steps to be taken by the public authority within defined time limits, and failure to comply can be taken to court. In addition, Section 77 makes it an offence to intentionally destroy or conceal information which has been requested.[9] In response to a Sunday Times journalist who persistently asked why the ICO was not prosecuting under Section 77,[10] Deputy Information Commissioner Graham Smith replied on 22 January 2010 that "The FOI Act makes it an offence for public authorities to act so as to prevent intentionally the disclosure of requested information. Mr Holland's FOI requests were submitted in 2007/8, but it has only recently come to light that they were not dealt with in accordance with the Act. The legislation requires action within six months of the offence taking place, so by the time the action came to light the opportunity to consider a prosecution was long gone." The ICO was looking into other investigations time-barred by statute of limitations restrictions to see if a case could be made to change the relevant law.[11] The journalist's report as published in The Times on 28 January portrayed this as an ICO decision that the university "broke the law by refusing to hand over its raw data for public scrutiny" but the ICO "could not prosecute those involved because the complaint was made too late."[12] The university promptly wrote to the ICO objecting to the damaging statement having been made to a journalist without first consulting the university. It described the misrepresentation by the press and requested retraction or clarification of the alleged breaches. The ICO declined to retract, but clarified that its investigation under Section 50 had not been completed and no decision notice had been issued. It said that "The fact that the elements of a section 77 offence may have been found here, but cannot be acted on because of the elapsed time, is a very serious matter. The ICO is not resiling from its position on this", and that "Errors like this are frequently made in press reports and the ICO cannot be expected to correct them, particularly when the ICO has not itself referred to penalties or sanctions in its own statement."[11][13]

In its submission to the Science and Technology Select Committee, the university denied allegations that it had refused to release raw data in breach of the FOI Act,[7] and said that the Deputy Information Commissioner's comments had been incorrectly reported as referring to such data.[14] In its inquiry report, the Science and Technology Select Committee criticised the ICO, which it said had made "a statement to the press that went beyond that which it could substantiate", and recommended that it should develop procedures to check its public comments, and "swiftly correct any mis-statements or misinterpretations of such statements". While there was prima facie evidence of a breach of Section 77 of the FOI Act, it would "be premature, without a thorough investigation affording each party the opportunity to make representations, to conclude that UEA was in breach of the Act." It called for a full investigation by the Muir Russell inquiry or by the Information Commissioner to resolve this issue, and accepted that the six month statute of limitations restriction was insufficient and should be reviewed. On the general issue of information requests, the committee said that information not covered by the exclusions provided by the FOIA should have been disclosed, and "in future information, including data and methodology, should be published proactively on the internet wherever possible." It blamed the university for mishandling Freedom of Information requests, and said it had “found ways to support the culture at CRU of resisting disclosure of information to climate change sceptics”.[11][15][16]

The ICO decision on Holland's requests published on 7 July 2010 concluded that the Environmental Information Regulations (EIR) applied rather than the equivalent FOIA, and the university had failed to provide responses within the correct time, but as Holland was content not to proceed with his complaint, no further action was needed. Regarding the question of whether there had been a breach of section 77 of the FOIA or its equivalent in the EIR, the ICO stated "Although the emails referred to above indicated prime facie evidence of an offence, the Commissioner was unable to investigate because six months had passed since the potential offence was committed, a constraint placed on the legislation by the Magistrates Court Act 1980." On the more general issue of emails suggesting that attempts to circumvent the legislation had been considered, and a lack of openness, the ICO would "now consider whether further action is appropriate to secure future compliance."[8] In an opinion piece, Edward Acton, Vice-Chancellor of the UEA, commended legislation in the US where "the FOI distinguishes between recorded factual material necessary to validate research findings, which must be disclosed, and ‘preliminary analyses, drafts of scientific papers, plans for future research, peer reviews [and] communications with colleagues’, which are exempted."[17]

In September 2011 the ICO issued new guidance to universities, taking into account issues raised in relation to the CRU information requests. This describes exceptions and exemptions to protect research, including allowance for internal exchange of views between academics and researchers, leaving formulation of opinions on research free from external scrutiny. It notes the benefits of actively disclosing information when it is in the public interest[18]