AUGUSTA — A huge number of emails and public documents from the administrations of Govs. Angus King and John Baldacci have likely been deleted from state government servers – some lost forever, others in storage on difficult-to-access backup tapes. The loss hampers the ability of citizens, historians, journalists and lawyers to learn how policies were developed and implemented.

State officials transferred emails and documents to backup devices as part of routine purges to free up server storage, not realizing many of them should have been retained in accessible forms and eventually transferred to the state archives, Maine State Archivist David Cheever said.

Most of the purges are believed to have taken place prior to the 2005 creation of the Office of Information Technology, which centralized and standardized state government computer services for the first time, but former officials say the practice continued sporadically until 2009.

The extent of the losses is unclear, in part because there are no records of what was deleted. Cheever said officials within OIT had told his office that almost all government emails sent before 2008 had been lost forever because the obsolete storage technology they were backed up onto can no longer be read, and that an unknown quantity of emails and other digital documents created between 2008 and 2016 may exist only on cumbersome backup tapes stored out of state by a data protection subcontractor.

A former employee of the Department of Administrative and Financial Services, OIT’s parent agency, told the Portland Press Herald/Maine Sunday Telegram that most purges took place before 2005, when each state agency and department operated its own computer networks and was often running out of server storage. The employee, who asked not to be identified because he didn’t want to harm his relationships with colleagues, said the purged documents had probably been backed up to magnetic tapes or other storage devices, but that the obsolete formats might be impossible to read. Even if they could be read, there is no practical way to search them using keywords.

“We cannot say with precision how many documents have been lost, because once deleted there’s no way to bring it back or even to determine if it ever existed,” Cheever said. “Accessing anything that’s on the backup tapes would be a fairly labyrinthine effort, expensive in terms of time and money, and could wind up being fruitless because they are only searchable by date.”

PROPERTY OF THE PEOPLE OF MAINE

The national organization of records managers cautions members against using backup tapes to satisfy document retention requirements, as they can’t be purged of items that have met their required retention periods and are not stable in the long term.

“Backup tape, over time, degrades, making some data irretrievable unless kept in a pristine environment and regularly tested,” says Nick Inglis, executive director of content and programming at the Association of Records Managers and Administrators, which is based in Overland Park, Kansas. “The storage of any long-term governmental data on a single backup tape without providing appropriate care, maintenance, and testing would likely violate our profession’s commonly accepted standard of ensuring information’s integrity.”

Documents and emails created after the state’s 2016 adoption of the Office 365 cloud-based software systems remain fully accessible, both Cheever and the former employee said, though the exact dates when the system went live varied by department and agency. Some parts of state government didn’t switch to the cloud-based platform until September 2017.

By law, most government emails and documents are the property of the people of Maine and are subject to public review and records retention schedules. Documents that dealt with or influenced policy decisions are supposed to be saved in perpetuity at the Maine State Archives.

Cheever and the former employee both said the purges had occurred innocently as state information technology workers were squeezed between limited budgets and a massive increase in the volume of digital documents starting at the end of the 1990s. As servers filled up, managers tried to identify and purge what was thought to be digital trash. Emails of former employees might be dumped if nobody had accessed them for two years, for instance. They likely didn’t realize some of the materials they were purging were supposed to be saved for retention in the archives, Cheever said.

AN EXPLOSION OF DIGITAL DOCUMENTS

David Heidrich, a spokesman for the Department of Administrative and Financial Services, declined to allow the Press Herald/Sunday Telegram to speak with OIT officials or staff for this story and would answer questions only in writing. In written responses to questions about the purges of documents from servers, he denied that any had taken place, at least since the creation of OIT.

Cheever and the archives staff report to Secretary of State Matthew Dunlap, while Heidrich’s department and OIT are part of the executive branch, headed by the governor.

Heidrich initially said his department believed all past emails and documents still resided on the servers of the agencies and departments that created them. He agreed to test this assertion by asking departments to confirm possession of the emails of a half dozen past officials chosen at random by a reporter.

Heidrich said the emails of four past LePage-era officials – Environmental Protection Commissioner Patricia Aho, the governor’s chief legal counsel Dan Billings, and Education Commissioner Steven Bowen and his spokesman, David Connerty-Marin – had been located at their respective departments. He said the governor’s office servers still held the emails of Gov. Baldacci’s deputy chief of staff, David Farmer, who served from 2007 to 2011, but that they could not locate the records of Gov. King’s chief of staff, Kay Rand, who served from 1995 to 2003, suggesting many King administration files may have been lost.

“We do have some pre-2003 records in our possession, but these are far more sporadic (than later years) because of the decentralized approach to information technology at the time,” he said.

The only pre-2008 emails that have been transferred to the state archives for permanent custodianship are those of King and Baldacci themselves, not anyone else on their staffs or in any other part of government, Cheever said. The archives do not yet possess any material from LePage’s administration.

“When Angus King left the governorship he printed off his emails and presented them to the archives,” Cheever said. “Governor Baldacci provided digital files, and they were far more numerous than we had from Angus King. From King to Baldacci to LePage, the volume of digital documents has exploded by like 15 or 20 times.”

‘ANY GAP … A CAUSE FOR CONCERN’

Inglis from the Association of Records Managers and Administrators says he understands how computer managers might have wound up relying on storage tape backups to free expensive server space. “As seen in similar cases, (it) was likely driven by budget constraints beyond their control,” he said via email. “Solving statewide information challenges requires that the professionals tasked with managing information (for the public’s best interest) must be adequately funded and trained, both of which would be necessary to avert future information crises like this one facing Maine.”

Maine State Historian Earle G. Shettleworth Jr. said that if state records were lost it would impoverish future decision makers. “The themes and issues we have been dealing with since we became a country and a state are constantly reoccurring, and it is very important to be able to go back to the written or digital record to find out how different administrations and governors dealt with them,” Shettleworth said. “As a result, official government record keeping is a very important responsibility for state governments.”

Maine has a fairly complete record of official government activities from 1820 right up to the dawn of the digital age during the King administration, Shettleworth said, when paper records and communications started a gradual but sustained shift to electronic forms. “As Mainers we can pride ourselves on the fact we have such a good record,” he said. “Any gap in that is a cause for concern.”

Related Stories