Navigation * Home / Consultancy / Achieving compliant email and archive solutions

Achieving compliant email and

archive solutions

More and more companies are being fined significant amounts of money for improper management of electronic records, especially emails. As a result, many vendors that provide email backup and archive software are now adding “regulatory compliance” to the list of features offered. In most, if not all cases, this is not true. At best these products are “more compliant” but nearly obeying the law is not good enough!

This document explains the issues of compliance, why many current products fail to meet the requirements and provides an overview of what a compliant solution must provide.

Compliance Overview

A lot of marketing is equating compliance with a product’s ability to “provide legal admissibility”. This turns out to be irrelevant due to the confusion that exists between the concepts of legal admissibility and evidential weight. Any electronic data can be submitted before a court of law, i.e. is legally admissible. The real question is that of evidential weight. Evidential weight is the extent to which the court can rely upon the electronic information.

Evidential weight is also more than just document retention. Evidential weight requires that all aspects of data are retained; the existence or occurrence, the data itself, any access to the data contents, any attempt to edit or delete the data, ensuring that the data created is the data that is stored and demonstrating that no data has been lost or inappropriately deleted. For example, just storing a backup or snapshot of a messaging system is insufficient as is a system that provides a policy for storing messages after a period of time or under user control.

Compliance means meeting the requirements of evidential weight, which implies legal admissibility and assumes that record keeping requirements have been met.

Electronic mail is increasingly being used in business and has now been recognised as a prime source of evidence. The spotlight is now on how a company manages its email systems in order to achieve the required level of evidential weight and demonstrate compliance.

The commonplace practice of taking a regular mail server backup is not sufficient. In a recent case, the US Court of Appeals for the Second Circuit held that a party could be held liable for failure to deliver electronic evidence even if the unavailability was not caused by gross negligence or bad faith of that party.

Sanctions could be imposed for failing to deliver, or discover electronic information even if that failure was due to ordinary negligence, in this case the inability to retrieve electronic mail messages from backup tapes. Aside from regulatory requirements the law in general requires that users, especially commercial users, retain their records until any potential legal action has ceased to exist. Litigators have discovered that email records, or rather the lack of such records are an ideal tool to use to win cases.

To quote Fortune:

“That means stratospheric legal fees for the banks and, yes, a whole new battery of lawyers delving into Wall Street’s email.”

If a user cannot prove that every email has been retained for the correct period, they have not satisfied the evidential weight requirements. All companies have an obligation to maintain records that could be used in a dispute for as long as a dispute may arise, as well as a general obligation to maintain records as the cost of conducting business. Companies that operate in highly regulated industries, however, must also meet additional specific requirements set down by their industry regulators. Irrespective of how they arise, the record keeping requirements are subject to the rules of evidence, which are quite separate and which apply in addition to these record keeping requirements.

Data protection regulations present the first conundrum. Emails must be retained for a specific period. However data protection laws in Europe require that any record or email that contains personal data must be deleted after its retention period expires. A compliant system must allow this, effectively making optical devices such as CD and WORM non-compliant.

Data privacy regulations present the second conundrum. Employers are not, despite what is often believed, allowed to instigate blanket monitoring of all communications by employees. They must ensure that they follow the guidelines set out by the Information Commissioner. This does not mean that they cannot archive emails. It simply means that they cannot read through the archives unnecessarily. However, if it is believed that an employee is improperly using email then it may be permissible to access the archive. All such access must be audited.

Popular Misconcepts

Several popular misconceptions need to be corrected if organisations are to be able to comply with full evidential weight requirements for email archives.

Allow users to decide what and when to archive

Many existing archive solutions are based around the end-user managing their data, the goal being to reduce the size of mailboxes and pst files.

This is non-compliant as there is no guarantee that all records are archived and in their original form.

The archive application must archive everything automatically, but it can leave the user or administrator to decide whether the relevant retention periods for the individual records need to be extended from a predetermined time limit or not.

Archive from a mailbox after a certain time or capacity limit

A variation on the user self-management theme is for the system to automatically archive data from a mailbox based on a time or capacity limit.

This is non-compliant as there is no guarantee that all records are archived and in their original form. A user can change or delete records before the archive takes place.

All messages should be archived before they are delivered to the recipient’s mailbox and immediately after the message has been sent from the sender. Any time delay between placing a message in a mailbox or mail spool and archiving that message leaves a time period during which that message can be lost, deleted or altered. Even a minute is too long to wait.

Archive using a journaling facility

Many mail archive systems provide the option to archive messages automatically using a journaling facility

This is non-compliant as there is no guarantee that all records are archived and in their original form. It is still possible to change or delete records before the archive takes place because they are effectively being buffered.

Journaling has another disadvantage which undoes part of the operational improvements that current archive products have been designed to meet – it causes an increase in the number of messages stored and processed by the mail server which can cause noticeable performance degradation.

Archive to optical device

Storage companies have pushed archive solutions based on optical devices as a cost effective solution.

In certain circumstances it may be possible to construct a compliant system using optical devices but the operational overheads and additional procedures required would make it impractical to do so.

Information and records that contain personal data must be deleted once the data retention period comes to an end. Users cannot just delete an index if that message could easily be retrieved. The record (i.e. the data) must be deleted. Optical disk technology cannot facilitate this requirement as the only way to ‘delete’ the information is to transfer the remaining live records over from one platter to another new live platter and then destroy the original platter. The transfer itself is a risky process that could damage, delete or alter documents.

Compliance with BSI-DISC PD0008 is sufficient

A lot of attention is now (quite correctly) being given to PD0008 but with the suggestion that compliance can be achieved just by meeting PD0008.

This is not the case; compliance with PD 0008 is necessary but not sufficient to achieve compliance.

PD0008 is a code of practice designed to help verify that the data retained on a system is the same as the data that was entered into and recorded in the system. It does not cover the steps and procedures that are necessary to prove the authenticity of data before it is stored in a compliant system.

BSI-DISC PD 5000:2002 describes the set of requirements to create systems that provide the maximum possible evidential weight for their electronic messages, documents, and ecommerce systems.

Access and audit trails are easy to implement

A compliant solution must control access to the records such that only authorised accesses are allowed and provide a full record of every event, or attempted event, in the life of the record – creation, access (including the type of access) and deletion.

An audit trail must be more than a simple record of messages. All access and attempted access to a record should be audited, as should each and every step or process during the lifetime of a record. That audit trail must not be kept with the record itself but stored separately.

The meaning of non-repudiation

An archive system alone does not address the full issues of evidential weight, and therefore compliance. One issue not addressed is the issue of non-repudiation. PKI vendors provide digital signatures, and define non-repudiation to mean that the sender cannot deny sending a message that has been digitally signed and it can be demonstrated that the message has not been changed.

A PKI system alone is non-compliant as it does not provide full non-repudiation. Public key technology is, however, one component of a compliant solution. A fully compliant solution needs to meet all the requirements of non-repudiation. Non-repudiation means much more than the recipient being able to prove that the sender sent them a message.

Non-repudiation means:

• The sender can prove that he sent the message;
• The sender can prove that the recipient received the message in a form that the recipient could read;
• The sender can prove that the message sent was the message delivered and in the form that it was sent;
• The sender can prove that the message has not been altered in any way;
• The sender can prove that his copy of the message has not been altered in any way;
• The sender can prove that the copy of the message that was sent was archived at the point of transmission before any alteration was possible;
• The sender has a full audit trail of the message;
• The recipient can prove that the sender sent the message;
• The recipient can prove that he received the message;
• The recipient can prove that the message received was the message sent and in the form that it was sent;
• The recipient can prove that the message has not been altered in any way;
• The recipient can prove that his copy of the message has not been altered in any way;
• The recipient can prove that the copy of the message that was received was archived at the point of reception before any alteration was possible; and
• The recipient has a full audit trail of the message.