It has become the hot topic within the regulated industry, and one of the most frequently asked questions- how does a firm go about reviewing audit trails? However, one cannot expect a simple response that will solve this conundrum. Instead, let’s break the question down into simplified, logical questions so one can resolve this FAQ in any situation.
Let’s start with what is the purpose of the audit trail?
I like to think of it as a user’s digital footprint. It contains critical metadata such as the user name, date, and time of activities while logged into a system. This information, along with a log of their actions and reason for change are key not only for reconstructing activities, but to assure the integrity of the data. It allows us to determine if only authorized individuals were logged in, and if there were any unauthorized changes or deletions to the data. Whether you’re dealing with electronic data in industries such as pharmaceutical, food processing, or banking, we can all agree that this information is critical and I don’t think any persuasion is needed to convince anyone, whether they’re purchasing medication, or buying a candy bar, that one would want to be assured there are ways of ensuring the integrity of data from research to manufacturing. It is for this reason that they need to be available for inspection, and regularly reviewed.
With that comes the obvious question of how does one go about reviewing the audit trail and how often should it be reviewed?
Now that we’ve established the reason the audit trail is necessary, the next step would be to determine the level of risk the data poses. Whether the risk is related to business or patient safety, one should have a means of determining the level of risk to permit a thorough and meaningful review of the audit trail. Some systems will generate a prodigious amount of information which would be impossible to efficiently and effectively review it in its entirety. For this reason, it would be helpful to look at the system in question and ask questions such as- does the system allow changes or deletions to data? If so is this restricted to certain users? It’s for this reason that during the validation of the system, system owners should determine along with the intended users, the need for certain access privileges. This will greatly reduce the risk to data integrity, and the review of the audit trail. Does the system allow one to filter the audit trail? This will help narrow down the review to those user roles that have access to modify data (note this is also something to consider when evaluating systems during the purchasing phase, and if not available, discussing with vendor if it’s possible to add this to the system’s functionality).
Now, how often? Using a risk based approach does not mean sparingly! It should always be reviewed for every data set that is intended to be used for regulated purposes. This is similar to changes documented on paper; auditors would not ignore changes when reviewing paper documents. At this point it’s helpful to identify the critical aspects of the workflow and ensure to the audit trail is reviewed for those activities.
Finally, who is responsible to perform this review? Most people respond – Quality Assurance, but this is of course fallacious. Both QA and operational personnel who generated the data would be responsible for reviewing the audit trail to ensure data integrity. While QA would primarily use the audit trail as an investigational tool when auditing, it is the data owners that are responsible for their data and thus making sure the audit trail has been reviewed so it is audit ready for QA. Quality Assurance will audit the data and audit trail in accordance with their procedures for sampling, again based on the criticality and risk of the data. Should errors or concerns be raised, typically the sampling size will increase and could result in a critical observation given the data should have been ready for audit in accordance with the applicable regulations and standard operating procedures.
And remember, there should be an SOP to describe the audit trail review process, highlighting the personnel responsible, how it’s to be performed, and the frequency.
For further information on this topic you can read Audit Trails: Assessing Compliance, a poster presented at the 2015 SQA conference. This poster explored the expectations from international regulatory agencies with respect to the implementation of compliant computer system audit trails and the requirements for their review.
Does your firm have a procedure in place for audit trail reviews? What are some of the challenges you’ve faced with implementing a process for its review? Share your experience with your peers!