“The definition of insanity is doing the same things over and over again but expecting different results”.  

I have been in the ERP implementation and ERP audit space for over 25 years. More often than not implementations are NOT “successful” – let’s explore why.  

Understanding Why Your Risk Advisory Firm Should Not Be Your SI Firm

Typically, management’s objective is to go-live, on time, and with the agreed upon functionality.  Management expects their System Integrator (SI) partner to support that objective. Yet, most SI’s don’t always have the same objective. 

I would argue that most SIs, whether they admit it or not, have the objective of winning the deal and going live without being blamed for project delays, cost overruns, or implementing with less scope than management intends. 

I am not trying to imply that System Integrators are dishonest – although some are. However, it is important for management to know that the last thing an SI wants to be blamed for is missing their agreed upon scope, milestones, and deliverables.  

Learning From Experience: Preparing for Success

In 2024, we did a three-part webinar series that focused on how to prepare for a project, how to execute a successful project, and how to manage the perpetual patch cycle that SaaS applications force on you after go-live. If you are considering or planning a digital transformation project, (Click here to watch them – One, Two, and Three).

The Most Overlooked Risk: Security and Controls Scope

One aspect of a “failed” implementation that I want to cover relates to the scope of security and controls.  

If you are a publicly traded company subject to Sarbanes-Oxley requirements or similar requirements in your country, compliance is not optional. Additionally, there are typically one or more data privacy regulations, such as GDPR or CCPA, that need to be considered. 

Most SIs are experts at implementation methodology and the functionality they are implementing. However, for organizations that care about security and internal controls, I would argue that most SIs do not bid their projects with the full scope needed to address management’s security and controls requirements. (For deeper insight on that topic, see Part 3 in the 3-part webinar series.) 

No ERP implementation is the same, which means a cookie cutter approach to implementation and configuration – or management’s requirements – will not work. 

As a side note, we recommend setting aside a budget of three (3) to ten (10) percent of the project for post-go-live care activities. There are always important aspects of the system that are set aside in order to go-live on time that will need to be matured.

Why Traditional Risk Advisory Approaches Fall Short

In our experience, the traditional risk advisory approach to identifying, mitigating, and managing risks is broken. Most risk advisory firms  spend most of their training on general ERP concepts rather than risks specific to a given ERP system. – yes, even those for which they are providing niche advisory services for. This often means the risk population is incomplete. (Read More on this topic here and here) 

Unfortunately, the primary SI firms receive even less training related to risks and controls than their risk advisory counterparts which leaves overall system security to them.  Additionally, many, if not most, projects we encounter do not accurately scope the engagement of external experts to ensure that compliance (SOX), cyber security, and data security requirements are properly addressed. Often management believes that they can use internal resources to “roll forward” their current approach to security and controls without hiring an external risk advisory partner. Or they rely on a big four / tier 2 provider to help address these security and controls requirements.  

Often, for very large implementations, management hires the same firm to do the risk advisory services work as well as the core SI work. We liken this to having the fox watch the hen house and it is something we highly advise against. Why? Because commonly, this approach to implementations end up with one or more Significant Deficiencies from external auditors. 

The PCAOB defines a Significant Deficiency as “a deficiency, or a combination of deficiencies, in internal control over financial reporting that is less severe than a material weakness, yet important enough to merit attention by those responsible for oversight of the company’s financial reporting. (link)” 

Two types of Significant Deficiencies are especially common. 

The first is related to role design. Poor role design inevitably results in users having overprovisioned access, resulting in segregation of duties risks and access to perform functions that do not align with their job responsibilities. It can also lead to users having the ability to circumvent established controls. The ability to override controls can come from many sources such as: 

• Improperly designed workflows 

• Ability to disable workflows or change the workflow design 

• Ability to override workflows via conversion processes, interfaces process, APIs, web services, and changes to configurations on which the workflow design depends 

The second most common Significant Deficiency is related to Change Management. This control deficiency is characterized by: 

• A lack of clarity about what is / is not subject to the change management process 

• End users or unauthorized IT users having access to configurations that are subject to the change management process 

• Changes being made where no ticket is logged and/or approved 

• Lack of documentation for the testing of the change 

• Unauthorized changes  

We see these two types of Significant Deficiencies often because we’re engaged to clean up the mess. Companies tend to only address them after go-live – usually identified by management or external auditors. This is part of the reason we recommend budgeting for an additional 3% to 10% of the project   for post-go-live remediation and maturing processes that aren’t addressed by SIs.  

What Services Should Firms Other Than Your Core SI firm provide? 

The following are some of the services not provided by your core SI team that management needs to consider: 

• Data migration and data cleansing 

• Data conversion preparation 

• Storage of historical data that isn’t being converted 

• Documentation of business requirements 

• Development and management of test scripts and test scenarios 

• Execution of test scripts and test scenarios 

• Management of issues 

• Development and management of training materials 

• Training of IT and functional resources 

• Requirements related to system roles,  

• Testing of RBAC requirements, including addressing the principle of least privilege in the design of system roles  

• Mapping roles to users and/or personas, jobs, or positions 

• Consideration of cyber security requirements including internal and external threats 

• Governance processes over shared master data and shared configurations 

• Verifying the completeness of system logs and designing the necessary monitoring controls using such logs 

• Developing a population of changes and implementing quality assurance over their change management process (IT governance). One of our most common recommendations to management is to develop and document IT governance processes and verify they are being followed by having a robust IT quality assurance function. We rarely see this implemented during a project, let alone after the system has gone live. 

This is just a partial list… 

Why Do You Need a Risk Advisory Firm Separate From the Core System Integrator? 

Should you hire a large system integrator like a Big 4 firm or another large international system integrator? There are pros and cons to this. Generally, we find the larger the firm, the less quality of resources and more turnover. 

 In reality, I would argue that management needs two independent firms, one performing risk advisory services and one performing functional implementation services.  

The risk advisory firm should be your independent check and balance versus your system integrator that asks the questions: 

• Have we identified and documented our compliance and cyber security requirements? 

• Have we addressed the risks and implemented proper controls? 

• Are we prepared to defend management’s assessment of risk, control design, and operating effectiveness of our controls to our auditors and regulators? 

The Inherent Conflict of Interest You Must Avoid

Your risk advisory firm should be advocating for your organization by helping management throughout the project – from requirements, to design, to testing, to go/no go decision.  

Let’s say you hire a risk advisory firm from a large SI and hire the same firm to do your implementation work. If your core SI team has not done the work required to prepare for a successful audit, do you think their advisory peers are going to tell you that their colleagues have  

not held up their end of the bargain? To expect the risk advisory team to throw their peers under the bus is the same as asking the fox (the SI firm with its risk advisory firm) to watch the hen house (your organization). 

Presumably, you have likely read one or more case studies of failed ERP implementations. In every failed implementation, either management willingly decided to go live with too much risk or did not fully understand the risks at hand. 

Because of the inherent conflict of interest between the SI practice and the risk advisory practice, we do not recommend these being the same firm. 

Final Thoughts: The Most Common Significant Deficiencies 

Most System integrators do not understand the complex compliance and cyber security requirements of today’s modern systems. 

We previously discussed the two most common Significant Deficiencies (SDs) related to Role Design and Change Management. These two SDs are normally two sides of the same coin.  

First, roles are not designed based on the principle of least privilege.  

Secondly, changes are made to configurations that should be subject to the change management process.  

Your Risk Advisory firm needs to be fully sold out to protect MANAGEMENT’S interest and not consider the interests of the System Integrator. Role design creates the foundation of your security and controls.  When this is done incorrectly it leads to people having access to things they should NOT have access to, which  may lead to a Significant Deficiency.. If your risk advisory firm is doing primary role design, how can you ask them to independently verify that their roles are based on the principle of least privilege and optimized to address Segregation of Duties issues. 

If you are scoping your project consider two different approaches: 

1. Have ERP Risk Advisors design and build your custom roles separate from your core SI and separate from your risk advisory team that is helping you identify and document your controls. In this case, ERP Risk Advisors will stand up to the rigor of a review you would expect from an independent risk advisory firm.  

1. If you have decided to use the same big SI firm to do your SI work and your risk advisory work, which typically includes the building of custom roles, ERP Risk Advisors can perform an independent verification to validate the work being done.  to ensure your system, security, and controls are ready to go live. 

Do not, under any circumstance choose option 3 – have the risk advisory division of your system integrator let you know if your system is ready to stand up to the scrutiny of your external auditors. Do not let the fox watch the hen house… 

If you would like to discuss this approach in more detail or have experienced this in your own implementation, we would love to hear your story and work with you to get your system secure and compliant. You can contact us at sales@erpra.net.