Are complex ERP data models holding organisations back or helping them move forward?

According to a recent report by leading analyst firm Gartner, heavily customised ERP (Enterprise Resource Planning) systems will have achieved the status of ‘legacy ERP,’ by 2016.

In addition they suggest that as more agile, flexible competitors enter the market and mature, CIO’s will need to plan on what to do with their ‘legacy ERPs’.

It is also interesting to note Gartner now defines legacy as: ‘any system that is not sufficiently flexible to meet changing business needs’ so are complex ERP data models holding organisations back from making more use and getting more value from those systems?

How easy is to find ERP data structures?

The customisation of large Enterprise systems takes place on two levels: business process and data.  The changes are delivered in response to a need for additional flexibility and to meet specific business needs which are not catered for in the standard application functionality.  Even the Enterprise applications from SAP and Oracle which are large and complex enough in their native state have, in many implementations, had a significant amount of customisation visited upon them.

Bearing in mind Gartner’s definition of ‘legacy’, I think that it is quite possible that ERP systems have already attained ‘legacy’ status in terms of data. By this I do not mean the data contained in the applications, I am referring to the way the data is structured in the underlying application data models and the ease, or difficulty, with which it can be understood in the context of supporting projects to meet new business needs.

What prompts me to come to that conclusion is that as customers become increasingly keen to leverage their investment in those systems to meet changing business imperatives they are encountering significant challenges in achieving this because of the complexity of the source data, the sheer numbers of tables involved and the level of customisation of those data structures.

These problems occur when trying to integrate ERP’s with new applications, extract data from them for BigData, Datawarehouse, Business Intelligence and Analytics projects or incorporate the information they hold in Master Data or other Information Management programmes.

The challenge these organisations face is that the underlying data models which are the foundations of these applications are incredibly complex and were not designed with the outside world in mind and there are no vendor supplied tools which allow easy access to them available for people who are not dedicated application specialists.  See this Bloor Research report for more information.  Even the Information Management vendors only provide a partial solution which relies on a significant amount of application knowledge.

To give you some context in terms of the size of data models involved (before customisations): JD Edwards EnterpriseOne comes with about 4000 tables, Siebel with about the same, PeopleSoft and Oracle E-Business Suite have around 20,000 each.  SAP’s current release of Business Suite has approximately 90,000 tables before any are added.  We, at Silwood Technology are aware of one of our customers whose SAP system has 117,000 tables and are sure that many are even bigger!

In addition there are added levels of complexity to overcome when trying to find what you need for your projects because of the naming conventions and the relationships between tables.  Also this information is not accessible from the data dictionaries of these products, it can only be found in their System Catalogues which presents another hurdle when trying to understand how the data models are architected.

Struggling to find the right data can lead to delays to project delivery and overspend or even the wrong data being introduced into the Enterprise data stream which can significantly impact the performance of the business and damage the reputation of IT within the organisation. By contrast Microsoft Dynamics and Salesforce both have under 300 tables, which is still a significant number, but an easier challenge for data architects and analysts to solve in the context of Information Management projects.

We suspect that the trend will be towards applications, whether cloud or on premise based, which are more tightly focused on meeting a specific business need with less complex data structures which are easier to understand and integrate with other applications and into the data ecosystem of an enterprise.

Andrew Kyte from Gartner sums up the challenges facing those customers with customised ERP’s: “When ERP was in its heyday, CEOs and business executives wanted reliable and integrated solutions, so they seized upon ERP as the way to provide this.  Business stakeholders still want these same qualities, but now they assume that these qualities will be present in any software solution, and their requirements have switched to the twin concerns of lowering IT costs and seeking increased flexibility. A system that is not sufficiently flexible to meet changing business demands is an anchor, not a sail, holding the business back, not driving it forward.”

I believe that this is equally important for data as business process.