This series is written for data professionals working on governance, analytics, data catalog, migration, and integration projects, particularly those whose organisations run ERP applications from SAP, Microsoft, Salesforce, or Oracle.
Each part can be read independently, but together they trace a path from the fundamentals of metadata through to the practical challenges of working with large, complex enterprise systems.
Contents
Part 1: Metadata Discovery and Data Intelligence
Part 2: Using Metadata with Data Intelligence Projects
Part 3: The Challenges of Metadata Discovery
Part 4: SAP and ERP Metadata: A Suitable Case for Treatment?
Part 5: Using Safyr for SAP and ERP Metadata Discovery
ERP applications are the operational backbone of most large organisations. They encompass finance, manufacturing, sales, distribution, HR, product lifecycle management, project management, and much more. Precisely because they touch so many business processes, they also hold some of the most important — and most complex — data in the enterprise. And that complexity makes metadata discovery from ERP systems a genuinely different challenge from working with a standard database or a modern cloud data platform.
In this blog, we focus primarily on SAP, which representsthe most extreme case of ERP metadata complexity. Many of the same principles apply, to varying degrees, to Oracle EBS, JD Edwards, PeopleSoft, Microsoft Dynamics, and Siebel. SAP is where the challenges are most acute and where the consequences of an inadequate approach are most keenly felt.
To understand why SAP metadata discovery is challenging, it helps to start with the numbers. A typical SAP S/4HANA system will contain over 130,000 base tables, more than 1.5 million attributes (columns), and around 270,000 relationships between tables. Even SAP ECC, the older on-premise predecessor, is comparable in scale. These are not figures from a reference model or a theoretical maximum; they reflect the reality of what a live, production SAP system contains.
For a data team trying to identify which tables are relevant to a particular use case (say, understanding the data model that underpins order-to-cash, or building a data product around customer master data) navigating 130,000 tables without specialist tools is not a realistic proposition.
The natural starting point for many data teams is the application's system catalog: the database-level repository of structural information about tables, columns, and data types. For a standard relational database, this is often sufficient. For SAP, it is not.
SAP enforces referential integrity at the application layer rather than as database-level constraints. This means that a system catalogs can will reveal no relationships between tables, with no indication of how Customer connects to Order, or how Order connects to Payment. The physical names in the system catalog are also purely technical: cryptic abbreviations that are meaningful to an ABAP developer but largely opaque to a data analyst or governance professional. The business-friendly names and descriptions that make SAP metadata intelligible live in SAP's ABAP Data Dictionary, at the application layer, not in the underlying database system catalog.
The practical consequence is that using a standard data modelling tool to reverse-engineer an SAP schema via ODBC will return a large volume of technically accurate but practically unusable information: physical names, no relationships, and no business context.
A further complication is that virtually every SAP installation is customised. Customers adapt the system to reflect their specific processes and operational requirements, adding fields, modifying table structures, creating custom objects, and extending the standard data model inways that are unique to their organisation. This means that generic data models, documentation, or templates found online or provided by SAP are unlikely to reflect what is actually implemented in any given customer's system. The only reliable source of truth is the customer's own ABAP Data Dictionary.
SAP provides tools for exploring its data model (SE11 forthe ABAP Data Dictionary and SE16 for table browsing, among others) but these are designed for SAP technical specialists, not for data analysts or governance professionals. They do not offer the kind of search, filtering, and curation capabilities that a data team needs to identify relevant subsets of metadata and prepare them for use in a catalog, a governance platform, or an analytics tool. Working with these tools effectively requires ABAP knowledge that most data specialists do not have and should not need to acquire.
Taken together, these challenges create a real risk to any data intelligence project that depends on SAP metadata. Data teams that cannot access the right metadata directly may find themselves relying on manual research, consulting with SAP technical specialists, working from incomplete documentation, or using generic reference models that do not reflect their actual system. All of these paths are slow, expensive, and prone to error.
The impact is predictable: delayed delivery of project benefits, cost overruns, and — perhaps most damaging — an inability to demonstrate the provenance of data to the business stakeholders who need totrust it.
If you are an SAP, or other ERP vendor customer, their metadata is almost certainly important to your data project. The real questionis whether you have a way to access it that does not, for example require specialist SAP knowledge, weeks of manual effort, or compromise on quality.
In Part 5: Safyr: Using Safyr for SAP and ERP Metadata, our final part of our series 'What is Metadata Discovery', we look at how Safyr, Silwood Technology's specialist ERP metadata discovery tool, addresses these challenges and puts metadata in the hands of the data teams who need it most.