Money isn’t buying success in the digital world. Companies sink billions into data integration projects, with up to 25% of IT budgets that now go toward data interoperability efforts. What do they get? Often, not much. The dream of seamless data sharing keeps hitting roadblocks. Organizations face a maze of competing standards, each claiming to be “the one.”  Also, data security adds another layer of frustration. Moreover, teams must navigate an ever-growing list of data protection laws while keeping information both accessible and secure. 

However, the biggest obstacle for data exchange and interoperability is not a technical or a financial challenge. On the contrary, it’s the data that we send does not get understood. Specifically, transmissions are not correctly interpreted when the data crosses system or organizational boundaries. Indeed, these digital messages often get lost in translation. So, let’s cut through this complexity. In this article, I’ll break down the five biggest data interoperability hurdles businesses face today and show you what’s really going on behind the scenes.

What is Data Interoperability and Is It Different from Data Integration?

Before looking at data interoperability challenges, let’s start with two definitions, data interoperability and data integration.

“Data interoperability is the ability to correctly interpret data that crosses system or organizational boundaries.”

Scott A. Renner – MITRE Corporation

“Data integration refers to the process of combining and harmonizing data from multiple sources into a unified, coherent format that can be put to use for various analytical, operational and decision-making purposes.”

IBM

For many years, my perspective as an IT integrator was that these terms were synonymous. Now, I think there is a difference. First, the term “data integration” is more technically focused on the process and the specifications of data formats. 

On the other hand, “data interoperability” takes a more holistic perspective, focused on the end result, “to correctly interpret data”. This is an important distinction because the term, data interoperability, has a measure of success for data exchange. Namely, the success factor for data exchange is when what was sent is correctly interpreted by the receiver. On reflection, I successfully integrated thousands of data interfaces, but I am not sure how many of these interfaces actually achieved data interoperability.

So, data interoperability is not fully achieved from just successfully completing the technical process of data integration. It is only when the data is correctly interpreted that data interoperability is achieved. Thus, with this better understanding of what data interoperability is, let’s now look at its major constraints.

The Five Major Constraints for Organizations Achieving Data Interoperability.

For businesses to successfully compete in a digital world, they need to achieve data interoperability between their partners’ systems and with their internal systems. Consequently, they first have to have a complete understanding of data interoperability challenges in order to mitigate them. Indeed, this is the first step to overcome these constraints. For the remainder of this article, I’ll examine the constraints involved in achieving data interoperability. These include an adhere to tech standards, compliance with data protection laws, secure access to data, the high costs of integration, and to correctly interpreted data.

1. Adhering to Tech Standards: The Lack Of Standards Frustrates Data Interoperability.

Let’s Breakthrough The Data Interoperability Nightmare: It Is The Best Way To Unlock Supply Chain Innovation

The absence of widely adopted data standards creates a fundamental roadblock in achieving seamless data exchange. Organizations struggle with incompatible systems that speak with disjointed syntaxes that use varying data structures, that apply different measurements, and follow different protocols. Indeed, this current dysfunctional digital landscape leads to costly workarounds, manual data entry, ambiguity, and many errors. While industry groups push for standardization, the rapid evolution of technology often outpaces these efforts. As a result businesses, both great and small, are left to navigate a maze of competing standards and protocols.

For more discussion on addressing the challenges with data interoperability, see my article, Let’s Breakthrough The Data Interoperability Nightmare: It Is The Best Way To Unlock Supply Chain Innovation. Also, for a detail technical perspective of data interoperability, see acceldata’s article, Data Interoperability: Key Principles, Challenges, and Best Practices

2. Complying with Regulations: Obeying Data Protection Laws and Corporate Policies.

Data Sensitivity: What You Need to Know For Your Business

The regulatory landscape adds another layer of complexity to data interoperability. Organizations must balance the need for seamless data exchange with strict compliance requirements, from GDPR to industry-specific regulations. Corporate policies, designed to protect sensitive information and maintain privacy, often create additional barriers to smooth data flows. Indeed, this constraining regulatory environment causes hesitancy among companies to share sensitive data. At the same time, this data is key information for organizations to obtain interoperability. Worse, zealous companies will implement sophisticated governance frameworks that slow down data sharing initiatives. For more information on data protection, see my article, Data Sensitivity: What You Need to Know For Your Business.

3. Securing Access to Data: The Puzzling Challenge to Verify, Authorize, and Authenticate.

Digital Identity In Logistics And What To Know – The Best Security, Scary Risks

Security remains a paramount concern that businesses must address as part of achieving data interoperability. Indeed, modern authentication systems must verify users, authorize access, and protect against breaches while maintaining the smooth flow of data. So, all organizations face the challenge of implementing robust security measures without creating bottlenecks in data access. Further, the rise of sophisticated cyber threats adds urgency to this challenge, requiring constant updates to security protocols and authentication mechanisms. 

For more discussion on balancing security with interoperability, see Strata Identity’s article, Solving the identity puzzle: How interoperability unlocks cloud security potential. Also, for more on authorizing and verifying access within the logistics industry, see my article, Digital Identity In Logistics And What To Know – The Best Security, Scary Risks.

4. Massive Tech Costs: The High Costs of IT Data Interoperability and Integration Projects.

It is truly amazing that data integration projects and services can consume up to 25% of IT budgets. Indeed, the IT costs for businesses to implement each new data integration project is staggering. This is because these types of projects need specialized expertise and custom solutions that drive up expenses. Worse, the complexity of these integration projects often leads to budget overruns. Further, organizations face ongoing costs related to upgrading infrastructure, system maintenance, updates, and staff training. So, businesses must weigh making substantial investments against the uncertainty of actually achieving data interoperability. For more specifics on the factors behind high costs of data integration projects, see StarfishETL’s article, How Much Does Data Integration Cost?.

5. Make the Data Sent Understood: The Absence of Shared Knowledge Across Systems and Organizations to Correctly Interpret Data.

^{“The question is why do businesses and their systems exchange data that is not understood?”}

Perhaps the most challenging aspect of data interoperability lies in ensuring that data shared is correctly interpreted when exchanged between systems and organizations. This is because there is a lack of clear business definitions and semantic understanding of data when it is exchanged between systems. Hence, this results in ambiguity and misinterpretation of data. The question is why do businesses and their systems exchange data that is not understood?

a. The False Assumption that Data Exchanged Is Data That Is Understood.

First, many businesses operate under a false assumption when exchanging data. Namely, organizations assume that once they transmit the data, that it is understood by the receiving organization and system. So, the crux of the issue lies not in the transmission of data, but in its interpretation. Indeed, most interoperability solutions focus on the technical aspect of data transfer. Namely, data integration. As a result, most organizations can successfully transfer data, but their integration efforts often overlook the semantic operational layer – the meaning and context attached to the data. So, without a shared understanding of what the data represents, the data is rendered useless even when it was perfectly transmitted. 

b. Example of Data Integration Where the Data Loses Meaning.

As discussed previously, a data integrator can successfully establish a data interface between two systems, but that does not necessarily mean data interoperability will occur. For example, particularly in supply chains, many times organizations have varying definitions of basic business terms such as “shipped” and “delivered”.  Specifically, this is because either the sender transmitted unintelligible data or the receiver misinterpreted it upon receipt. For instance, a warehouse system may transmit a “shipped” status message for a package when in reality the operation has only printed a barcode label for the package. As a result, the customer reasonably expects that the package is on its way upon receiving the “shipped” status message versus it still sitting on the shipping dock.

c. Lack of Knowledgeable Context and Clear Definitions to Share Meaningful Data Across Systems.

Achieving Logistics Interoperability: The Best Way to Breakthrough The Tangle Of Dumb Data Integrations

So, there is a gap between organizations integrating their systems and them actually achieving true data Interoperability. More specifically, what is usually the case is that there is a knowledge gap in regard to the data transferred between the two organizations and their corresponding systems. Indeed, the data in most cases was received, but it is missing knowledgeable context and clear definitions to make it understandable. What is needed is semantic interoperability where organizations and their systems exchange meaningful data with their partners’ systems. Thus, closing this knowledge gap and achieving data interoperability.

For a more detailed discussion, see my article, Achieving Logistics Interoperability: The Best Way to Breakthrough The Tangle Of Dumb Data Integrations, In this article, topics include semantic interoperability benefits and recommended solutions to improve interoperability by exchanging meaningful data. 

Logistics Data Interoperability: Advice To Make It Understandable, Usable, Secure.

For more from SC Tech Insights, see latest articles on Data, Interoperability, and Information Technology.