ROI For Data Integration and Management: Understanding Financial Metrics For Data In Multi-cloud/Hybrid environment

Binayaka Mishra,Solution Architect / Project Manager- Tech Mahindra, India

Introduction

This article is primarily based on primary research into trends in the data integration market and how users perceive them. This report focuses on why user companies chose specific Informatica data integration options, what they deemed to be the most important technical considerations, and how they perceived data integration to fit into the larger data management landscape. In this context, this article will summarise the findings of a discussion of various deployment options (the use of microservices, cloud-based implementation, and managed services) as well as consider the various technologies that are complementary to data integration. In financial terms, I detail – typically using third-party research – the associated risks (when not implemented) and rewards of each of these technologies.

However, before delving deeper into the results, it is worth having a brief discussion about TCO(Total Cost Of Ownership) and ROI (return on investment), as this devotes a significant amount of space to the latter. While TCO is fairly simple to calculate, ROI is more difficult. There are numerous potential benefits, including cost savings, risk reduction (of various types), increased sales (via better data quality), and so on, as well as various methods of calculation, such as NPV (net present value). I will discuss relevant measures in the sections that follow, as appropriate. We present statistics that relate to these metrics when they are available, but when they are not, we simply present the principals involved.

I'd like to add that I'm happy to avoid discussions about TCO in general because, aside from competitive comparisons between vendors (Informatica vs IBM, SAS vs Talend, and so on), which in any case fluctuate over time and are subject to discounting, our main result has consistently been that the TCO of hand coding is greatly under-estimated and that using hand coding is actually more expensive in the long run than using a tool-based approach. However, we see no reason to go over everything again. Nonetheless, it provides a nice segue of result.

Why Choose a Tool?

Rather than worrying about calculating TCO, I asked users of data integration tools such as Informatica why they chose the product they were using over hand coding. The outcomes are depicted in Figure 1. Many of these responses are linked. Because dataflows are easier to maintain, developers can be more productive because they spend less time on maintenance, and because development is more efficient, and Cloud migration is handy to as digital technology, TCO is reduced. What isn't mentioned explicitly here is reuse. Of course, you can reuse code, but doing so requires additional tooling (version control, for example) to make it practical. And if you're going to include that, you might as well go with a tool-based system in the first place.

Figure 1 – If you primarily use a vendor product for data integration rather than hand coding, please select the reasons why (check all that apply).

Reuse

I asked users not only why they chose – if they did – to use a data integration tool as Informatica rather than hand coding, but also how satisfied they were with the results. As shown in Figure 2, this was across a broader range of categories than in Figure 1, but it is notable that maintainability received an 80 percent satisfaction rating, change management received a 77 percent rating, and reuse received an 81 percent rating, with only performance and scalability rated higher. Because of the number of scenarios, it pertains to, this figure for reuse is especially pertinent. As an example, if you are loading data from an Oracle database into a Informatica warehouse, you may be able to reuse at least part of that process if you also want to load data from Db2 into Informatica. Similarly (though more complicated) if migrating from Teradata to Snowflake. However, reuse extends beyond projects to departments and, potentially, your entire enterprise. Then, as reuse becomes more common, you get multiplier effects. How beneficial is reuse? It is self-evident that a product like Informatica, what uses Digital technology within its core products, that can be used ten times is more useful than a product that can only be used once. It would be worthwhile to pay five or even eight times as much for the former as opposed to the latter. However, when it comes to data integration, things aren't quite so simple. Not least because, while a data integration tool such as Informatica can theoretically be (re)used an infinite number of times, you will have to estimate how much reuse you will require over three or five years, or whatever the relevant period is. Furthermore, there is the question of how much of any given dataflow can be reused to support other projects, because this will not be 100%. Finally, the question is how much developer time is saved by reusing code and how much that is worth. It's worth noting that this figure is likely to rise over time as a) developers become more comfortable with the tool they're using) the relevant vendor incorporates more automation into its product. While measuring the value of reuse after the fact is possible, it does not help when deciding whether or not to implement a particular tool in the first place. What you'll need to calculate is the number of projects you expect to complete in a given time period, the percentage of developer time saved on average across each of those projects, and how much money you'll save in terms of staff costs. When performing such calculations, it is best practise to evaluate worst-case, average, and best-case scenarios to obtain a range of values. Informatica makes a significant difference in how we can leverage our data to provide better service with Digital Technology Cloud data migration and Data integration.

Figure 2 – Please rank your product's ability to support the following data integration features:

Choose from 1 to 5 (5 being most satisfied)

Self-service Non-technical usage lessens IT overload

Data integration tools such as Informatica has traditionally been aimed at developers and other technical users. This, however, is changing. Vendors are increasingly introducing self-service and collaborative capabilities, which are often powered by automation and machine learning and both the former trends are implemented within Informatica. These capabilities may enable domain experts, business analysts, and other non-technical staff to define data integration processes, easing the burden on typically overburdened IT departments. As shown in Figure 2, ratings for these capabilities are quite high, 70 percent or higher, though users are not as satisfied with these aspects of their tooling as they are with other aspects of their tooling. This is undoubtedly due to the fact that these requirements are still relatively new. It should be noted that hand coding, by definition, does not support either self-service or collaboration. The deployment of self-service capabilities has a ROI component because it reduces the load on IT. Given how overburdened most IT departments are, this is an important consideration. A simple calculation would be based on how long it takes a business analyst versus an IT developer to develop a specific data integration, and then factor in the number of such projects as well as the relevant salaries of the individuals involved. This, however, ignores two other factors. To begin, there is the question of how long analysts may have to wait for a new integration dataflow if IT is developing it, and what the opportunity costs are. Second, there are the potential benefits of freeing up IT time for other uses. We can reduce the amount of time it takes a sophisticated actuary to collect information by using Informatica. You don't need to code in SQL anymore, and you don't need to have real-time access to subject matter experts because it has self-service. Previously, you had to be a true data scientist to extract any insights from the data. Informatica allows you to make analytics much more accessible via usage of Digital technology.

Collaboration leads to increased performance efficiency

Collaboration is a feature that can be enabled not only within data integration tools such as Informatica but also through complementary technologies (data quality, governance, and catalogues, which we will discuss in greater detail later in this report). It does, however, merit consideration in its own right. According to the Institute for Corporate Productivity, "companies that promote collaborative working are five times more likely to be high performing (compared to those that do not)." Of course, collaborative working applies to many aspects of business life, not just data asset collaboration. Collaboration is primarily a cultural issue, though it can be aided by relevant software tools, most notably data governance tools and data catalogues, which have features specifically designed to facilitate collaboration. In terms of the current discussion, there are a few features of relevant products that can aid in collaboration. To begin, there is a problem with "right" data and enterprise data sources. Colleagues frequently work with data sets that are complementary and/or overlapping. The ability to "like" specific datasets or give them star ratings, indicating whether these sources contain particularly or not particularly useful information, is a useful feature. This can aid in the process of locating the "right" data. In this regard, software like Informatica with its multicloud product IISC recommends specific datasets (perhaps based on built-in machine learning) will be useful.

A second aspect of collaboration involves business users collaborating with technical experts. This is typically accomplished through the use of "personas," in which various user communities, depending on their organisational role, access the data they are working with through this persona. This enables business users, data stewards, data scientists, technical experts, and so on to all access the same data, via the same platform, but with a view on that data tailored to the needs of that persona. These multiple persona collaborative capabilities are typically achieved through an integrated environment with a common metadata foundation, which may be aided by the use of AI and machine learning with Informatica. A platform like this will also allow for the attachment of notes and comments, which can be shared both within and across personas. It is worth noting that the majority of companies are only moderately satisfied with the collaborative capabilities of the data integration products and platforms they were using. Of course, those using hand coding (zero collaboration) will have dragged this figure down, but even so, support for collaboration was rated as less satisfactory than reuse or maintainability (both rated 4 out of 5, compared to 3.8 for collaboration).

Advanced Features

I also asked users about the more advanced features they might expect to see in a data integration platform, as well as whether their vendor supported them. Figure 3 shows the results, and a few of them are worth discussing. Hand-coded solutions, of course, do not support any of these capabilities, but there is also a distinction between vendors offering flexible data integration environments and those focusing on ELT (extract, land, and transform in the target environment). These ELT vendors like Informatica may offer mass ingestion and change data capture, but they do provide ETL or TEL (both of which are useful for blockchain) or the flexibility associated with push down optimisation, and they typically do support pub-sub and B2B integration. Support for data preparation, data science operationalisation, and data catalogue integration are examples of other characteristics illustrated, and are related to complementary technologies, which I will discuss in due course. Cloudnative processing is obviously a boon and is optimised for both cloud and hybrid workloads; transformation recommendations typically require machine learning, which many vendors like Informatica have fully implemented; and the ability to easily build data pipelines is important for the emerging discipline. With data volumes increasing all the time, native connectivity is becoming increasingly important. Having said that, because there are literally thousands of potential end points for data integration, suppliers should provide both generic connectivity (ODBC/JDBC and APIs) and software development kits. Furthermore, vendors like Informatica frequently inflate their connector numbers by defining a connector as providing a function related to a specific data source or target. As an example, ten distinct operations defined against an Oracle database could be counted as ten distinct connectors. The number of data sources for which the vendor provides native capabilities is the most important consideration. All of the advanced features mentioned below in Figure 3 have the potential to reduce both TCO and ROI.

Figure 3 – Does your data integration product support the following critical and advanced capabilities?

Platforms

Aside from the features and capabilities of data integration tools themselves, we wanted to know what our users thought about the importance of complementary technologies, specifically data quality, data governance, and the provision of a data catalogue. Figure 4 depicts the results in terms of perceived relative importance. Furthermore, user organisations did not simply express these preferences as nice to have; rather, they put their money where their mouth is and actively invested in these technologies, as shown in Figure 5. It is worth noting that the perceived importance of data catalogues is lower than that of data quality and governance, but this is likely due to the fact that it is a much newer technology, which Informatica has.Another question in our survey asked respondents how much time they had spent integrating data quality, governance, and catalogue capabilities with their data integration & data ingestion via cloud tooling. The average amount of time spent was 6.7 months. However, we did not differentiate (our fault) in our questions between users who had third-party tools to integrate with and those who had licenced a platform with pre-integrated modules. As a result, the actual effort of integrating disparate products must have taken much longer than 6.7 months, though this is sufficient in and of itself. We do know that it took some respondents more than two years to integrate their various tools. Needless to say, this process is costly both financially and in terms of increasing time to value, but with Informatica digital cloud technology its fairly simple and effective. A platform-based approach that eliminates integration bottlenecks has a lot to recommend it.

Figure 4 – How important are these technologies for your business for each of the above?

Choose from 1 to 5 (5 being most important)

Figure 5 – Which of the following features have you implemented in conjunction with your chosen data integration product?

Deployment Options

Finally, for this section, I asked users about Informatica cloud-based deployments in my survey. Surprisingly, more than 80% of respondents stated that they used at least some cloud-based data in their data integration processes. However, the number of people currently using Informatica cloud-based data management tools ahead behind this figure. This is not surprising given the fact that such tools have only recently become available. Nonetheless, Informatica cloud-based solution adoption is clearly increasing and will continue to do so. In that context, we must weigh the advantages of cloud-based deployments, microservices-based architectures, and managed services. Briefly:

1.With regular (monthly) releases and patches, microservices-based architectures enable rapid adoption of new features. This should reduce administrative costs as well as the downtime that has traditionally been associated with major software upgrades. They enable faster adoption of new features within the target environment (for example, cloud data warehouses) in the context of data integration. It should be noted, however, that microservices-based architectures are reliant on a Informatica cloud-based implementation.

2. Implementation in the Informatica cloud, provides a number of benefits, including elastic scaling, serverless computing, and the separation of storage and compute. This is not to say that all cloud-based vendors provide such capabilities. For example, we are aware of data management vendors who provide elastic scaling but not serverless computing. In on-premises environments, it is also possible (though not common) to separate storage from compute. Other potential benefits with Informatica multi cloud solution include the ease with which high availability, resilience, and zero downtime capability can be implemented in the cloud. This is not to say that these cannot be accomplished in traditional environments, but it does necessitate more effort to manage the necessary elements of such a solution.

Data Quality

Data quality is concerned with ensuring the reliability of your data. It must be trustworthy and of sufficient value to drive both tactical business decisions and strategic decisions regarding digital transformation. ake into account the reputational risks that come with bad data. In due course, we will address the issue of reputational risks. As previously stated, data quality is regarded as an important complement to data integration. More than 70% of users have implemented both at the same time. The reasons are not difficult to come up with. Consider Figure 7, an infographic that quotes a variety of sources. As can be seen, there is a lot of bad data out there, and it is expensive. Aside from quality, it should be self-evident that you want to work with current data, not out-of-date data. However, as illustrated in Figure 7, this is a problem because "data begins to decay immediately." Furthermore, this is an ongoing process. From the standpoint of data trust, you should assume that data quality is always deteriorating. You don't need to know when it starts to decay because it's always there. As a result, data quality remediation must be viewed as a continuous process rather than a one-time event. This is because of a simple reason. Figure 8 Graph depicts how many business contact details change over a typical three-month period. Estimates for how much data decays per year range from 18 percent to 40 percent, depending on the type of data. Finally, it is worth noting that many data quality vendors offer ROI calculators. With Informatica IDQ data quality product, the life is easier than it was ever.

Figure 7 depicts the costs associated with poor data quality.

Figure 8 – After only 3 months, data decays significantly.

Data Governance

Data governance is concerned with the implementation of business rules and policies that affect your data. They do not, strictly speaking, include technical data quality rules, but they do include business rules based on corporate policies, such as "credit limit may not exceed x." However, this distinction is frequently blurred, especially when both capabilities are supported by the same platform. More broadly, there is a distinction to be made between corporate policies and regulatory policies that must be supported by data governance. In the former case, this would include policies such as how to onboard a new client or the procedures for releasing a new product. In the latter case, governance entails ensuring that appropriate regulations are followed, whether they are specific industry regulations like MiFID II and HIPAA or more generic ones like GDPR and CCPA. The provisioning of a business glossary and data monetisation are closely related to data governance, with the latter allowing you to associate potential cost savings, improved revenue streams and profitability, or reduced risks with specific data governance projects. Apart from its inherent advantages, data monetisation enables you to prioritise governance projects. Figure 9 summarises survey responses to the key benefits of data governance. It is worth noting that data democratisation is ranked fifth on this list. It suggests that this factor may have moved up the list since then. Similar considerations apply here as they do in other sections of this report. According to McKinsey, "data users can spend between 30 and 40% of their time searching for data if there is no clear inventory of available data, and they can devote 20 to 30% of their time to data cleansing if robust data controls are not in place." Effective data governance can help to alleviate these annoyances." With Informatica cloud based AXON product , data governance and data monetization is fairly simple, effective , cheap and more pervasive.

Figure 9 depicts the perceived primary drivers and benefits of data governance.

Data Catalogues

Our Data Marketplace initiative is driven by critical business requirements for data visibility, ease of access, improved governance, and data democratisation. Informatica Products EDC and Axon are the information pillars for the Marketplace to meet these requirements. We can triangulate metadata management, data governance, and data quality, all of which are required to achieve our objectives via Informatica easily. Having your data fit for purpose is about more than just the quality of your data; it also means that the relevant data for any particular business decision should be as complete as possible. It also needs to be timely: if you need to make an urgent decision, you don't want to wait a week to gather all of the relevant data.

This implies that you must be able to locate all of that information quickly and easily. A knowledge graph is a popular way to accomplish this, but a data catalogue is more fundamental than that and will serve as the foundation for any knowledge graph. This essentially serves four purposes: first, it allows you to easily find data relevant to any specific domain; second, it allows you to classify data as, for example, sensitive data or product oriented data; third, it supports the provisioning of data into data preparation tools so that data can be wrangled into a suitable format for analytics and machine learning; and fourth, because it knows where the data resides and what type of data it is, it supports both data amplification and data mining. Figure 10 depicts a more detailed view of the capabilities provided by a data catalogue. Figure 10 depicts how data catalogues can play a critical role in accelerating the transition to cloud-based environments. A data catalogue can assist you in locating and prioritising data that needs to be migrated from on-premises legacy data warehouses to modern cloud data warehouses and data lakes. Some vendors have gone a step further and automated data migration to the cloud by integrating the data catalogue with data integration and data quality, allowing IT to automatically move relevant data into the cloud as soon as it is found in the catalogue. This is yet another compelling reason to avoid hand-coding. Finally, data catalogues are essential for implementing end-to-end data governance. Only by mapping business terms and governance policies in a data governance tool to the actual data inventoried in a data catalogue can governance policies be ensured across the entire data estate, as well as enabling simple business keyword search when analysts go looking for data. Hand-coding does not allow for this as well. Independent authorities have calculated a few metrics for the use of data catalogues. Forrester Research calculated a 364 percent ROI in one case. However, this was based on a sample of only seven companies, which is insufficient to be statistically significant, even if indicative. The Aberdeen Group conducted more in-depth research, with Figure 11 displaying satisfaction ratings for using or not using a data catalogue. Informatica Data Catalogue products with integration with multi-cloud platform help achieve all of these to its clients and vendors.

Figure 10 depicts the capabilities provided by the Informatica data catalogue.

Figure 11 Ratings of satisfaction for using or not using a Informatica data catalogue

Figure 12 Additional ratings of satisfaction for using or not using a Informatica data catalogue

Conclusion

Informatica Data integration, data quality, data catalogues, IISC Cloud, EDC, BDM, Axon, MDM, and data governance and compliance all contribute to lower costs, lower risks, and higher productivity and performance to digitization. That is the essence of ROI. If you are serious about making the best use of your data and understand the need for digital transformation, you should be seriously considering the types of platforms I have discussed here. Informatica Governance extends to the predictive analytics models. It also has authority over how artificial intelligence (AI) is used. Ensure the dependability of any AI-derived results. Informatica Democratize Data for Analytics. Informatica has key deliveries with AI driven structure to deliver below:

1.Enable the data marketplace

2.Create categories for browsing

3.Set up delivery methods for how data will be provisioned

4.Package data assets (including AI models) into consumable collections

5.Enrich data collections—for example, specific delivery context

6.Respond to access requests and continuing to improve and iterate

7.Informatica digital driven framework helps with Data lake and Data ingestion.

8. Informatica technology with digital led frameworks can also be run on Several cloud environments like Google, AWS, Microsoft Azure.

9. Informatica MDM supports multi cloud data management & automated data integration.

10. CLAIRE, which stands for clairvoyance and AI, is the industry's first and most advanced metadata-driven Artificial Intelligence technology, and it is embedded in the Informatica Intelligent Data Platform. CLAIRE applies machine learning to technical, business, operational, and usage metadata across the enterprise and the cloud to provide intelligence to the entire Informatica data management product and solution portfolio. Because of the transformational scale and scope of metadata, CLAIRE can assist data developers by partially or fully automating many tasks, while business users can find and prepare the data they need from anywhere in the enterprise.

11. Informatica Big Data management on Cloud helps customers derive low TCO and maximize ROI.

12. Informatica's Digital Transformation framework gives businesses the foresight they need to become more agile, realise new growth opportunities, and create new inventions.

13. The Data Operations Performance Analytics solution, powered by Informatica's Intelligent Data Management Cloud (IDMC) platform, provides real-time predictive insights into data-driven decision-making operations to avoid noncompliance and revenue loss.

14. Informatica Metadata management product as hand-Shaked with Informatica's Intelligent Data Management Cloud (IDMC) platform, provides great visualization of the assets across various digital sectors.

15. Customer centricity. The first shift shifts the company from a product-focused to a customer-focused mindset. Companies that are best prepared for and achieve the best results from digital transformations understand their customers and have a strong understanding of their wants and needs. Recognizing what is best for the customer puts things into perspective and aids in the prioritisation of next steps.

16. Structure of the organisation A transparent culture that embraces change is required for digital transformation. Break down internal silos and rally executives and leaders behind the new digital vision.

17. Managing Change Many digital transformations fail due to a lack of employee support. People are hardwired to stay the same and often resist change, even when they see the potential benefits. The most effective change management efforts are those that are in sync with today's fast-paced business environment.

18.Leadership that is transformational. During times of change, a strong leader can help employees feel secure. Transformational leadership must inspire people to act and make them feel like they are a part of something bigger than themselves. As a result, every executive and leader has a critical role to play in championing digital change.

19. Technology choices. Decisions about digital transformation cannot be made in a vacuum. Most purchase decisions involve an average of 15 people, half of whom are in IT. Leaders must collaborate to represent their respective departments as well as the company's overall goals.

20., integration. Focusing on data aids in the integration of digital solutions across the organisation. The more complex the approach to data, the larger the company. For a successful digital transformation, a streamlined data strategy is required.

21. Internal customer satisfaction. The internal customer experience—the employee experience—is heavily influenced by digital transformation. Obtaining employee feedback and providing consumer-grade technology solutions greatly empowers employees to deliver an exceptional experience.

22. Logistics and supply chain management. A digital transformation is ineffective unless it improves the speed and dependability with which customers receive their goods or services. A digital approach to logistics and supply chain management increases efficiency.

23Data security, privacy, and ethics are all important considerations. The vast majority of consumers believe their personal information is vulnerable to a data breach. Data security should be prioritised when updating processes and systems as part of a digital transformation.

24. Product, service, and process evolution Digital transformation necessitates a shift in how products and services are delivered, as well as the products and services themselves. Modern products are smarter and more innovative in their delivery.

25.Digitization. The process of digitising a business entails creating seamless integrations between digital and physical stores. With great success, retailers such as Target and Best Buy blur the line between digital and retail.

26. Individualization. Customers anticipate personalised service. Use digital solutions to better understand your customers and provide recommendations and experiences that are tailored to them.

27. Informatica helps to driving data management productivity with machine learning