In a Forrester survey, more than 2,000 firms were asked which is the focus of the organization’s digital business transformation. 34% of them chose data and analytics, which shows that Data and Analytics is the top priority in the digital transformation process.
Why advanced data and analytics is a key focus of digital transformation. AISHU commissioned Forrester Consulting to launch a research in June. We interviewed 214 decision-makers responsible for data management and analytics, strategy, and digital transformation. They showed high expectations of using data to drive transformation. 82% of respondents want to build a data-driven culture, drive the evolution and upgrading to stay ahead of their competitors. Sadly, according to their self-assessment, only 28% of them consider themselves to be at a mature or complete level in terms of data-driven operations and data-driven innovation.
Multiple Challenges Faced by Enterprises to Realize Data-driven
Why did this happen? Because to improve data management, all of the dimensions - strategy, technology, structure and process - are essential, and there are many challenges in any of the dimension.
1. Data Strategy
69% of respondents believe that there are different perceptions of data value and assets within the organization, and the lack of a unified goal makes it difficult to collaborate effectively. Talent and skills are also constraining the implementation of the strategy. 67% of respondents said they lacked the professional data talent, as well as the experience and expertise to address large volumes, and diversified types of data. From a cultural perspective, 67% of respondents said that the culture and mindset of data-driven operations and innovation needs to be further enhanced. Many respondents said that data strategy must be an executive-sponsor project, otherwise it would be very difficult to drive collaborations and implement the strategy.
2. Data Technologies
The variety of technologies and the complexity of business applications are generating technical challenges. More than 75% of respondents said that their existing data infrastructure is outdated and unable to fully utilize their diversified data, and they lack the common data access capabilities to enable faster business adaptiveness and continuous business innovation.
When they looked to external partners, 79% of respondents said they lacked a partner with the full range of t service capabilities to help them solve the data problems. Particularly, 82% of respondents said that existing technologies mainly focused on structured data, less on semi-structured and unstructured data. More than 80% of respondents said digital businesses are generating large amount of unstructured or semi-structured data, requiring new technologies and integrated product portfolio to manage these data.
3. Data Silos
The structure and process challenges are leading to a severe problem existing for many years: data silo. With the rise of the data lake concept, data silo has another being: data swamp. Among the firms who are building their data lakes, 1/3 of them just get data swamp. Over years, the data silo problem seems to get even worse. 65% of respondents said that as the proportion of digital business are increasing, new data silos are created by the new business application systems. However, data silo is not a solely technical problem, it more comes from structure and process. 67% of respondents said, data is scattered across different application systems and departments, resulting in many data silos, and around 60% said that data silos create breakpoints in internal collaboration, make it difficult to fully realize the value of data assets.
Reshape Digital Productivity with Multidomain Data Capabilities
How can we address these challenges and help firms to leverage internal and external data to improve digital experience, digital operations, promote digital ecosystems, cultivate digital innovation and deepen digital transformation? Forrester believes that firms can’t rely on simple, isolated data products and lack strategy, structure and process support. Firms need multidomain data capability, to reshape data productivity.
- Data Productivity
According to the three basic elements of productivity, the three elements of productivity are labourer, means of labour and subjects of labour.
Labourers is about who is the user of the data within the firm? It cannot be limited to just technicians. As data becomes a common resource within the firms, anyone can consume the data products according to their authority. Therefore, in terms of data productivity, the labourer should be the cross-organization users.
Second is means of labour. For data, what kind of production tools do firms need? Database, NoSQL, data lake, these are far not enough. Data production tools need to have full-stack functions, to access the diversified raw data, utilize data assets, and monetize data with knowledge and intelligence.
Finally, subjects of labour. Firms must be able to handle all scenarios, all types and all-lifecycle data.
The combination of cross organization users, full stack functionality and multidomain data is the multidomain data capability we need.
- Multidomain Data
Multidomain Data means data from all scenarios, with all types and across full lifecycle.
1. All Scenarios
The enterprise applications are originally designed to meet business needs, with PLM in R&D, MES and planning in production, market automation, and CRM. What's more, in some firms, for some historical reasons, different product lines use different PLM for R&D, different scheduling systems for production, different MES for different factories and so on.
At present, many firms have online channels, but at the same time many firms use different systems for online and offline, which brings difficulties for operations. Therefore, the first step to realize the data capability of all scenarios is to integrate data from various application systems to eliminate data silos. Respondents also plan to connect data from different business systems through data asset mapping and knowledge graph to unlock as much data as possible within the firm to provide a 360 view of the business to support key business decisions and optimize operational performance.
2. All Types
Currently, firms are investing in data technology focusing on structured and semi-structured data. And there is a distinct lack of investment in the vast amount of unstructured data. As we discussed, multidomain data capabilities need to enable cross-functional users. Business users and key decision makers rarely work directly with structured data. Instead, they mainly work with unstructured data. One of the executives in the survey said that unstructured data reflects the nature of the business and can guide business optimization.
89% of respondents want to use unstructured data to gain insights into the customer journey and find ways to improve customer relationships. Over 85% of respondents believe that unstructured data is an important source of asset. Fully utilizing unstructured data can enhance the digital support for business operations, enhance customer satisfaction, understand the problems behind historical sales data, and find the direction of improvement, thus enhancing the core competitiveness.
The vision is good, but the respondents said the mainstream way of handling unstructured data is still manual tagging and cannot process it with content analysis. As a result, 64% of respondents want to build unstructured data processing capabilities with middle-office mindset, and 66% of respondents want to strategically place structured, unstructured, semi-structured and other data assets under a unified data management system to provide development, deployment and maintenance services for business analysis, and business operations.
2. All Lifecycle
69% of respondents said that firms need to strengthen their data lifecycle management. The data lifecycle could be long. When developing data products and data applications, firms need to discover and capture relevant, high-quality data, where the data objects might be a table or data assets that have already been developed and tagged with a business label.
- Reinvent Data Productivity Requires Full-stack Functions
Finally, from means of labour perspective, reinventing data productivity requires full-stack functionality, which help firms to access, utilize and monetize data.
1. Raw Data Layer
At the lowest level, the raw data layer, we need a variety of different products and technologies to store data across the multi domains. From data lifecycle perspective, hot data is stored in high performance products such as flash, in-memory databases. As the data gets cooler, it is stored in more cost-effective storage through backups and archiving. From data type perspective, we need relational databases to store highly structured data. For document, time series and graphs, we need document database, time series database and graph database or multimodal database.
The above products are mainly for structured and semi-structured data, but there is a large amount of unstructured data in the operation process: like documents carrying key information, contract images, customer service audio and video. For these unstructured data, firms need unstructured data lake. These data sources cannot be isolated, otherwise they will create data silos and will require many ETLs. Data virtualization and data fabric can be used to create a complete, single view of trusted business data without real data movement. The main use cases include customer 360, real-time BI, advanced analytics and enterprise search.
2. Data Asset Layer
Accessing data is not enough, we need data assets. Therefore, we need data preparation, cleaning and processing, and data integration to transform raw data across the multidomain into strategic data assets with a certain business theme, for a certain business scenario and across business scenarios. In this layer, firms need comprehensive data quality capability. In addition, there needs to be a standardized and tightly integrated business labelling system to form Data Catalog for structured and unstructured data through intelligent data analysis and data mapping. Data security and data governance are important throughout the lifecycle of a data asset.
We cannot rely on isolated data governance teams or fragment data governance products. In the survey, 88% of respondents said they need to embed data governance into their business processes to enable just-in-time governance. We also need DataOps to speed up the data assets development. The DataOps framework has four key components: enablement to define the goals and values of the data product; development to focus on collaborative development; and deliver focuses on generating data services or packaging them into data APIs; governance includes business process validation, code quality checks, automated testing and ethical data checks. As data has risen to become a key factor of production, firms need to make full use of their internal data and seek for external data to generate greater value, but customer privacy cannot be violated, so privacy preserving technologies are gaining popularity to realize cross-organization data utilization without compromising customer privacy.
3. Knowledge & Intelligence
The top layer is to extract knowledge and intelligence to realize business value. In fact, data itself has no fixed value, its value relies on the business behind. In our interviews, some data leaders told us, if the business case is not clear, he would rather keep the data at rest. For example, through the unified management of unstructured data and intelligent analysis, we can refine the knowledge of different scenarios: How should a contract be composed? How should we issue a paper? How should supply chain processes be accelerated? How to solve a customer's problem during agent calls?
All of these use cases can form knowledge services. Respondents from biomedical industry focus on smart OA, which helps employees access documents that carry business knowledge in a timely manner to improve productivity. They also focus on smart production prediction, which uses historical data as well as real-time external data to predict changes in sales and schedule production to maximize operational efficiency. More than half of the respondents in the financial services sector focus on risk management through real-time monitoring across all channels.
These are all contents of how to reshape data productivity with cross organization users, full-stack functionality and multidomain data. For more information, please read the upcoming whitepaper "Improve Multidomain Data Capabilities for Data-Driven Operations".
Watch the AISHU SMART 2022 replay to hear from the principal analyst of Forrester Consulting about the Multidomain Data Capabilities: https://smart2022.aishu.cn/pc/248?channel=main&vodId=dfa043da&vodTitle=Danny%20Mu.
