This blog focuses on strategies to transform enterprise data into strategic real-time AI driven insights, and outlines the requirements for Board Directors, CEOs and leadership teams to increase their data literacy management skills.
There have been many definitions on what is big data over the years, ranging from Gartner describing big data analytics as high volume, high velocity and high variety assets that aid in decision making. Other academic experts, like Ackoff has focused on the conversion of data into cognitive value or wisdom, where an organization focuses first on data, then information, then into knowledge and then into meaningful insights or wisdom.
According to Wiki Encyclopedia, big data is a field that treats ways to analyze, systematically extract information from, or otherwise deal with data sets that are too large or complex to be dealt with by traditional data-processing application software.
According to Technavio, the big data market will witness an incremental growth of over USD $247 Billion at almost 18% CAGR during 2021-2025. This incredible growth has been driven by the acceleration of adopting consumer based applications resulting in huge volumes of both structured and unstructured data.
With the data tsunami and growing adoption of Industry 4.0 digital transformation programs, big data analytics to derive more wisdom (aka insights) is increasingly a core competency for critical operational and managerial decision making practices.
MORE FOR YOU
Technology architectures of moving all corporate data to the cloud for end users to easily access and distribute data access across multiple machines, while using the pooled power and storage to surmount monolithic bottlenecks is a top CIO priority globally.
I remember when Hadoop was released in 2006 by Apache, co-founded by Doug Cutting and Mike Cafarell as a project, originally based on Google’s Google File System Whitepaper. Amazon is even based on the Apache Hadoop, a Java based software programming framework that supports the processing of large data sets in a distributed computing environment.
Fast forward to today and there are hundreds of leading big data platforms to choose from, including market leaders like: Databricks Lakehouse Platform, GoodData, Google BigQuery and Google Cloud, Cloudera, Hortonworks Data Platform, IBM SPSS, Microsoft SQL Azure Server Platform, Pentaho, Snowflake, etc.
Top 5 Challenges of Big Data for Leaders to Take Seriously
1. Access to Knowledgeable Professionals – Big Data management is complex business and requires skilled expertise, ranging from: data software engineers, data mining analysts, data visualization experts, data scientists, communication and change management expertise and most importantly, business process experts that understand the process linkages and metric outputs to operate the business. There are risks in every large multi-national in data stewardship and in middle management and emerging companies, the risks are even higher. Do a quick survey of your board of directors, your CEOs and executive teams, and ask these two questions: Is anyone certified in data management? Then go to your next leadership level, VPs, and Directors and ask the same question. If you want to become more alarmed, drill down further and ask how many of the executive team are trained on AI and Machine Learning and Advanced Analytics and repeat this question at the second tier level. I have been advocating more accelerated digital literacy training for all management and certifications to modernize more efficiently, and I must admit in every organization I speak to these days on my zoom calls, data lineage and data accessibility and data stewardship guard rail challenges persist everywhere.
2. Ensure Actionable Insights – It’s easy to get lost in the data wrangling world and whipping up Power BI or Tableau visualizations, etc. – but the real key is ensuring that organizations have actionable insights that business users value and own for continuous improvements. Increasing visualization design skills is critical in enabling these tools to be designed carefully and also ensuring adherence to diverse standards like the International Business Communications Standards (IBCS) which can help to derive best practices for business analysis reporting, and most importantly organizations must ensure that they are paying greater attention to ADA and WCAG compliance standards for accessibility.
3. Laying Data Foundations the Right Way – many companies fail in their big data initiatives in their vision to migrate all customer and supplier data to the cloud, then providing end users with the data sourcing tools and visualization sense making toolkits, only to find that there is a lack of insufficient knowledge and understanding of what the data fields, elements mean and how they are generating formulas to predict metrics. Having sufficient data dictionary, data lineage and data ownership operating frameworks for data stewardship with risk controls is critical to master – otherwise, erroneous conclusions will be made in data insights driving the wrong actions versus the right actions.
4. Mastering Unstructured Data and Documents – The majority of a company’s knowledge today is sourced in unstructured data sources where most of the wisdom really is and it is not easily accessible or being mined due to insufficient expertise and policy making and controls on managing unstructured data sources. Note: There’s an often-repeated statistic that 90% of all data that exists today has been created in the last two years. IDC is already predicting that the total sum of the world’s data will be 175 Zettabytes by 2025 up from 33 zettabytes in 2018. To appreciate the magnitude of this point, a zettabyte is roughly 1,000 exabytes. An exabyte has the capacity to hold over 36,000 years worth of HD quality video…or stream the entire Netflix catalog more than 3,000 times. In addition, over 90% of an organizations data is estimated to be unstructured data, from your powerpoint, word, excel documents, text files, social media, videos, etc., and growing at over 55-66% each year. Being able to source this knowledge is difficult to track and manage and only with advanced AI search capabilities will companies be able to unlock the challenges of unstructured data. Leading companies profiled by Gartner Group recently include Canada’s Coveo, Google, IBM, and MindFreeze (what a cool name!). See more Gartner Group info on advanced AI search solutions here.
5. Data Security. I recently spoke at an international conference on Cybersecurity discussing the value of AI innovations but also how advanced cybercriminals are in always staying ahead of the market investing in their own digital transformation strategies. With knowledge increasing due to Smart Cities, IoT always on connected devices, mobile (Bring Your Own Device – BYOD to work trends), and cloud initiatives, then overlaying AI and machine learning technologies we have created a perfect world for a perfect storm where these cyber-criminal engineers can more easily distribute malware in highly targeted methods, and reach larger audiences with move invasive probes. Cybersecurity is one of the top board director and CEO concerns and it must be taken very seriously and increasingly as more employees are working from home, opening up more risks and vulnerabilities.
Organizations need to plane for traceability to ensure core business processes and operating functions support big data perspectives, both top-down and bottom-up, and be agile to meet diverse stakeholder requirements, at the same time, educating stakeholders on data management risks and ensuring proper controls are in place for sourcing data and evaluating production risks are areas all companies need to improve upon. It’s easy to use words like digital transformation, but remember CIOs can only go so far in creating strong technical infrastructures to manage data and enable access. Business leaders must improve their leadership skills and knowledge in data stewardship, and in AI and Machine Learning areas. It’s a business imperative to evolve.
A good book to read is The AI Dilemma to help board directors and CEO’s improve their operational knowledge of the importance of data management and AI applications. In addition, the Forbes Channel blogs that I have written over 2021 provide learning and development knowledge to help leaders increase their digital literacy in the field of advanced analytics, powered by AI.
Ackoff, R.L. (1989).From Data to Wisdom. Journal of Applied Systems Analysis, 3-9.