You receive a notification on your phone that a critical shipment from your China factory has missed its filing deadline with the customs broker. Your logistics manager is alerted that there is an 80% chance that the components he’s waiting for are likely to be delayed another 48 hours by excessive port traffic and your GTM software advises diverting the shipment to an alternate port facility. Your compliance officer is informed that there is a 95% chance that a shipment of parts from Malaysia is likely to be held for up to three days to be subjected to a detailed customs inspection.
If you think this type of information would be of great assistance to your supply chain business planning and operations, you are not alone. It is this type of integrated data and communications that are becoming the backbone of the Big Data led revolution underway in supply chain.
The human brain can only process and make use of a limited amount of information before it becomes overwhelmed and unable to effectively recognise patterns and trends. But powerful algorithms and the software platforms they drive can take in almost unlimited numbers of data points and process them to generate insights impossible for an individual or even an entire organisation of individuals to identify. And powering this technology-driven transformation of supply chain is Big Data.
BIG DATA VS SMALL DATA
To really understand how technology is transforming supply chain, it is important to understand how Big Data differs from any other form of information gathering. Data has always been crucial to efficient supply chain operations so what has actually changed in recent years? How is “Big Data” different from the analysis of “small data” that has always occurred in the industry?
Big Data refers to sets of both structured and unstructured data with so much volume that traditional data processing systems are inadequate to cope with it all. It can be further defined by some of the basic properties that apply to it:
- Variety – data being generated from a wide number of varied sources
- Volume – while there is no set distinction between where small data stops and Big Data starts, Big Data requires large storage requirements for the data, often measured in many multiples of terabytes
- Velocity – the speed at which the data can be acquired, transferred and stored
- Complexity – difficulties encountered in forming relevant relationships in data, especially when it is taken from multiple sources
- Value – the degree to which querying the data will result in generating beneficial outcomes
The most important property related to Big Data is as the name implies, volume. We normally think of data purely in terms of text or numbers but it can include everything from the billions of emails, images, and tweets generated every day. In fact, data generation is expanding at a rate that doubles every two years. And human and machine-generated data is growing at 10 times the rate of traditional business data. IT World Canada projects that by 2020, you would need a stack of iPad Air tablets extending from the earth to the moon to store the world’s digital data.
But the real focus behind a preference for Big Data analysis over small data systems is the ability to uncover hidden trends and relationships in both structured and unstructured data. In most cases, using small data collection and analytics processes simply cannot identify crucial information in a timely manner to allow key decisions to be made or opportunities to be taken advantage of. In other cases, using small data systems is simply a waste of resources and leads to disruptions to supply chain operations.
By contrast, if used correctly Big Data is the key to enhancing supply chain performance by increasing visibility, control, agility, and responsiveness. Making decisions based on high quality information in context can benefit the full range of supply chain operations – from demand forecasting, inventory and logistics planning, execution, shipping, and warehouse management.
BIG DATA POSSIBILITIES
Big Data analytics becomes a vital tool for making sense of the huge volumes of data that are produced every day. This data comes from a whole range of activities undertaken by people associated with supply chain, whether they be customers, suppliers, or your own staff. The range and volume of this data is continuously increasing, with billions of data points generated by sources we see as directly linked to supply chain such as network nodes and transaction and shipping records as well as other areas that more indirectly impact supply chains such as retail channels and social media content.
But it is increasingly becoming necessary to harness this data in order to remain competitive. This is evident from statements made by people such as Anthony Coops, Asia Pacific Data and Analytics Leader at KPMG Australia, who believes that “Big Data is certainly enabling better decisions and actions, and facilitating a move away from gut feel decision making.” At the same time, he recognises that solutions need to be put in place that allows for people and organisations to have complete faith in the data so that managers can really trust in the analytics and be confident in their decision making.
The need for confidence in the analytics is evident when considering the examples such as where GTM software has the information and capabilities to advise ahead of time to divert shipping stock to an alternate port or that a product is likely to be held up in customs. These types of decisions have potentially large financial consequences but when implemented correctly, it is easy to see how supply chain operational efficiency can be significantly boosted by effective use of Big Data analytics.
Many organisations are also using Big Data solutions to support integrated business planning and to better understand market trends and consumer behaviours. The integration of a range of market, product sales, social media trends, and demographic data from multiple data sources provides the capability to accurately predict and plan numerous supply chain actions.
IoT and AI-based analytics are used to predict asset maintenance requirements and avoid unscheduled downtime. IoT can also provide real-time production and shipping data while GPS driven data combined with traffic and weather information allows for dynamically planned and optimised shipping and delivery routes. These types of examples provide a glimpse into the possibilities and advantages that Big Data can offer in increasing the agility and efficiency of supply chain operations.
DISRUPTIVE TECHNOLOGIES
What is driving these possibilities is the development of numerous disruptive technologies as well as the integration of both new and existing technologies to create high-quality networks of information. Disruptive technologies impact the way organisations operate by forcing them to deal with new competitive platforms. They also provide them with opportunities to enter new markets or to change the company’s competitive status. By identifying key disruptive technologies early, supply chain organisations can not only be better placed to adapt to changing market conditions, they can also gain a distinct advantage over others in the industry that are reluctant to embrace change.
In terms of Big Data based disruptive technologies, these are largely driven by the effects of constantly evolving and emergent internet technologies such as the Internet of Things combined with increased computing power, AI and machine learning based analytics platforms, and fast, pervasive digital communications. These technologies then act as drivers that spawn new ways of managing products, assets, and staff as well as generating new ways of thinking about organisational structures and workflows.
IOT
After being talked about for many years, we are now starting to see the Internet of Things really taking shape. There will be a thirty-fold increase in the number of Internet-connected physical devices by 2020 and this will significantly impact the ways that supply chains operate.
IoT allows for numerous solutions to intelligently connect systems, people, processes, data, and devices via a network of connected sensors. Through improved data collection and intelligence, supply chain will benefit from greater automation of the manufacturing and shipping process becomes possible through enhanced visibility of activities from the warehouse to the customer.
Cloud-based GPS and Radio Frequency Identification (RFID) technologies, which provide location, product identification and other tracking information play a key role in the IoT landscape. Sensors can be used to provide a wealth of information targeted to specific niches within supply chain such as fresh produce distribution where temperature or humidity levels can be precisely tracked along the entire journey of a product. Data gathered from GPS and RFID technologies also facilitates automated shipping and delivery processes by precisely predicting the time of arrival.
BIG DATA ANALYTICS
Big Data analytics encompasses the qualitative and quantitative techniques that are used to generate insights to enhance productivity. The more supply chain technologies are reliant on Big Data, either in their business model or as a result of their impact on an organisation, the more organisations have to rely on the effective use of Big Data analytics to help them make sense of the volumes of data being generated. Analytics also helps to make it possible to understand the processes and strategies used by competitors across the industry. Using analytics effectively allows an organisation to make the best decisions to ensure they stay at the forefront of their particular market sector.
As corporations face financial pressures to increase profit margins and customer expectation pressures to shorten delivery times. the importance of Big Data analytics continues to grow. A Gartner, Inc. study put the 2017 business intelligence and analytics market at a value over USD$18 billion, while the sales of prescriptive analytics software is estimated to grow from approximately USD$415 million in 2014 to USD$1.1 billion in 2019.
Over time, the effectiveness and capabilities of analytics software also continue to improve as machine learning-based technologies take forecast data and continually compare it back to real operational and production data. This means that the longer an organisation operates its analytics software, the iterative nature of artificial intelligence powered algorithms means that the performance and value of the software improve over time. This leads to benefits such as more accurate forecasts of shipping times or supplier obstacles and bottlenecks.
CONSUMER BEHAVIOUR ANALYSIS
Although it may not initially seem as vital to supply chain as other disruptive technologies, consumer behaviour analysis can have a huge impact on businesses working in supply chain, especially e-commerce businesses. Known as clickstream analysis, large amounts of company, industry, product, and customer information can be gathered from the web. Various text and web mining tools and techniques are then used to both organise and visualise this information.
By analysing customer clickstream data logs, web analytics tools such as Google Analytics can provide a trail of online customer activities and provide insights on their purchasing patterns. This allows more accurate seasonal forecasts to be generated that can then drive inventory and resourcing plans. This type of data is extremely valuable and is crucial for any organisations operating in the e-commerce space. While retailers and consumer companies have always collected data on buying patterns, the ability to pull together information from potentially thousands of different variables that have traditionally been collected in silos provides enormous economic opportunities.
POTENTIAL DRAWBACKS AND CHALLENGES
Despite the huge opportunities presented by implementing Big Data powered solutions, there can be intimidating barriers to entry when it comes to putting in place Big Data collection and analytics solutions. This can emerge across a range of areas including the complexities around data collection and the difficulties of putting in place the technologies and infrastructure needed to turn that data into useful insights.
GETTING COMPLETE BUY-IN
One impediment to adopting a holistic Big Data approach centres around having unified support at all levels of your company to adopt comprehensive Big Data systems. Management commitment and support are crucial and large-scale initiatives of this type usually occur from the top down. However, Big Data analytics type initiatives usually originate at mid-level, from people who actually collect and use data day to day. This means that for this issue, the need for implementation must often be sold upwards. In some cases, upselling the importance of Big Data to management that doesn’t understand why that type of expense is necessary is extremely challenging.
SOURCING CLEAN DATA
One of the other main challenges is undoubtedly sourcing appropriate and consistent data. There’s no use getting high-quality data if it doesn’t directly apply to your particular market sector. Nor is there much benefit to be gained from obtaining high-quality data but being unable to consistently source it at the same regularity to enable it to build a long-term profile of the company’s operations and market forces. These challenges are often related to technical issues such as integration with previously siloed data or data security concerns.
Richard Sharpe, CEO of Competitive Insights, a supply chain analytics company, believes that the data quality problem is a complex issue that can have many different causes. However, he believes that these challenges can be overcome by management having a clear understanding of what they’re trying to achieve. “You have to show that what you’re ultimately trying to do with supply chain data analytics is to make the enterprise more successful and profitable.” This then leads to support being provided by company leadership who, in tandem with operations managers, can develop the processes required to govern quality data collection. This includes proper consultation with subject matter experts who can help ensure that all data is properly validated.
MANAGING DATA VOLUMES
New technologies make it possible for supply chain organisations to collect huge volumes of information from an ever-expanding number of sources. These data points can quickly run into the billions, making it challenging to analyse with any level of accuracy or lead directly to innovation and improvement.
This means that despite many organisations embracing Big Data strategies, many do not actually derive sustainable value from the data they’re accumulating because they begin to drown in the sheer volume of data or don’t have the appropriate software and management tools to make use of it. A common phrase used to summarise this effect is “paralysis by analysis”. Without a thorough understanding of the technologies and systems needed to process and store the data collected, this can be an easy condition for an organisation to become afflicted by.
BUILDING THE INFRASTRUCTURE
Companies need to invest in the right technologies to have a true 360-degree view of their business. And in many cases, these technologies can involve large initial capital outlays. Getting the infrastructure in place is key to being able to collect, process, and analyse data that enables you to track inventory, assets and materials in your supply chain.
Putting in place the infrastructure may also require additional training expenses, so that staff are properly trained in how to use new software platforms or to maintain sensors and other new IoT devices. In some cases, this will extend to requiring hiring new talent capable of using and interpreting new analytical tools.
CONCLUSION
Big Data offers huge opportunities to supply chain organisations, as vital information contained within multiple data sources can now be consolidated and analysed. These new perspectives can reveal the insights necessary to understand and solve problems that were previously considered too complex. New insights can also encourage organisations to scale intelligent systems across all activities in the supply chain, embedding intelligence in every part of the business.
There is also no doubt that implementing comprehensive Big Data solutions can involve new and significant challenges. However, once the new infrastructure and processes are in place, the nature of modern Cloud-based networks allows for data to be accessed easily from anywhere at any time. It also allows for other benefits beyond cost reduction and production gains to be realised over time, such as ongoing rather than just one-off efficiency gains and improved transparency and compliance tracking across the entire organisation.
Bastian Managing Director, Tony Richter, is a supply chain industry expert with 7+ years executing senior supply chain search across APAC. He works exclusively with a small portfolio of clients and prides himself on the creation of a transparent, credible, and focused approach. This ensures long-term trust can be established with all clients and candidates.
To find out more about the challenges and opportunities offered by Big Data to supply chain, or to gain a deeper understanding of the technology and Big Data trends currently underway in the industry, talk to an industry expert at https://bconsult.io/
