Stibo Systems - The Master Data Management Company
Stibo Systems - The Master Data Management Company

Need Big Data? Think Small Data First

 |
March 24 2021
| 4 minute read

Together, small data and multidomain MDM reveal new insights into your business

Over the past few years working in data governance, I have encountered many projects aiming to re-write decision making processes using ”big data.” Data lakes, predictive analytics, AI and a cacophony of other acronyms have for many people veiled the complexity of big data, which pledged to deliver the four golden eggs of data insight; 1) better decision making, 2) happier customers, 3) freedom from risk and 4) identification of new opportunities.

I have much sympathy for the managers I have seen, left to contemplate their befogged bar charts, delivered from obscure data sources and massaged into place by employees with futuristic sounding job titles, unaware of their existence until last week.

I can hear the questions from the management team that follow. “I can see a kind of pattern, but what decision do we need to make? Why is this happening? How sure are we about the trend and the cause?”
An uncomfortable silence ensues.

A recent report by BearingPoint highlighted this challenge, observing that “the heads of big companies act more on gut feel than on hard data.” That is a situation any informed business can no longer find acceptable.

managing small data

The importance of finding cause in data

While it is true, thanks to big data, voluminous sources of seemingly disparate information can now be combined, analyzed and correlated to identify trends and patterns, it does not necessarily follow, that such analysis can explain causes. For example, a trend in customers cancelling contracts does not necessarily mean they are all unhappy. Indeed, how do you guard against bias in your trend analysis in the first place?

So why is it important for you to understand cause? Quite simply, because the data behind cause weighs heavily on data-driven decision making. As correlation does not imply causation, there is significant scope for misjudgment using trend analysis alone.

Big data: Finding patterns and inferring correlations
in large data sets often constitutes goals of big data
projects. The data can be largely transactional,
historical, of dubious quality and without common
(metadata) definition. AI and machine learning
initiatives will, nevertheless, use such sources to
improve analytics and learn how to automate
processes; the goal being to accelerate the
data-to-decision process.


Those suffering most in adapting to new decision-making processes are often more traditional companies whose leaders are still unversed in data culture and its potential impact on business transformation. Their experience, bias and unquestioning belief in the accuracy of the data presented, forms the basis of their sedate decision-making. They witness, in contrast, the arrival of new and disruptive competitors whose data-centric mindset is intrinsic to their culture and vision, making bold yet informed decisions at speed, seeing cause through the lens of “small” data.

Small data: Small data refers to reduced data sets
that are understandable, often simple and collected
dynamically. They often don’t need vast computing
resources to collect and analyze. Small data can
be pertinent and insightful on its own and relate
to a particular event, for example, recording
the customer’s sentiment at a moment in time.


What is happening now? No… right now!

As anyone who manages data can attest, volume and quality are two forces that don’t always go hand-in-hand. As data has gotten ever bigger, the ability to manage and derive value and insight from small data has become increasingly consequential. More digital connections are being made in real-time. Customer attention spans are shrinking, while windows of opportunity are closing faster. It’s an environment that demands faster decision-making, which allows for less time to curate and present the data that supports it.

Small data provides discernible signals to assist in this type of accelerated decision environment. Small data is able to tell you what is happening right here, right now. What is this machine currently doing? Where is that package located? What type of sentiment is your customer expressing?

Small data and multidomain MDM go hand-in-hand

When it comes to data-driven decision making, multidomain MDM is the ideal home to create insight and value from small data. Here’s why.

small data lives in the intersections of data sets and domains
Small data lives within zones of insight created by multidomain MDM.
  • Decision making at the edge needs governed master data

Trustworthy decisions need accurate, transparent data, along with the necessary governance to make them informed and auditable

    • Cause analysis needs master data

Small data describing the context of what is happening now, coupled with its corresponding master data, forms the basis for making better, data-driven decisions

  • MDM improves trend and correlation outcomes

The resulting combination of master data and contextually small data can be fed back into big data repositories, resulting in improved trend analysis

  • Correlated data must be accurate and unbiased to support cause detection

MDM provides the governance mechanism for the required transparency of both master data and the accompanying small data

  • Small data benefits from multidomain MDM

Decisions supported by small data often find themselves at the intersection of multiple domains; for example, the customer (master data), their location (small data), the product (master data), their current basket contents (small and master) and the weather (small). MDM provides an ideal mechanism to unite this data at a moment in time while keeping it under control with data governance policy rules.

  • Cause needs to be shared

For action to be taken based on small data, it needs to be governed and distributed almost immediately. MDM was designed for this very purpose.

Ultimately, it is the ability of MDM to support the accuracy of cause determination that makes it so fit for this purpose, while also supporting the auditability of any decisions taken related to the cause. Managing small data with MDM is part of an augmented MDM strategy, whereby additional business value is developed via the bias of having trustworthy master data.

Small data and multidomain MDM in action

Small data and multidomain MDM are helping to make a big impact in very diverse applications. Here are some examples.

icon_employee_2c Employee retention

Detection of small changes in the ways workplace collaborative tools are used, such as meeting agenda changes and numbers of messages exchanged, can capture levels of motivation and help to establish retention strategies.

• Master data domains: employee, location, organization, role

• Small data: collaboration tool, usage frequency, date, duration

icon_shopping_cart_2c Personalized shopping

New levels of personalized shopping will be enabled using a small data approach coupled with MDM.

The combination of wearable and consumer held devices, sensors on shelves, surveillance cameras and tagged products has the potential to produce highly valuable small data sets that can drive improvement strategies in the customer’s in-store experience.

• Master data domains: customer, product, location

• Small data: Location, time, sensor data

icon_support_2c Customer service

When a customer contacts a helpdesk, recognition of the phone number can accelerate handing of the enquiry. For example, knowing that a passenger’s flight is delayed, their call is likely to be related and automated call answering can respond accordingly.

• Master data domains: customer, product

• Small data: phone/device, location, service in progress

A small data approach can have a significant impact on the ability to take fast, data-driven decisions. Multidomain MDM provides the ideal place to host your small data against the key domains with which it is associated – customer, product location, etc. Multidomain MDM provides the governance capabilities needed to ensure that data-driven decisions are based on reliable, auditable and transparent data.

Related content

To learn more about innovative ways to find unique, valuable insights in your data, listen to Making Data Human, part of our Data Matters podcast series.


Master Data Management Blog by Stibo Systems logo

During the past 20 years, Darren has been advising companies on selecting and implementing software tools that support their data governance strategy. As Director of Industry Strategy, Darren helps business leaders understand and quantify the positive impact that good data governance and, in particular, master data management, can have on their organization.



← Previous Post
Next Post →