Dip Your Toe in the Data Lake
You’ve surely heard about the concept of “Big Data.” Whether you work in technology, financial services, or marketing, Big Data has become one of today’s most used buzz words. Companies are collecting more data about customers, vendors, business processes, finances and many other subjects than ever before. It is important that you are able to leverage the Big Data into something valuable. You don’t want your data to become something you don’t or can’t use. You don’t want data to be simply a cost on your income statement rather than something that is driving additional value.
Some companies make a decision to quit storing data or to trim down their data after they aren’t able to leverage value from their data. Maintaining their data becomes too costly and with no value being produced they start archiving and divesting themselves of old data.
For companies that leverage Big Data and data science, however, keeping all of the data isn’t a cost…it is an opportunity. One way to keep all of the information available is to create a “data lake.” One definition of a data lake is “a large-scale storage repository and processing engine [that provides] massive storage for any kind of data, enormous processing power and the ability to handle virtually limitless concurrent tasks or jobs.” The data lake is simply a way to store ANY data you can contain and positions that data to be analyzed and utilized.
For many companies, Big Data can end up being a “glacier” of information—impressive in size and scale, but without changes the value is inaccessible and intimidating to even consider. A data lake, in turn, could be thought of as a melted down version of that glacier. The data lake enables Big Data action. The data lake facilitates turning data into value and providing the insights you need.
As the Chief Technology Officer at VeriShip which is focused in the parcel logistics industry, I’ve seen our data lake become the key tool in helping our customers negotiate the best possible carrier rates for their shipping profile. Most shippers wouldn’t have the data or time to consider the numerous data points that
make up their shipping profile. Our data lake allows us to utilize data from disparate sources to maximize value we are producing for our customers.
Every shipper can benefit from a deeper understanding of the many data points that make up their parcel spend. From invoices to transit data, communication errors to tracking information, there are insights to be gained by taking a deep dive into the data of a package, account, customer, distribution center and more.
Here are a few of the ways we have been able to leverage our data lake to provide our customers with real insights that had a measurable impact on their parcel spend:
- Benchmarking: With access to more than a decade of data across 3,300+ customers, our data lake empowers our business with the ability to compare one customer’s performance against everyone else’s, identifying areas where contract rates should be better and where they are over performing.
- Forecasting & Simulating: The data lake allows us go beyond benchmarking and into simulating contract behavior. By identifying customers of similar shipping profile (volume, shipping type, distribution footprint, contract discounts) we can forecast or simulate changes to shipping behavior, contracts or distribution methodology. We are able to forecast a customer’s current rates and shipping behavior against alternatives. Thanks to our data lake we are able to identify savings opportunities along with forecasting options for an improved shipping experience for our customer’s customers.
- Identifying Inconsistencies: Whether a carrier has a late delivery, an incorrect charge, or a simple typo caused by human error, our data lake allows us store and identify the error. We are able to process this against past data and resolve it in real time. This allows us to continue to process invoices for our audit along with providing customers valuable insight into their shipping, without delay, within the VeriShip Intelligence Platform.
“Big Data” as a term exists for a reason; in today’s world data is growing exponentially. Remember though, that without the proper resources and technology, it won’t be possible to leverage Big Data to drive actual value creation.
So consider this. The next time someone starts throwing buzz words at you like Big Data, make sure you’re asking them the most important question of all: Are you actually using your Big Data to create value or is it just sitting there buzzing?