If you’re looking to upgrade your pricing by leveraging the tools and benefits of a pricing solution, then you’ll need to get your data ready for implementation. And if you’re thinking that’s going to be as simple as “download all, upload all,” … you may want to think again.
To ensure project success, you need to dedicate time and focus to exploring the intricacies of your data (where it sits, how it’s stored, who owns it, what it does, etc.) and how it feeds your business processes.
A lack of clarity or completeness in data can cascade through your project, slowing down, even halting, work or creating the need for reworking where data is invalid or incorrect. In order for software to be implemented effectively, the scope of the data that supports it needs to be well understood. Otherwise, it’ll be like trying to build a house without a foundation.
At Pricefx, we love getting dirty with data and helping our customers achieve pricing excellence with theirs. But we’ve seen many a company stumble at this first hurdle because they didn’t fully understand the importance of understanding their data or what that really entailed.
The goal of this article is to take that pain away for anyone getting their data ready to be used in a pricing solution.
In this article, we’ll explore:
What constitutes a business process?
What data sourcing is and why it’s important
How to go about data scoping
How to achieve data integrity
Why accessibility of data is crucial to project success
When to engage a vendor
What to expect from them and what they will expect from you
How long could your project take and how to keep that to a minimum
How to maximize chances of project success
The better defined and prepared your data is prior to engagement with a pricing vendor, the better your chances of staying on time and on budget.
So, let’s dive into the nitty-gritty of what “getting your data ready” really means.
First Pillar of Data Readiness: Identification
Identification is about understanding the data landscape within your organization and involves:
identifying your business processes
working out where your data is stored and who owns it (data sourcing)
pinpointing what data each business process requires (data scoping)
Let’s start at the beginning:
Identify Your Business Processes
A business process should be defined as a set of steps you go through in order to do your pricing. Wherever there are significant amounts of variation in the way you price, you have a separate business process. For instance, you’re a chemicals division and adhesives division prices entirely differently, so they are two different business processes.
The number one problem we see in our customers is in their understanding of their business processes. Some struggle because their processes are informal or even undocumented. Others simply don’t understand them; the pricing team sends something to accounting and gets something different back. They don’t know what happens in between. And while it’s a clear step in the process, it’s a black box.
Understanding the context of everything that happens in your pricing processes is your starting point. Your process may (and probably should) change with the adoption of a pricing solution. You should consider what your target state might look like while staying grounded in the needs of the current process.
Working out Where Your Data is Stored and Who Owns It
Once you have identified all your business processes, you move on to data sourcing; figuring out where the data your processes require is stored. It may be in a variety of different places, like internal systems, external providers, Excel, direct user input, etc. You need to know what systems are involved, who owns those systems, and who your relevant subject matter experts (SMEs) are.
It’s not uncommon for companies to have five or six different versions of the same data floating around that various teams are working from. You want to make sure you’re going straight to the horse’s mouth for yours. This is where data ownership comes in. To ensure the data you’re using is the best representation of that data in your organization, you need to know who to go to for the “source of truth”.
Each system that holds critical data should have SMEs identified from both business and technical groups for consultation. Your IT team may well have to adhere to certain processes when extracting data for provision to third-party vendors, and you will likely have to account for some security-oriented concerns. It’s good to have these things cataloged along with a point person who can submit the necessary requests, as well as some understanding of how long such a request might take.
These are some things businesses can (and should) undertake on their own, long before they even engage with a pricing solution vendor. In our experience, however, most do not. However, understanding these considerations before engaging vendors can go a long way in helping articulate the complexity that would come along with the implementation of a solution. A good understanding of these factors will create a much more predictable path to the success of the solution.
So, Who Should Be Involved in the Data Sourcing Process?
The most important people are often the ones left out of the equation: your operational users and data analysts. They are on the ground, know the specifics, understand where everything’s coming from, and what the data needs to look like. Your operational users are going to give you the best picture of what needs to be accounted for. They’re also privy to parts of the process that you may not be aware of.
Perhaps Fred in the corner cubicle has a spreadsheet that everyone runs their numbers through, but you have no idea! You’ll want all these things accounted for.
We’ve already discussed the importance of SMEs. They own certain data, fully understand it, and are able to articulate its meaning and usage of it. Requirements can’t be completed without a good understanding of the data drivers.
We also mentioned IT, who will be able to give you a general idea of what data context you’re looking at. You’ll need to understand what options are available for data extraction and what those timelines will look like – this is crucial as some data can take months to become available.
And, while not necessary, we see our customers have more success when they have business analysts on board to help bridge the gap between business and technology.
You want the teams involved to examine each discrete business process that will be supported in the pricing system and to catalog the data contexts and data elements that are needed to support those processes. This becomes your to-do list for the next step: data scoping.
Pinpointing What Data Each Business Process Requires (Data Scoping)
Data Scoping is the most critical phase of identification. It’s where you start digging into your business processes, exploring them from the top-down, examining what happens in each one and at each step. This might sound daunting, but a clear understanding of the data drivers in your processes is key. Without a firm grasp of the underlying process, the solution could veer off in a direction that doesn’t effectively solve the problem. You can certainly start a project before you scope your data, but scoping is a strong indicator of success.
You need to look at your inputs, work out when they go through a pricing process, and what information is needed to serve the logic required to deliver the correct outputs.
Each company’s inputs are going to be slightly different. Your main considerations should be:
Your Big 3 – your products, customers and sales transactions, including any attributes, hierarchies, and relationships that are unique or specific to the pricing process. Your Big 3 are the most critical pieces of data to pricing analytics and processes.
Data needed in pricing calculations (costs, external index data, competitive data, units of measure conversions, currency conversions, etc.)
Pricing rules data used to formulate prices (calculation equations, conditions for setting prices, etc.)
Sales structures, lines of business, and regional hierarchy data
User permissions, restrictions and roles
Other analytics data and calculation elements
Anything else used in your pricing processes
Of course, your key output is your price, or rather, your set of prices (because they will still be conditional depending on things like where you’re shipping from). These may be conditional or situationally specific, say by region or distribution channel. You may have other data that your pricing system needs to generate as well in order to keep other systems in sync.
The data scoping phase can be lengthy depending on the complexity of your processes. Make sure you give yourself enough time. But remember that this process will never be 100% complete. This is an exercise best done with the vendor and with some context on what the data structures look like, so it makes sense for it to be a solutioning activity.
We would like to recommend that our customers literally diagram out their processes, walk through them step by step, and then identify what data needs to come in at each one of those phases.
Considerations During Identification
Here are a few extra things you’ll want to keep in mind as you progress through this phase:
Depending on your vendor, you may want to master all your data in your pricing solution. We recommend that the pricing solution be the place that you master pricing process output data. If this is the case, work out what data will need to be mastered in your ERP or CRM.
Capture and understand stray data, like one-off data sets that are floating around but important to your pricing. They are usually manually maintained or brought in from outside teams. It’s important to ensure these are well understood and to determine whether they (and their extraction) can be automated.
Consider integration options for maintaining data in your new pricing system. Many organizations will employ middleware applications and supporting teams, but it’s worth knowing the standards they adhere to and their capability limits before you start.
Consider how often you need data updated on your various systems that support pricing. Are daily extracts sufficient? Or should data be sent or pulled in real time? This will impact implementation.
It is also important to understand your data in its extractable form and how it differs from the way it is presented to users (which is often enriched for greater clarity or to support quoting). This can be an obstacle from an identification standpoint as what you’re looking for technically doesn’t exist. But more importantly, if you’ll want your new pricing system to replicate the enrichment, the nature of the changes and any calculations involved will need to be well understood.
Find out if your vendor has any requirements with regards to data format or delivery.
The Goal of The Identification Phase
By the end of this process, you want to have:
Identified any data that would be used for analytics and pricing processes
Considered all inputs into each business process
Determined how data is used in each calculation
Identified and engaged data SMEs
Located or prepared documentation on data contexts
Identified data exchange processes
What technical implementation options are available?
How often does data need to be exchanged and with what systems?
What interfacing standards and practices must be observed?
Established data and technical SMEs and be prepared to discuss data concepts
Completed internal discussions to determine required data for each process
Identified all relevant data artifacts and sources
Engaged IT resources to provide initial data exports
Defined data export plan and reviewed integration protocols
Prepared initial extracts of data for project team review and configuration
Provided data documentation as a reference for solution provider project team
The Second Pillar of Data Readiness: Data Quality
What is Data Quality?
Data quality refers to how well suited your data is to do what you want it to do and do it well. Accuracy, validity, completeness, consistency, and uniqueness of your data all feed its quality. Most important is accuracy (Are the numbers and values correct?) and coverage (Do you have everything that you need to adequately address the scope of expectations and are all business cases covered?).
The Fundamentals of Data Quality
Before getting involved with a vendor, we advise that you at the very least identify where you might have data quality issues. These are places where you have duplicates, inaccuracies, ambiguities, inconsistencies, etc. This will help you get rid of things that you don’t need so you can focus on the data that actually helps your pricing processes.
While getting the fundamentals of this nailed down is an important step in the success of your project, don’t get too bogged down in it. It is not always essential for your data to be of the highest quality at the start.
Many of our customers bring low to moderate quality data into our solution to actually help them identify data quality gaps and outliers. In some circumstances, however, this could derail your entire implementation, so check with your vendor.
But where you’re not able to clean it up, you should at least be able to identify where those issues are. This is key. With this information, your vendor will either be able to reassure you with their clever workarounds or advise you to put the project on hold until a higher level of data quality can be achieved.
One of our clients had three different databases, each for a different line of business, that they needed to aggregate. They sent all data sets to us to manually upload. What they hadn’t told us was that the data across the databases was not normalized, so each used ID1 to represent a different product. This meant that when we imported the data, we only had a third of what was needed in the system. You must articulate underlying items and issues with your vendor.
Another client was trying to reuse an existing interface (created for another purpose) to send their transaction data to us. After much digging and tweaking, we realized they had sent us their tax and fees instead of actual transactions. But no one could tell this was happening because of the team that was providing the data from the interface didn’t actually know what it was or how it worked. All they knew was that it provided transaction data and thought they could re-use the interface.
The data couldn’t be validated. It eventually took them nine iterations and over 6 weeks to get right. Six weeks might not seem like a lot for an enterprise-level software implementation but that’s time that you could already be using the software. Interestingly enough, those situations happen often.
So, the two steps to achieving data quality ahead of implementation are:
figuring out if you have any known data quality issues
having a plan to mitigate them
Third Pillar of Data Readiness: Data Availability
Data availability is about the timeliness and accessibility of your data. Are you able to get to the data you need in order to do what you want to do in a timely fashion? Data availability is (or rather, the lack of it is) actually one of the things we often see severely delaying or shutting a project down before it even starts.
It is critical that you be in sync with your IT teams, and their schedules as well as to understand your options for extraction. It’s also important to identify which departments and teams you’ll need to interact with, whether there are middleware teams to account for, and what the various timelines are for the teams and each system you’re extracting from.
In an implementation, there are typically three phases to using your data.
Sample data – you suggest a representative sample set of data that is enough to design around and provide context for your requirements. It should give a good idea of what your structure should look like and cover the scope of concepts you’re working with (e.g. a subset of tables or 10 – 20% of rows from all tables).
Full manual data – at this stage, your vendor will perform a “manual full extract”, pulling all data in your tables that you want to have in your pricing system. (This is where data coverage is so important as lack of functional coverage can lead to late-stage defects or reworkings.) Testing is dependent on your data, so you must be ready for this in terms of quality and availability. You will need to be prepared to support the extraction from your systems to make the data available to the vendor. In some cases, manual extraction can be skipped and you can proceed directly to full automation – provided it fits the proper time schedule.
Fully automated data – finally, the vendor will implement and facilitate interfacing with your systems so that data automatically refreshes on the agreed basis (hourly, daily, weekly). At this stage, everything will look as it would in production and you will be testing all your business cases on your new system.
Being able to access the right data in a timely manner during your project is crucial to a successful implementation. Delays in receiving data sets can significantly stall progress.
We had one client initiate a big project with us before consulting with their IT team about their timelines. Everyone was ready to go when we suddenly learned that they couldn’t deliver the data for four to five months! Needless to say, this created a significant impact on the project’s timeline and budget.
When To Involve Your Pricing Solution Vendor
There are a lot of things that can be done prior to the start of a pricing software project that don’t require input from the vendor. Getting started early on those activities can be extremely beneficial to the project’s progress.
We recommend that you start working on business processes, data sourcing and data scoping during vendor selection. You should be around 75% through your scoping exercise and have a good sense of where your data quality issues are and what your data availability timelines look like by the time you engage.
At Pricefx, we actually like to do a lot of data work with our customers and to make their build an interactive process.
Maximizing Chances of Project Success
Remember, your project’s success is in your hands. To get your project implementation delivered on time and on budget:
Make sure you fully understand all your business processes and the data that support them.
Know where that data sits, who owns it, what it means, and how long it will take to access.
Ensure you understood and have cataloged every data input (Big 3, etc.) and the logic applied (pricing calculations, etc.) to deliver an accurate output (set of prices).
Address data quality and identify where issues exist ready for communication with your vendor.
Get aligned with IT and have a clear idea of data accessibility timelines and protocols.
Initial data deliveries are critical to maintaining project timeline, so have the right data ready for your vendor at the right time.
The earlier data can be provided, the better. Ideally, you would want at least some sample data available for project kick-off.
“How long does it all take?”, you ask.
As with most projects, the answer to this question is: it depends. Anywhere from a couple of weeks to six months. Most of that time, you’ll probably have started working with your pricing software vendor of choice. Data readiness can be done in tandem with other implementation processes but it’s imperative that you with at least the basics, even before you’ve chosen your vendor.
Things that impact your timeline will be:
The complexity of your business cases
Data security and audit protocols
How many of your processes are informal, manual or offline
How well your data is understood in the organization
Your choice of vendor
Time To Get Your Data Ready
This article has been designed to educate and inspire, not scare you away from the entire process. The main takeaway is: do your homework! None of this is outside of your capabilities, it just requires planning.
The better prepared you are ahead of your implementation, the smoother it will go. You should know your business processes inside out, understand your data and how it supports them, be aligned with the teams who own the data, have a clear view of its quality and where issues lie, and have a good understanding of accessibility timelines.
Not doing your homework could have a negative impact that cascades through your project, creating extra work and delays, even shutting it down completely.
But if you go through each of the steps laid out in this article, you should be well on your way to a successful pricing software implementation.
Krishna Sudhakar is the Director of Partner Advisory Services at Pricefx, based in Chicago. He has over 20 years of experience in software development and delivery with a focus on designing technology solutions to solve complex business problems. Before pricing, Krishna spent time working with systems in the software, healthcare, defense and financial industries. When not helping businesses solve pricing problems, Krishna spends time traveling, trying new restaurants and getting intentionally electrically shocked running obstacle course races.