DanMaycock.com

Analytics, Strategy, and Agriculture

Tag: data science

Don’t Go on a Date with Data, Marry It

There are a lot of companies out there today sitting on several types of data, with various levels of integration. For most, it’s a process involving multiple individuals to extract, transform, and develop that information into a useful dashboard or decision-support system given the company has legacy infrastructure and several different third party vendor platforms all producing useful information but it’s in own specific format.

The big challenge then, that companies face today, is how to tie all of this information together in a way that provides meaning but more importantly, provide it in a way that can be near real time with the ability to tie different types of data from all over the enterprise together.

Yes, you know how much you lost and earned in a given period of time, but can you determine why with the ability to stitch together different data sources? Can you do your own regression analysis, or perform correlation research to determine what trends might be causing a rise or decline in monthly revenues? Or do you have to hire it out, and wait several weeks, while analysts inside or outside your company work to produce the answer to that one simple question?

There are lots of people willing to charge you for the ability to do this month to month, on an ongoing basis, but if you’re content with having systems kept in disarray while you’re driving up the cost of labor to mine it every time you have a question about your numbers, then you’re simply dating your data.

You’re content leaving data at arm’s length, getting to know it to a point, but you’re not spending the time and effort it takes to really get to know the data inside your company and make the investment to make data a key part of your professional life. Marriage means accepting the ugly truth about someone, and acknowledging to them the same sometimes ugly truths as well, to build a close relationship.

Marrying data means you’re accepting the truth that legacy infrastructure, silo’ed data sets, and weeks spent building a single dashboard isn’t working for you and you’re ready to spend the time and effort it takes to bring data closer to the core of your business. It means making the investment to build an agile analytics platform that allows for ad-hoc analysis, and spending the time it takes to get your leadership team on board with understanding what a regression is and what level they feel comfortable being trained to help drive insights without an army of external analysts.

Marrying data also means accepting the things you can’t change about it, and learning to live with limitations as they are. It doesn’t make sense to send everyone to school to learn data science, but it does make sense to get up to speed on what it means and getting people trained in the vocabulary of data so that those highly trained resources can build what is needed and make sure everything you’re investing in has a clear return at the other end.

Making the ultimate commitment to data, vs having an off again on again relationship, means you’re wiling to spend the time it takes to make the upfront investment to clean up the silos and tie together the systems keeping your really valuable insights locked up. It means knowing that it’ll take time to see the value, but it’ll be worth the investment, vs continuing to grow your OpEx budget on consultants and FTEs working with what’s there now, and spending time sorting and tying each system together for one-off requests.

More importantly, marrying data means you’ve accepted that the key to a happy company is a happy data warehouse, and that Innovation in it’s most meaningful way means you’re able to draw out from your past what solutions and ideas might help fuel your future. By spending the time and effort to build meaningful data interconnectivity, along with the systems necessary to analyze and understand that data, you’ll be able to see what trends are coming your way and how you can be proactive to meet the challenges in an ever changing industry environment.

You’ll reduce the risk of being disruptive, you’ll be armed with answers before the questions get asked, and you’ll be able to walk hand in hand with your data into the sunset while every division within your company gets insights they need to better track what’s working and what isn’t along with driving new revenue streams to your customers.

A happy marriage for some seems like a fairy tale, and it’s not going to solve every problem you encounter. However, if you’re willing to put in the time it takes and accept that there’s things you could be doing to pay better attention to your data along with spending the time to care of it, your data will produce insights and become more open to analysis as a result which will always work better than keeping it in a chaotic state and spending time doing one-off reports.

So consider what marrying your company data looks like for you, and build a plan and a roadmap to make data more meaningful and acceptable for your company. I can guarantee you, your competitors are probably already doing the same.

Dan Maycock is the author of “Building The Expo”, which shares best practices on leveraging #Innovation in meaningful ways and saving the concept from it’s overused but underutilized past. The book has first hand stories, and best practices from Dan’s years of experience working with Fortune 1000 companies dealing with emerging technology adoption in an increasingly dynamic business environment. You can purchase the book atAmazon.com or learn more about Dan at http://www.transform.digital

Drive Revenue Growth Without Driving Off the Road

Data – it’s something everyone is hearing about these days, whether it’s IBM stating how twitter can help you build better products to Google talking about self driving cars saving the environment through the power of data-driven route optimization.

That’s all well and good, but when you’re sitting at your desk looking at the list of things you have to accomplish today, and bring about new ways to grow revenues or build efficiencies into your business, where does one even begin?

No, you don’t always need a small army of data ninjas (though that can sometimes help) nor do you need a lot of high priced tools and solutions to help find buried gold in your troves of silo’ed business data.

To really drive to change, and begin leveraging your data to drive revenue it starts with the following steps

A) Start with the right hypothesis

If you don’t know what questions you’re trying to ask, then it’ll be very hard to find what you’re looking for that’ll help you achieve your business objectives. Asking questions like “where should I sell my goods” or “what products should I build” would require very expensive, intelligent systems capable of translating english into problems that computers could try and solve along with the tribal knowledge and understanding of the industry you’re in. Those systems exist, but man are they expensive along with the IBM consultants you’d need to hire to go between you and the outcomes.

Instead, focus on a specific question based on the types of data you know your business has. Start with things you think you know about your business, like what your key demographic is or where you source your raw materials from. Then go deeper and ask why those folks are your core demographics, or why you went to that one country for zinc. Good data mining starts with understanding the problem space better, and exploring things at a granular enough level that you can understand where intuition, guessing, or laziness came into place vs finding the right outcomes at the right level. You tell me you sell products in Seattle to women 44-55 years of age, I’ll ask what neighborhoods is that least or most true and if that answer is biasing you from growing your customer base because you’re focusing on metropolitan areas from a broad national study you paid a firm to do 5 years ago.

Sure, the same answer might be true, but having those metrics and answers in place means you’ll be able to see the shift and know when those answers are no longer the case, more importantly it’ll cause you to ask why the answers are they way they are and those levers or foundational factors will become more obvious and allow you to get granular enough to spot the outliers biasing your answers which get lost in the high level aggregated dashboards most execs use today.

B) Understand where your data lives

If you live and die by your profit and loss statements, or your quarter earnings reports, chances are that there is a complicated network of data analysts and administrators that compile all that information together to come out with a single answer. If you are getting your core business metrics from the same group you’re measuring against outcomes, be careful about unintentionally biased data that leans on rounding up vs rounding down and know where that data is coming from.

Too much data exists today, and decisions get made by people along the way on how that data is compiled and delivered, so build a culture of transparency and make sure you don’t have data points stacked on top of data points where errors can slowly creep in.

Aside from transparency, agile systems built the right way means you can do ad hoc reporting and build your own metrics with the ability to drill up and down without having to wait weeks for someone to compile a report on only the question you asked. Too much legacy infrastructure, and data scattered across the company along with an over reliance on key information being locked away in spreadsheets means a mess for really getting down to the bottom of things.

C) Figure out how the data is (or is not) related

If you’d like to see how twitter is affecting your supply chain, spend a little time thinking about how the two inter-relate. Twitter is going to be something tagged by date and location at a high level, but is a bunch of key words and a user name so prepare to invest in interpretive systems that aggregate and analyze or figure out ways twitter data might tie to a critical business system. There’s lots of ways to get at the answer, but high quality dashboards with pretty graphics may be just interesting and not at all useful if you don’t have the right data behind that tool giving you meaningful answers. It’s not about big data, it’s about meaningful data.

D) Ask an expert (whether or not you intent to hire one)

Data, like engineering or medicine, is a very complicated space that gets increasingly complicated by the day. Rather than becoming a data scientist yourself, find someone you know and trust that works with data and use them as a sounding board to run your ideas and suggestions by. It’s not that you may have a bad idea on how to leverage data to achieve business insights, but having it structured in the right way while learning what’s possible and what isn’t without a lot of investment is important to finding meaningful, bite sized ways to leverage data without breaking the budget and overspending for fancy whiz bang data systems.

E) Start small, grow big, track and measure along the way

The most important thing is to not bite off a big problem, like how do you end world hunger, but something small you know you could get good insights around relatively easily, such as where you spent what and how that goes against what you make with the ability to drill down into where that changes. If you typically get profit and loss statements saying you’re profitable in washington state, understand what city that is and is not the case and why one would be different than the other. Sometimes it’s just getting more granular data to what you already receive that can have the biggest insights into your day to day business.

At the end of the day, data isn’t a silver bullet, but it can make a difference in a big way when approached the right way. Start small, build a meaningful hypothesis, and strap in for the revenue growth that will follow.

Daniel Maycock is the Director of Strategy and Analytics at Transform, a data services
company. Our mission is to help drive impactful outcomes with data for our clients. We do this by providing tailored solutions that help people get tangible applications from their information.

His new book, Building The Expo, was published in January, 2015.

4 Ways to Effectively Use Data In Your Job

With all the excitement around how companies are using data today, it’s hard for anyone outside of a job specifically dealing with data to know how to effectively use it for their day to day work. Yet, there isn’t a single career that isn’t impacted by the use and understanding of data, and the more effective someone becomes at harnessing and understanding data mining, the more they can impact the things that impact their professional ecosystem.

From impacting your online brand, to better tracking variables you deal with around a given task at work, knowing how to leverage data can make a big difference in any number of careers.

1. Start with a question

Before diving into a number of articles or tools regarding data, start with the question of what you’re trying to answer. It sounds basic, but you’d be surprised how often I’ve worked with people that have said data is the answer without first having the question. Figure out what are the most pressing business problems you, your boss, or your company are facing and see how data might help provide insights to answering those important need to knows.

2. Start with a small amount of data, build from there

It doesn’t take petabytes of data to answer questions, sometimes it can be a relatively small set of data to answer big questions. With all the hype around big data, sometimes it’s hard to realize that with only 100 or so records, and a pivot chart, you can get to important answers that are far more useful than what a million records could show, depending on the type of data and the question you’re looking to solve.

3. Leverage third party data that’s free

There’s a TON of data out there that’s completely free, and useful to use. US Census is a great place to start, and there are a number of sites, such as Google’s public data directory that’ll let you explore it. Furthermore, you can download the data for free and combine it with your own internal data to add greater context for things like taking your company’s store sales by zip and seeing how demographic trends within those zip codes may impact certain buying habits.

4. Learn about Data Mining 

The key to making data useful is by learning methods that allow you to tap into data, and find useful data points that can help solve the business problems you’re looking to tackle. Data mining is the practice that helps you start to uncover trends and patterns in data, and is a great discipline to begin with, whether it’s using Excel and a little bit of data or tapping into RapidMiner and starting to dive into Hadoop, Data mining spans the gambit on complexity and data quantities. Remember the first three points to keep the right context and not go overboard too soon though, and you’ll be in good shape.

Regardless of your career, there is a way data can no doubt help you professionally and impress your co workers and higher ups in the process. Start with the fundamentals, help answer important questions, and simply build from there and you’ll be a bonafide data analyst before you know it.

Why Everyone Should be a Data Miner

In thinking about the topic of data mining, a lot of different types of roles pop up in people’s minds. From data scientists typing away in giant data centers, to DBAs sitting in cubicles processing large amounts of corporate data, to an analyst building a spreadsheet for an annual report contribution.

Maybe it’s something far more physical, bringing up images of pick axes and hard hats and a big block of data (however that’s visualized, probably with 1’s and 0’s – all matrix like). Regardless of the image that comes to mind, it’s probably hard to fathom every business professional in some form or another becoming adept at data mining, and considering it a critical competency to keep in their professional toolbox in the years to come. Yet, when we explore the topic, we can easily see how data mining could become one of the preeminent skills that set folks apart in an era where it’s harder and harder to stand out from an increasingly noisy and competitive work climate. Lets start by looking at the six attributes that make up data mining (as defined by Wikipedia)

  • Anomaly detection (Outlier/change/deviation detection) – The identification of unusual data records, that might be interesting or data errors that require further investigation.
  • Association rule learning (Dependency modeling) – Searches for relationships between variables. This is sometimes referred to as market basket analysis.
  • Clustering – is the task of discovering groups and structures in the data that are in some way or another “similar”, without using known structures in the data.
  • Classification – is the task of generalizing known structure to apply to new data. For example, an e-mail program might attempt to classify an e-mail as “legitimate” or as “spam”.
  • Regression – attempts to find a function which models the data with the least error.
  • Summarization – providing a more compact representation of the data set, including visualization and report generation.

Though the definitions seem somewhat dense, think about how you’d be able to take any job – from being able to use regression analysis to construct a real estate data model to improve pricing predictions, to using summarization to build a better financial report for your senior leaders to interpret how great of a quarter you had.

Though some methods of data mining are harder than others, and you can quickly get in way over your skis without proper learning, knowing how to sift through data, and pull out the useful stuff, will give you a greater sense of the world you work in by understanding the data that matters and it’s so easy these days to learn data mining techniques online!

Just typing in “data mining classes online” produces hundreds of leads, from Coursera to MIT open courseware. Though some options go into areas like Data Science, which is much deeper level analysis, it all starts with understanding data and how best to derive meaning from it – regardless of how deep into the weeds you want to go.

This in turn gives you a big foot up against your competitors, who are largely relying on other services / people to hand them processed data and conclusions to do something with. Going from a commodity to a distinct competitive advantage means going in a direction others aren’t, and just having a nicely worded dictionary isn’t enough these days – you need to be able to turn that dictionary into a novel, and tell a story with the data that will reveal things about your business or your industry that’ll drive better decisions through unique insights.

© 2018 DanMaycock.com

Theme by Anders NorenUp ↑