The Lean methodology grew out of the Toyota Production System that was developed in the 50s and 60s.  The original focus was on optimizing factories, but it is applicable to many different industries.  Lean is laser focused on providing customer value.  Anything that does not contribute towards customer value is waste and should be removed.  The data space has two major differences from traditional Lean practitioners.  The first is that our customers are usually internal to the company.  The second is that building software is not building the same product over and over like a factory.  However, If we abstract a bit we find that these differences do not matter much.

According to the lean methodology there are 8 types of waste with the nice acronym “DOWNTIME” to remember them:

·      Defects

·      Overproduction

·      Waiting

·      Not utilizing talent

·      Transportation

·      Inventory excess

·      Motion waste

·      Excess processing

Some of these are specific to manufacturing, but many of these ideas can apply to data / analytics projects in any industry.  Years of experience using lean methodologies have allowed me to discover a few of the reasons these types of data waste so often show up, here are a few common sources of waste that I’ve encountered.

Bugs

Bugs become increasingly expensive to fix the later they are found, so it is crucial to establish thorough security procedures as soon as possible to. Poor testing or quality controls can lead to extended debugging sessions that see hours of time and money lost.

Underutilized Hardware

Hardware that you bought, but are not using is waste.  For example, In Snowflake you are charged for every second a virtual warehouse (compute) is running.  If no queries are running, this becomes wasted money. (Yes, caching complicates this, but you get the idea.)

Underutilized Software

Similarly, companies purchase software and either do not use it, or do not use it to its full capabilities. An example would be a company paying for Power BI premium but is not using any of the features that the Premium platform provides.

Unused reports/data

Reports and tables are only valuable if they are used. Data that isn’t being used or is not needed is a big source of waste.  This is one of the worst forms of waste.  As you pay for hardware, software, and the labor to build them. Reports and data are only valuable if they are used.  

Searching

How difficult is it to find the reports/analytics a user needs?  McKinsey says employees can waste up to 9 hours a week just looking for data they need to do their jobs.

Waiting on reports

Waiting for reports to run, waiting for visuals to update, etc. Small pauses can quickly add up to lots of lost productivity over time.

Duplicated effort

Example: Employee A creates a sales report.  Employee B doesn’t know about it, so they spend a week creating their own sales report. Happens often when departments are siloed and do not communicate.

Excess Administration

Meetings that could have been an email, extensive paperwork, unnecessary signoffs, excessively long stand ups.  The business world is rife with wasteful contributions.

Unused talent

Usually appears in the form of talented (expensive)employees doing menial tasks.  Data engineers doing manual data entry, analysts wasting time copy/pasting values into Excel, or subject matter experts who make sub-optimal decisions because they don’t have the data they need.

Conclusion

As we delve deeper into the age of data-driven decision-making, understanding and actively eliminating the Eight Deadly Lean Wastes of Data is pivotal for businesses to flourish. Embracing Lean methodologies in data practices not only refines operations but maximizes the ROI on both human and technical resources. The dynamic and ever-evolving nature of data strategy means that this isn't a "set it and forget it" approach. Continuous monitoring, assessment, and refinement are essential.

To aid businesses on this journey, it is wise to partner with experts in the realm of data strategy, MDM, and data governance. These professionals bring the right blend of expertise, tools, and insights to transform data waste into invaluable assets. By leveraging these services, companies can ensure they remain competitive, agile, and most importantly, equipped to deliver unmatched value to their internal customers and stakeholders. This is the new era of data optimization, and every company needs a guiding hand to navigate its complexities.

‍

‍