Death by a Thousand Dashboards
Dashboards are the accepted standard for displaying data and metrics. Companies that develop dashboarding solutions are part of a multi-billion dollar industry, with giants like Tableau, Looker, Power BI, Zoho, and more.
Although dashboards are universally recognized as a powerful way to access data and inform business decisions, developing and maintaining them presents some challenges. In this article we describe the phases that a growing business typically goes through when adopting a dashboarding tool, and we reflect on the issues that arise within each phase.
Phase 1: adoption
When a business starts to grow, so does the complexity of the numbers that need to be tracked. While initially it’s easy to monitor simple metrics like number of clients and transactions in spreadsheets, over time there is a need to keep track of more complex aspects of the business, for example:
Clients metrics: new clients, returning clients, clients demographics.
Funnel metrics: leads, conversions, time to conversion, returning clients.
Financial metric: revenue, profit, gross bookings, costs.
Online products: usage, impressions, clicks, time, experiments.
Customer support: interactions, resolutions, returns, credits.
When multiple teams start to manage different parts of the business, and each area has a need to monitor metrics, react to signals and make decisions, there are important analytics decisions to be made: how to standardize important metrics across the company? What tools should be used to display these metrics?
Dashboarding tools are typically adopted to provide a user-friendly interface to access data. Basic functionalities include:
Ability to read data directly from multiple databases.
Flexibility on visualization options, like line charts, bar charts, pie charts, and more.
Dynamic Filters to interact with dashboards and dissect the data.
Option to periodically refresh the displayed data.
Dashboards can be set up by Data Analysts, Data Scientists, or any other team members who are familiar with the data and are interested in tracking the products and services that they are working on.
When adopting a new dashboarding system, initial challenges include:
Transitioning from an existing tool (for example spreadsheets) to a new dashboarding system requires some work, both technical and cultural.
Estimating the number of users to negotiate pricing for the tool is not straightforward. Pricing typically depends on the number of user licenses and active users, which will also determine the computing capacity.
Phase 2: popularity
The need to access data and visualize key metrics grows proportionally with the business. More teams are interested in tracking their products, while experiments and trials generate more and more data. Interdependencies across teams require more visibility into many metrics. More teams need to build and maintain dashboards.
At this stage the team responsible for maintaining the dashboarding system will face new challenges and will need to pay attention to several dimensions:
Monitoring the underlying pipelines that generate the data for the dashboards.
Monitoring the utilization of computing resources of dashboards.
Establishing a queuing system for queries, so that the system is not put under excessive pressure and all queries are guaranteed to be eventually executed.
Establishing a process to support users who reach out for help.
Providing guidelines on how to build efficient dashboards.
At the same time, users will run into a few problems related to the use and contents of dashboards:
Metrics may not be standardized across teams. This can result in 2 or more dashboards showing a different version of the same metric.
Dashboards and queries may start to be used in place of business processes that haven’t been developed yet. For example, a dashboard could track the status of clients and leads, until a proper CRM process gets adopted. This solution is suboptimal, as business processes require a level of reliability and responsiveness that dashboards might not guarantee.
Phase 3: overload
A mature business can track thousands of metrics across multiple teams. Several roles create and maintain dashboards, and maintaining the system can become a full-time job for an engineering team.
In this phase, managing existing dashboards becomes as important as creating new ones. In particular:
Some dashboards may not be used anymore, but they still refresh periodically, wasting computing resources.
A few dashboards may contain duplicated data, making it difficult for users to know where to look for information, and how to agree on a source of truth.
Some dashboards rely on inefficient queries and contain too many visualizations, consuming significant computational resources.
Nonetheless, dashboards have proven to be an effective way to visualize data and surface information. For this reason, they might also start to be used for research purposes, in place of more appropriate tools (e.g. scripts). Analysts will spend a lot of time generating dashboards for ad-hoc requests, fishing for signals in the data. Since these dashboards will not be used to monitor metrics over time, they might be viewed by users only once or twice, before being deprecated with a low return on investment.
Solutions
Many of the challenges presented above can be mitigated but not completely eliminated, as some inefficiencies are part of the “cost of doing business”.
The team(s) managing dashboarding systems within a company can establish a few processes to facilitate the efficient creation and management of dashboards:
Monitoring metadata on:
The use of dashboards: the only purpose of a dashboard is to be viewed by users. The owner of a dashboard should be able to see how many people viewed and interacted with their dashboard, so that they can maintain it or deprecate it accordingly.
Computational cost: some queries are more expensive than others, and some dashboards with too many visualizations have higher computational costs. Insights into this metadata can help users create more efficient dashboards.
Users: dashboarding systems typically have several tiers of licensed users, from “admin” to “viewer”. It is important to monitor the activity of licensed users to manage costs and transfer licenses to users that need them the most.
Written documentation is an effective way to make users self-sufficient, show how to request licenses, prevent common mistakes, and disseminate knowledge and best practices.
Office hours can also be helpful to onboard new users into the dashboarding system, show demos, and answer technical questions.
One way to mitigate issues is to align the incentives of maintaining good practices with the teams that will incur in the cost of not-so-well maintained dashboards:
Each team should maintain a unique definition of the key business metrics representing their work. However, it is hard to uniquely define all metrics across various teams within the same company.
Every team should do a periodic review of dashboards and their usage, and actively deprecate unused dashboards.
Dashboards that are being used as a short-term solution for an operational process should be owned by the team in charge of creating the long-term solution for that process. For example, dashboards used to track the status of clients and leads should be maintained by the sales team in charge of evaluating and adopting a proper CRM tool.
Lastly, one of the most important aspects of adopting and managing dashboards is a cultural one. A data-driven organization has metrics in place to monitor the health of the business, and to make business decisions in reaction to trends and anomalies that are detected in the data. However, this goal is often used to justify the request to build a dashboard with hundreds of metrics for research purposes. This occupies the bandwidth of Analysts and Data Scientists, who could be spending their time on more efficient tasks, and often does not generate the desired results: an overcrowded dashboard is hard to read and interpret. It can lead to contrasting signals and, even worse, it will be abandoned by users. When building dashboards at any level (project, team, quarter, company, etc.), an effort should be made to maintain a handful of key metrics that can be easily monitored and interpreted. When other investigations in the data are needed, there is always an opportunity to do additional ad-hoc analyses without creating dashboards for them.
Need help analyzing your data and setting up dashboards? Data Captains can help! Get in touch with us at info@datacaptains.com or schedule a free exploratory call.