Why Off-the-Shelf Dashboards Fail We have walked into enough companies to know the pattern. There is a TV on the wall showing a dashboard nobody looks at. It was bought from a vendor or built from a template. It has twelve charts. Nobody can explain what three of them mean. Off-the-shelf dashboards fail because they are built for an average business that does not exist. Your finance team defines revenue differently than your sales team. Your operations team cares about shipment exceptions, not totals. The generic dashboard shows totals. The result is predictable. People open it once, find it useless, and go back to their spreadsheets. The dashboard becomes wallpaper. Here is what generic dashboards get wrong: They show metrics someone else defined, not the ones your team argued about for three months They assume a standard data model, but every company has quirks in how data is stored and labeled They prioritize looking impressive over being useful They cannot adapt when your business process changes A dashboard is not a trophy. It is a tool. If it does not help someone make a decision or do their job faster, it has failed. The Dashboard Trap: Building for the Wrong Audience The most common mistake we see is this: a dashboard is built for executives, approved by executives, and then handed to operators who never wanted it. Executives want the high-level view. Total revenue, monthly trends, top customers. That is fine for a Monday morning glance. But the person actually doing the work needs different information. They need to know which shipment is stuck at customs, which invoice does not match its purchase order, which employee has not logged hours for three days. The dashboard trap happens when nobody talks to the people who will use the dashboard daily. The executive sponsor is happy because the numbers look good in the demo. Three weeks later, usage drops to zero. We learned this the hard way on an early project. We built a beautiful executive dashboard with fifteen charts. The CEO loved it. The operations team never opened it. When we asked why, they said: “None of those numbers help me do my job.” We rebuilt it from their perspective. We asked what questions they had at 9am every morning. We built the dashboard around those questions. Usage went from zero to daily. The lesson: build for the person who opens it every day, not the person who approved the budget. How to Design Dashboards People Actually Use Start with the question, not the data. This is the single most important rule. Most dashboard projects start with: “We have all this data, let’s visualize it.” That is backwards. You end up with charts that exist because the data exists, not because anyone needs them. Here is the process we follow: Talk to the people who will use the dashboard daily. Not their manager. Them. Ask what decisions they make in a day and what information they need to make those decisions. Write down the specific questions the dashboard must answer. Rank the questions by importance. Build only enough charts to answer those questions. A good dashboard answers questions. A bad one raises them. Other design principles we follow: One screen, no scrolling for the most important information The top-left chart gets the most attention, so put the most urgent metric there Use color for status (green, yellow, red), not decoration Remove any chart that nobody has looked at in the last two weeks Show the trend, not just the current value, so people can see direction Metric Definition: Why “Revenue” Means Five Different Things Before you build a single chart, you need to define your metrics. This is where most projects stall. Ask your finance team what revenue is. They will say: recognized revenue, booked in the general ledger, excluding taxes. Ask your sales team. They will say: total contract value signed this month, including future commitments. Ask your operations team. They will say: invoiced amount for goods shipped this month. Three teams, three definitions, one word. If you build a dashboard without resolving this, you will have three people looking at the same chart and arguing about whether the number is right. We run a metric definition workshop before building any dashboard. Here is what it covers: The exact formula for each metric The source system and table What is included and excluded Who owns the definition How often it is refreshed This is not glamorous work. It is the difference between a dashboard people trust and one they argue about. Role-Based Dashboards: Different Views for Different People A single dashboard cannot serve everyone. Role-based dashboards solve this. Executive Dashboards Executives need: trends, comparisons to targets, exceptions that need attention. They look at the dashboard weekly or monthly. They want to know if something is off-track, not the details of why. Charts that work: trend lines, KPI scorecards, variance against target, top-five exceptions. Manager Dashboards Managers need: team performance, bottlenecks, items requiring action. They look daily. They need to spot problems early and assign work. Charts that work: team workload, cycle times, aging reports, exception lists. Operator Dashboards Operators need: their specific work queue, what to do next, what is blocking them. They look hourly or constantly. They need actionable items, not analysis. Charts that work: task lists, status boards, alerts, item-level detail with drill-down. The same underlying data feeds all three. The difference is what each role needs to see and do with it. Real-Time vs Batch: When You Need What Not everything needs to be real-time. This is a common and expensive misconception. Real-time data is necessary when: A delay causes a problem (a shipment stuck at customs needs attention now, not tomorrow) An operator takes action based on current state (a dispatch team routing trucks) You are monitoring for exceptions that escalate quickly (fraud detection, SLA breaches) Daily or batch data is fine when: The decision cycle is daily or slower (monthly revenue review, weekly team
Why Most Integration Projects Fail and How to Get It Right
The Failure Statistics and Why They Happen Industry studies have shown that a majority of IT projects fail to meet their objectives. Integration projects fail more often than most. A widely cited McKinsey finding on large IT transformations found that 70% fall short of their goals. Integration work is harder because it touches multiple systems, multiple teams, and data that nobody fully trusts. We have seen integration projects fail in predictable ways. The technology is rarely the problem. The failures come from unclear requirements, bad data, untested assumptions, and people who were never brought into the process. This post covers the seven reasons we see most often, and what to do differently. Reason 1: Unclear Requirements and Scope Creep Nobody wrote down what “done” means. This is the most common failure. The project starts with a vague goal like “integrate our sales and inventory systems.” Everyone agrees. Nobody defines what that means in practice. The sales team thinks it means real-time inventory checks during order entry. The inventory team thinks it means a nightly sync of stock levels. The IT team thinks it means a shared database. Three months in, everyone discovers they were building different things. Scope creep follows naturally. Since nobody defined the boundary, every new request becomes “part of the integration.” The project grows, the timeline slips, and the budget breaks. What to do instead Write a one-page scope document before any work starts. It should answer: What systems are being integrated and why What data flows between them and in which direction What is explicitly out of scope What “done” looks like, with specific acceptance criteria Who signs off on completion If a new request comes in, it goes through change control. It either replaces something in the current scope or it becomes a phase two item. Scope is a contract, not a suggestion. Reason 2: Messy Data Integrating bad data just moves the problem faster. We worked on a project connecting a CRM to an ERP. The CRM had 40,000 customer records. Roughly 8,000 were duplicates. Another 3,000 had incomplete addresses. The plan was to sync these records to the ERP, which would have propagated every error. Data quality problems do not fix themselves during integration. They get worse because now the bad data exists in two systems instead of one. Common data problems we encounter Duplicate records with slightly different names (ABC Ltd, ABC Limited, A.B.C. Ltd) Missing required fields that the target system needs Inconsistent formats (dates as DD/MM/YYYY in one system, MM/DD/YYYY in another) Orphaned records that reference data that no longer exists Free-text fields where people typed whatever they wanted What to do instead Run a data audit before integration begins. Profile the source data to understand its actual state. Fix data quality issues at the source, not in the integration layer. A data cleaning step in the middle is a bandage, not a solution. Build validation rules into the integration so bad data is caught and flagged, not silently passed through. Reason 3: Undocumented Legacy Processes The person who built the old system left five years ago. The system runs on tribal knowledge. We inherited an integration project where the source system was a custom-built application from 2012. The original developer had left the company. There was no documentation. The only person who understood it was an accountant who had learned its quirks over eight years. She knew, for example, that the system recalculated inventory every night at 2am, and that any record changed after that would not reflect until the following night. None of this was written anywhere. Legacy systems are full of this kind of invisible logic. If you integrate without understanding it, you will build something that works in testing and breaks in production. What to do instead Document the existing process before designing the integration. Interview the people who use the system daily. Map the data flow, including the manual steps and workarounds. Identify the hidden rules, the batch jobs, and the exceptions. This documentation becomes the foundation for the integration design. Skipping it means you are integrating against assumptions, not facts. Reason 4: Weak Testing No parallel runs, no reconciliation, no real user testing. Testing is where integration projects die quietly. The team runs a few sample records through the integration, sees them come out the other side, and declares success. Then it goes live and the edge cases appear. What proper testing looks like We insist on three levels of testing: Unit testing: Each component of the integration works on its own. Field mappings, transformations, error handling. End-to-end testing with real data: Pull a representative sample from production, run it through the full integration, and compare input to output. Parallel runs: Run the old process and the new integration side by side for a defined period. Compare the results. Any difference is a bug. Parallel runs are the most valuable and most skipped step. They catch the problems that unit testing cannot because they expose the integration to real-world data variety and volume. Reconciliation and real user testing For financial integrations, reconciliation is non-negotiable. If you move financial data between systems, the totals must match. We build automated reconciliation checks that compare source and target after every sync. If they do not match, the integration stops and alerts someone. Let the people who will use the integrated system test it before it goes live. Not the project team. The actual operators. They will find the problems you cannot because they know how the work really happens. Reason 5: No Change Management The new system works but nobody uses it because they were not involved. We delivered an integration that connected a company’s order management system to their warehouse system. Orders would flow automatically. The warehouse team did not need to re-enter anything. It worked perfectly in testing. Three months later, the warehouse team was still entering orders manually. When we asked why, the answer was: “We do not trust it.” They had never
Modernizing Legacy Systems in the Sri Lankan Import Export Sector
Custom Integration Case Study: Legacy Migration Lessons A large Sri Lankan import-export company came to us with a problem that should sound familiar. They had a legacy system holding their core operational data, and it could not talk to anything else. Bookkeeping lived in one tool. Employee management lived in another. The legacy system sat in the middle, isolated, with staff manually copying numbers between systems all day.This post walks through what we actually did, the parts that worked, the parts that hurt, and what we would do differently if we started over. The Client and the Problem The company operates across three sectors: metal, textile, and chemical imports and exports. That means shipments, customs paperwork, supplier payments, employee time tracking, and financial reconciliation all running in parallel, every day. Their legacy system had been in place for years. It stored shipment records, inventory movements, and supplier data. On its own, it functioned. The problem was everything around it. The bookkeeping system had no connection to operational data. Finance staff retyped invoice totals and shipment costs by hand. The employee management system had no link to operational work. Productivity reporting meant spreadsheets emailed back and forth. Manual entry created errors. Wrong decimal places, duplicated rows, missing shipments. Each error took hours to trace and fix. Reporting was reactive. By the time someone built a monthly summary in Excel, the data was already two weeks old. The company did not need a flashy new product. They needed their systems to share data, on schedule, with checks in place. That is the work we took on. Our Approach: Understand Before You Code The first instinct on a project like this is to start mapping APIs and writing sync jobs. We did not do that. We spent the first phase understanding the existing processes. We sat with the operations team and watched how a shipment moved through the system. We tracked a single metal shipment from supplier invoice, through customs, to warehouse receipt, to the finance team’s books. We did the same for textile and chemical flows. That exercise changed our plan. We had assumed the legacy system was the source of truth for shipment data. It was not. The operations team kept a parallel spreadsheet for chemical shipments because the legacy system could not handle batch numbers the way they needed. Any integration we built would have to account for that, or we would lose data. Lessons from this phase: Map the actual process, not the documented process. The two are rarely the same. Find the shadow systems. Spreadsheets, side databases, and manual workarounds exist because the official system has gaps. Talk to the people doing the data entry. They know where the errors come from better than anyone. Only after we understood the real flows did we start designing the integration. Phased Migration, Not Big Bang We did not rip out the legacy system in one go. A big bang migration on a company this size would have stopped operations for days and risked data loss nobody could afford. Instead, we phased it. Phase 1: Stand up the new system alongside the legacy system. We built the new operational system and ran it in parallel. New shipments went into both systems. This let us validate that the new system handled the data correctly without committing to it fully. Phase 2: Build the bookkeeping integration. Once the new system had live data, we connected it to the bookkeeping software. This was the highest-value integration because finance was spending the most time on manual entry. Phase 3: Build the employee management integration. With bookkeeping stable, we added the link to the employee management system for productivity tracking. Phase 4: Cut over. Only when all integrations were running and verified did we stop entering data into the legacy system. Each phase had a rollback plan. If the bookkeeping integration started sending bad data, we could disable the sync and fall back to manual entry for a day while we fixed it. That safety net mattered more than speed. The Integration Work This was the core of the project. Three pieces: the new operational system, the bookkeeping system, and the employee management system. Each connection had its own challenges. Connecting to the Bookkeeping System The bookkeeping software had an API, but it was not designed for the volume or the shape of data we needed to push. Invoice records in the operational system did not map cleanly to the journal entry structure the bookkeeping system expected. We had to build a data mapping layer. Each operational invoice became one or more journal entries, with account codes resolved based on shipment type and cost category. Metal shipments hit different accounts than textile shipments. Key decisions we made: Sync schedule: We ran the bookkeeping sync every 15 minutes during business hours. Real-time was unnecessary and would have overloaded the bookkeeping API. A 15-minute lag was acceptable to finance. Idempotency: Every sync record carried a unique reference. If the sync ran twice, the bookkeeping system would not create duplicate entries. This single decision prevented most of the data corruption issues we would have hit otherwise. Reconciliation reports: After each sync, we generated a report comparing what the operational system sent against what landed in the bookkeeping system. Discrepancies went to a queue for manual review. Connecting to the Employee Management System The employee management integration was different. Its job was to pull operational work data so productivity could be tracked per employee, per department, per shipment type. The challenge here was identity. The operational system tracked work by shipment reference and team. The employee management system tracked work by employee ID. We needed a mapping between the two. We solved this by adding an employee ID field to operational records at the point of entry. When a team member logged work on a shipment, they selected their employee ID from a dropdown sourced from the employee management system. That gave us a reliable join. This required a