Common Data Migration Mistakes to Avoid

I’ve audited 120+ ERP data migrations in the last 5 years. 80% of them failed. And most ERP failures are not because it’s SAP, Oracle, or Dynamics. Not even the custom build from 2012. They fail because the data going in was never cleaned. Here’s what I keep seeing (even in $10M+ projects): In 80% of failed ERP migrations, I found: ☠️ UOM mismatches that break inventory. ☠️ Customer and vendor duplicates. ☠️ Zombie SKUs and dead warehouses. ☠️ Orphaned transactions. ☠️ No audit trail of what got transformed. Here’s my Data Migration Checklist (to use before go-live): ✅ Units of Measure (UOM): → Are all UOMs mapped 1:1 between legacy and new ERP? → Have we tested conversion logic in live transactions? ✅ Master Data Uniqueness: → Do we have duplicate SKUs, vendors, or customers? → What’s the deduplication logic? Who owns it? ✅ Historical Data Mapping: → Are all past transactions (GR/IR, payments, returns) traceable? → Can we audit them after go-live? ✅ Open Transactions Review: → How many open POs, SOs, GRNs exist in legacy? → Who validated carry-forward rules? ✅ Dummy Runs with Real Data: → Did we run full-cycle transactions with migrated data in UAT? → Were accounting, tax, and inventory balances reconciled? ✅ Cleanup Ownership: → Who is responsible for final data sign-off—IT or Finance? → Is it documented? I think ERP is not an Excel import. It’s a financial and operational rebirth. And the data is either your foundation or your downfall. How confident are you in the quality of the data being loaded into your next ERP? ♻️ 𝐑𝐄𝐏𝐎𝐒𝐓 so others can learn.

A single report migration took one month. (We started coding before asking the right questions.) Brownfield data migration. Legacy SQL Server. Stored procedures from a DBA who left 2 years ago. Zero documentation. We needed to migrate one report to the cloud. Timeline: 4 weeks. 𝗛𝗲𝗿𝗲'𝘀 𝘄𝗵𝗮𝘁 𝘄𝗲 𝗱𝗶𝗱 𝘄𝗿𝗼𝗻𝗴: Dove straight into development. Mirrored the legacy logic. Field by field. Join by join. No context for why the query filtered on three specific procedure codes. No idea why date logic was limited to 4 months. No understanding of whether half the fields were even needed. We reverse-engineered 400 lines of SQL without knowing what the business actually needed. 𝗢𝗻𝗲 𝗺𝗼𝗻𝘁𝗵 𝗹𝗮𝘁𝗲𝗿: Still not done. Scope creep. Complexity everywhere. Stakeholders asking: "Why is this taking so long?" 𝗪𝗵𝗮𝘁 𝘄𝗲 𝘀𝗵𝗼𝘂𝗹𝗱 𝗵𝗮𝘃𝗲 𝗱𝗼𝗻𝗲: 𝗦𝘁𝗲𝗽 𝟭: 𝗖𝗼𝗻𝗳𝗶𝗿𝗺 𝘁𝗵𝗲 𝗿𝗲𝗽𝗼𝗿𝘁'𝘀 𝗶𝗻𝘁𝗲𝗻𝘁 What question is the business trying to answer? Why does this report exist? Don't start coding until you know. 𝗦𝘁𝗲𝗽 𝟮: 𝗖𝗼𝗻𝗱𝘂𝗰𝘁 𝗮𝗻 𝗶𝗻𝘃𝗲𝗻𝘁𝗼𝗿𝘆 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀 List every field from legacy. Decide what's actually needed. Document it in a spreadsheet: Field Name, Table, Need (Y/N), Notes. Cut the noise before you code. 𝗦𝘁𝗲𝗽 𝟯: 𝗗𝗲𝘃𝗲𝗹𝗼𝗽 𝘄𝗶𝘁𝗵 𝗰𝗹𝗮𝗿𝗶𝘁𝘆 Now you know what to build and why. No wasted joins. No unnecessary complexity. 𝗧𝗵𝗲 𝗹𝗲𝘀𝘀𝗼𝗻: Legacy logic is often wrong. Unnecessary fields. Outdated filters. Complexity for no reason. Don't blindly mirror it. Ask questions. Document what's needed. Then code. 𝗧𝗟;𝗗𝗥: Starting development before understanding the business need kills timelines. Confirm intent. Inventory fields. Then build. That's how you avoid month-long migrations for a single report. P.S. - Full breakdown of the 3-step process in this week's newsletter. Link in comments. 👇 ♻️ Share this if you've reverse-engineered legacy code without knowing why half of it existed. Follow me for real talk on brownfield data migrations.

“Wait, so we can just export our list from our old platform, upload it to Klaviyo, and start sending?” A potential client actually asked me that a few days ago. If I had a dollar for every time I’ve heard this, I’d have a separate revenue stream called “Migration Optimism Tax.” Let’s get real: If you think migrating to a new ESP is as simple as exporting a CSV and importing it into Klaviyo, you’re setting yourself up for a mess. Don't assume it's a 5-minute job only to get blindsided by the details. Here’s what gets missed often when brands decide, "let's just do it ourselves," and why it matters: 1. Data Cleanliness Is Non-Negotiable It’s not just about “removing bounces.” You need to hunt down and suppress dead emails from old contests, bots, fakes, and duplicates from messy imports. If you skip this, you’ll tank your sender reputation on Day 1. 2. Segmentation Logic Gets Lost in Translation Your old platform’s logic for “engaged” or “VIP” segments does not just port over. Segments are built on event data that usually doesn’t make the jump. In Klaviyo, you have to rebuild each one manually, mapping rules and sometimes starting from scratch. Test before trusting. Wrong group, wrong message, instant performance drop. 3. Preferences and History Aren’t Automatic Order data syncs from Shopify, but years of custom events, loyalty info, or tags don’t. Same for signup source or list preferences. Miss these, and you lose the context that powers your targeting and flows. Make a mapping doc before you even export. 4. Consent, Tags, and Compliance Require Attention Bulk import makes you responsible for keeping unsubscribes, bounced, and suppressed emails out of future sends. Audit opt-in status, consent dates, compliance tags—anything with legal consequences. If you get this wrong, expect complaints, spam reports, and a headache with compliance teams. 5. QA Is Where Migrations Fall Apart Every flow, segment, and campaign needs to be tested in a real inbox, not just “previewed.” Check dynamic content, make sure you don’t have broken links or “Hi, first_name.” Look at mobile and desktop. Every minute spent here saves you days of chaos later. If you think migration is easy, you’re either new to this or you’ve never checked your numbers after launch. The teams that treat migration as a technical project, not an admin task, are the ones who scale without months of cleanup. If you’ve lived through a messy migration, I want to hear your worst surprise. If you’re planning one, what’s the one thing you’re worried about?

If you're about to migrate a database: 1. Inventory everything. 2. Sequence by latency sensitivity. 3. Build a rollback plan and assume you'll need it These 3 rules saved us after 3 failed moves. Let me tell you why each one matters. When I was consulting for HP Enterprise, we were tasked with consolidating massive data centers. Not cloud to cloud migrations. This was moving entire enterprise accounts from old EDS data centers to HP's new facilities across the United States. The stakes were enormous. Every move required shutting down production systems, copying terabytes of data, and hoping everything worked when we flipped the switch back on. Rule 1: Inventory everything We learned this the hard way. You can't just look at the obvious stuff like databases and applications. You need to map every single connection, every dependency, every integration that touches your data. That random reporting tool someone built 3 years ago? It's going to break your migration if you don't account for it. Rule 2: Sequence by latency sensitivity Not all data is created equal. Some systems can tolerate a few milliseconds of delay. Others will fail catastrophically if there's any lag between the application and database. We'd spend weeks just figuring out what could move first, what had to wait, and what couldn't be separated at all. Rule 3: Build a rollback plan and assume you'll need it This is where most people get cocky. They think their plan is perfect and skip the "what if this goes wrong" scenario. We had full backup strategies for every single migration. Good thing, because we needed them more often than we'd like to admit. Long maintenance windows would start. We'd shut down systems, start copying data, begin the cutover process. Then something would go sideways. The database didn't sync properly. An application couldn't connect. Performance was terrible in the new environment. When that happened, we'd roll everything back and try again later. Those painful experiences taught me there had to be a better way. That's actually what led me to start Verge Technologies. The idea that you could move stateful systems like databases without massive downtime, without all the planning overhead, and without the constant fear of failure. But even with better technology, those three rules still apply. Inventory everything. Sequence by latency. Plan for rollback. Because no matter how good your tools are, migrations are inherently risky. The difference between success and disaster often comes down to preparation.

After years building event-driven systems. Here are the top 4 mistakes I have seen: 1. Duplication Events often get re-delivered due to retries or system failures. Without proper handling, duplicate events can: • Charge a customer twice for the same transaction. • Cause duplicate inventory updates, messing up stock levels. • Create inconsistent or broken system states. Solution: • Assign unique IDs to every event so consumers can track and ignore duplicates. • Design event processing to be idempotent, ensuring repeated actions don’t cause harm. 2. Not Guaranteeing Order Events can arrive out of order when distributed across partitions or queues. This can lead to: • Processing a refund before the payment. • Breaking logic that relies on correct sequence. Solution: • Use brokers that support ordering guarantees (e.g., Kafka). • Add sequence numbers or timestamps to events so consumers can detect and reorder them if needed. 3. The Dual Write Problem When writing to a database and publishing an event, one might succeed while the other fails. This can: • Lose events, leaving downstream systems uninformed. • Cause mismatched states between the database and event consumers. Solution: • Use the Transactional Outbox Pattern: Store events in the database as part of the same transaction, then publish them separately. • Adopt Change Data Capture (CDC) tools to track and publish database changes as events automatically. 4. Non-Backward-Compatible Changes Changing event schemas without considering existing consumers can break systems. For example: • Removing a field might cause missing data for consumers. • Renaming or changing field types can trigger runtime errors. Solution: • Maintain versioned schemas to allow smooth migration for consumers. • Use formats like Avro or Protobuf that support schema evolution. • Add adapters to translate new schema versions into older ones for compatibility. "Every schema change is a test of your system’s resilience—don’t fail it." What other mistakes have you seen out there?

The “Before & After” Data Transformation Story In the lead-up to our SAP migration, we weren’t just preparing systems — we were unearthing years of neglected, inconsistent, and chaotic data. If we are honest, most of the time, it felt less like digital transformation and more like an archaeological excavation. We were buried in layers of spreadsheets, conflicting legacy reports, and systems that hadn’t seen a clean-up in over a decade. Each click revealed more clutter: customer names spelled five different ways, address fields mixing “St.” and “Street” like it was a coin toss, duplicate records stacked on top of each other, and critical fields left blank or filled with guesswork. It was more than just messy — it was risky - A complete nightmare! Data was being pulled from everywhere and nowhere. No single source of truth. No consistency. Just a patchwork of outdated inputs fuelling vital business operations. The worst part? We had to tackle it manually. A Time Sink: Highly skilled people stuck doing low-value, repetitive tasks. An Error Magnet: Fatigue set in. Errors crept through. Fix one issue, uncover two more. A Business Risk: Dirty data meant dirty output. Reports couldn’t be trusted. Customers were misbilled. Orders were sent to the wrong place. And confidence in the system? Gone. We knew we couldn’t carry that baggage into SAP. Something had to change. At this point, we built a purpose-specific solution which was created to automate and streamline data cleansing and validation, giving us the ability to: Proactively identify and rectify errors with precision. Ensure data consistency across all records. Validate information against business rules before migration. This impacts business by: 🔹Reducing Pre-Migration Data cleansing and validation Effort by Up to 75% Freeing up SMEs for strategic tasks, cutting contractor costs, and accelerating migration timelines. 🔹Delivering >99% Accuracy in Key Master Data Minimising migration errors, de-risks go-live, building trust in the new SAP system from day one. 🔹Reducing Migration Delays and Rework by 20–40% Fewer surprises in load cycles and UAT, protecting timelines, budgets, and overall project momentum. 🔹Achieving 100% Data Auditability and Compliance Ensuring full traceability, streamlining audits, and providing a defensible position on data quality from day one. 🔹Reducing Post-Go-Live Errors by 15–30% Fewer issues like misbilling and mis-shipments, leading to smoother operations, faster user adoption, and trusted SAP insights. If any of this sounds familiar, you're not alone. The good news is that we have built a solution which has already helped others through their migration journey, and we’d be happy to share it if it’s useful. Just drop us a message. Created in collaboration with Pawel Lipko ↗️

Dear IT Auditors, Auditing Data Migration Data migration projects are among the riskiest IT initiatives an organization can undertake. Whether it’s moving from on-prem to cloud, consolidating legacy systems, or integrating after a merger, the stakes are high. A single error can lead to data corruption, compliance violations, or business downtime. That’s why data migration assurance has become a critical part of IT audit and GRC. Here’s how auditors can add value when reviewing migration projects: 📌 Pre-Migration Planning: The foundation of assurance is in the planning. Review project charters, migration strategies, and risk assessments. Confirm that the scope is clearly defined (which data, which systems, what timelines). Lack of upfront clarity is often the root cause of failed migrations. 📌 Data Mapping and Transformation Rules: Check whether data mapping is documented and transformation logic is validated. Auditors should ensure data formats, field lengths, and relationships are consistent across systems. If this step is rushed, errors cascade downstream. 📌 Test Migration Runs: Review evidence of test migrations. Were trial loads conducted with sample data? Did the organization reconcile totals and critical records? This is where issues surface early, and auditors should confirm there’s evidence of structured testing. 📌 Reconciliation and Validation: After migration, controls should validate that all data migrated accurately and completely. Audit procedures include reconciling record counts, financial totals, and critical data fields between legacy and new systems. Spot checks on high-risk data (like customer balances) are essential. 📌 Access and Security Controls: Migrations often involve temporary elevated access for IT teams. Confirm that privileged access was approved, monitored, and revoked post-migration. Review whether sensitive data was encrypted in transit. 📌 Business Continuity and Rollback: Strong migration assurance requires consideration of what if the migration fails. Auditors should verify rollback procedures, data backups, and business continuity testing. It’s not enough to hope the migration works; the plan must cover failure scenarios. 📌 Post-Migration Monitoring: The job isn’t done after cutover. Review post-migration monitoring reports, error logs, and end-user acceptance testing. Assurance means confirming that business processes continue smoothly without disruption. Data migration assurance goes beyond ticking boxes. It provides stakeholders with confidence that systems, data, and compliance remain intact during one of the most disruptive IT events. For auditors, this presents an opportunity to demonstrate real business value, not just control testing. #DataMigration #ITAudit #RiskManagement #InternalAudit #DataGovernance #GRC #CyberSecurityAudit #ITControls #CloudAudit #ITRisk #CyberYard #CyberVerge

Thinking of migrating from Salesforce to HubSpot? I’ll say this upfront.... it’s not just a “data move.” It’s a logic shift. And if you treat it like a simple export/import job, you’re setting yourself up for chaos. See, we’ve helped 150+ brands through this switch at Mavlers. And the pattern’s always the same: most teams underestimate the mapping work. The most common issues that they face are - -- Custom fields that don’t match -- Required fields that suddenly break things -- Workflows that simply don’t translate -- Relationships between contacts, companies, deals .... lost in translation I still remember a fintech brand that botched their first migration. Cost them nearly £450K in missed pipeline within 60 days. I know it's brutal. So what should be the move? -- Audit everything in Salesforce... standard & custom -- Build a detailed field mapping doc, not just objects, but every property -- Clean your data before it lands in HubSpot -- Use a sandbox or pilot batch before full send -- Don’t forget ... not everything is importable. Reports, campaigns, scoring models… most need to be rebuilt If you’re in the stage of planning or you’re halfway through and already hitting friction... we put together a no-BS guide based on real migration projects. Read it here- https://lnkd.in/d5a4UbDD Hope it helps.

We interviewed data leaders who've been through major platform migrations, and one thing became crystal clear: migrations are so critical to the data team’s ability to impact the business that they define the data team’s success and data engineer's careers. The migration I ran at Lyft certainly defined mine — and put me on a path to build Datafold. Migrations are a perfect storm of high stakes, tight deadlines, and zero room for error. You're not just moving data – you're juggling pipeline re-engineering, data validation, stakeholder management, and keeping production running. The insights from these leaders were surprisingly consistent: > Lift-and-shift first, optimize later. The temptation to fix everything during migration is strong but often creates more problems than it solves. > Stakeholder trust is everything. Without clear proof of data parity, skepticism creeps in, systems run in parallel longer than planned and costs balloon. > The "last mile" is where migrations go to die. When you think you're done, stakeholder reviews uncover edge cases that trigger new refinement cycles. But here's what gives me hope: we're finally at a technological inflection point. What used to take years can now be done in months., and whatrequired armies of engineers can now be automated. We've compiled these stories, lessons, and a practical framework for modern migrations in our latest guide: https://lnkd.in/em-BBqM4

🚨 Thinking about switching from Salesforce to HubSpot (or vice versa)? Pause. Because doing it too fast — or without a rock-solid plan — is one of the fastest ways to destroy your CRM’s data integrity. Here’s what I’ve seen happen far too often: 🧩 Custom objects don’t map cleanly. 🔁 Automations break mid-flow. 🕳️ Historical data gets dropped or misclassified. 🧼 “We’ll clean it up later” turns into 6 months of chaos. The result? 🚫 Sales loses trust in the data. 🚫 Marketing can't segment or personalize. 🚫 Leadership flies blind on forecasts and reporting. And worst of all — your RevOps team ends up duct-taping systems just to keep the engine running. CRM migrations aren't just data projects. They’re revenue operations transformations. Before you migrate: ✅ Audit your current system thoroughly ✅ Map out every field, object, and automation (and I mean EVERY field) ✅ Define what must be preserved vs what can be left behind ✅ Build a sandbox environment to test before going live ✅ Involve RevOps early — not just IT or dev team Done right, a CRM migration can level up your GTM engine. Done wrong? It’s a six-figure cleanup job waiting to happen. 💬 Thinking of migrating this year? Attached is a migration checklist that has saved my clients significant time and money!