MIGRATION DECISION / REDSHIFT → DATABRICKS
Redshift → Databricks
Migration Strategy
Find the lowest-risk path based on your data, workload, migration window and coexistence requirements.
02 / RECOMMENDATION
ASSESSMENT CONTEXT
Redshift → Databricks migration assessment
Data12 TB · 340 tables · largest 650 GBWorkload1–5% daily change · MERGE / upsertTransitionRedshift active · 3–6 months · Scheduled incremental loadsTargetDatabricks becomes primary platform · No preference
RECOMMENDED MIGRATION STRATEGY
Iceberg Coexistence
Use Iceberg tables in S3 as an interoperable layer during a phased transition.
77/ 100Low confidence
Iceberg Coexistence is preferred, but UNLOAD → S3 is also a very close option.
Migration approachPhased migration
Execution orderETL-first
Data movementIceberg Coexistence
Execution readinessLow risk · 80/100
WHY THIS STRATEGY
- ✓Redshift must remain active during the transition.
- ✓Both platforms can use an agreed S3 location.
- ✓The coexistence window is long enough to reward an interoperable data layer.
- ✓The data volume makes repeated full copies expensive.
CHECK BEFORE IMPLEMENTATION
- !Validate special data types against both engines before committing to Iceberg.
- !Confirm catalog ownership and one clear writer for each table.
- !Validate data types and table features against both engines.
- !Foreign Iceberg tables in Databricks can be read-only; do not assume symmetric writes.
ALTERNATIVEUNLOAD → S3Better if the coexistence window can be shortened and a final cutover is acceptable.
75/100 TARGET PATH
Migration architecture
DURABLE DATA LAYERAmazon S3Iceberg tables
SOURCEAmazon RedshiftExisting BI
TARGETDatabricksNew workloads
PHASED TRANSITIONALL FOUR OPTIONS
Strategy comparison
| Criterion | Iceberg77 | Federation20 | UNLOAD → S375 | JDBC0 |
|---|
| Data copy | Low | None | Required | Required |
|---|
| Large datasets | Excellent | Poor | Excellent | Poor |
|---|
| Near-zero downtime | Excellent | Excellent | Good | Medium |
|---|
| Coexistence | Excellent | Excellent | Medium | Poor |
|---|
| Long-term architecture | Excellent | Poor | Excellent | Medium |
|---|
| Migration simplicity | Medium | Excellent | Medium | Excellent |
|---|
IMPLEMENTATION OUTLINE
Suggested migration steps
- 01
Inventory Redshift tables, data types, write patterns, and dependencies.
- 02
Identify tables compatible with a shared Iceberg model.
- 03
Establish the S3 location, catalog ownership, and per-table writer.
- 04
Migrate a representative table and validate access from both platforms.
- 05
Redirect workloads gradually, then retire remaining Redshift dependencies.
PRODUCTION GATES
Required before cutover
- ✓
Automate table and schema comparison, including row/column counts, numeric aggregates, and distinct-value checks.
- ✓
Run both pipelines in parallel and reconcile results throughout the agreed validation window.
- ✓
Validate upstream feeds, downstream applications, BI dashboards, and business KPIs against the Databricks result.
- ✓
Prove the scheduled incremental loads path meets the downtime objective.
- ✓
Define production approval, rollback, and Redshift pipeline deprecation criteria with named owners.
TECHNICAL EVIDENCEWhy StrategyShifu recommends this
Checking official documentation…
Capability checks validate the deterministic result; they do not choose the strategy.
StrategyShifu does not just tell you where to migrate.
It helps you decide how to get there.