mirror of
https://github.com/t8y2/dbx.git
synced 2026-11-15 11:36:17 +08:00
175 lines
13 KiB
Plaintext
175 lines
13 KiB
Plaintext
---
|
||
title: Data Transfer
|
||
description: Copy tables between connections, databases, schemas, or database engines and track the result of each table.
|
||
---
|
||
|
||
Data Transfer copies tables or collections from one source into another connection, database, or schema. Use it for environment migration, test data preparation, and one-off consolidation. It is not continuous replication, CDC, or bidirectional synchronization.
|
||
|
||
<Callout type="warn">Cross-engine transfer passes through DBX field and type conversion. Target privileges, constraints, triggers, defaults, and engine semantics still determine the result. Rehearse important migrations and validate row counts and key fields afterward.</Callout>
|
||
|
||
## Entry Points
|
||
|
||
### Data Transfer Wizard
|
||
|
||
The wizard is designed for multiple tables. Select the source and target connection, catalog, database, and schema, then configure table creation, write mode, target-name case, and batch size.
|
||
|
||
### Sidebar Copy and Paste
|
||
|
||
After copying one or more tables in the object tree, paste them at a target location:
|
||
|
||
- Within the same context, choose structure and data, structure only, or data only, and edit each target name
|
||
- Pasting into another connection, database, or schema opens the Data Transfer wizard with the source tables and target prefilled
|
||
- Whole-row copy excludes generated or computed columns and applies path-specific handling for PostgreSQL identity and SQL Server identity columns
|
||
|
||
### Export Query Results to a Database
|
||
|
||
Choose **Export to Database** from the query result grid's export or context menu, then select the target connection, database, schema where applicable, and table name.
|
||
|
||
After selecting the destination, use **Execution preview** to review the source, destination, currently loaded row and column counts, source columns, and planned operations. Preview does not write to the database; execution still loads the full exportable result and checks limits. **Start Transfer** opens a separate confirmation step; only **Confirm transfer** begins execution. Overwriting an existing table explicitly warns that all existing target rows will be deleted. Returning to settings preserves your selections.
|
||
|
||
- **Create target table**: for a single-table `SELECT * FROM table` on the same engine, DBX reads the source DDL and preserves column types, lengths, precision, defaults, comments, primary keys, indexes, and table options while replacing the table name. Schema-wide index and constraint names are adjusted to avoid collisions. Unsafe or unavailable DDL stops the transfer instead of silently creating text columns.
|
||
- Projections, joins, expressions, and cross-engine queries generate a table from result fields. They have no unique whole-table DDL. Insufficient metadata or target timestamp precision uses text types to preserve values without inferring source constraints. Writes to existing tables reject temporal values that cannot retain their precision before writing.
|
||
- A new target name must not already exist. To write to an existing table, disable **Create target table**. Append keeps its rows; overwrite runs `DELETE + INSERT` in a transaction while keeping its structure, foreign keys, and triggers. Existing MySQL targets require InnoDB.
|
||
- The transfer checks the complete result, export row limit, pagination, and large-value previews. Procedures and statements that modify data are not rerun for export; only their fully loaded results can be reused.
|
||
- Results with an active cursor or manual transaction must first load all rows in the grid. Ordinary pagination reads through the source query session to preserve session settings and avoid reading different data on another connection.
|
||
- Copying source table structure inside a manual transaction is not supported. Commit or roll back and rerun the query before choosing **Create target table**. MySQL and H2 temporary tables also stop structure copying instead of creating a target with a different lifetime.
|
||
- Cancel during preparation. Once writing starts, wait for the transaction to finish. Drivers that implicitly commit table creation attempt to remove the table created by this transfer if writing fails; cleanup failures identify that table.
|
||
|
||
This entry point lists target drivers with implemented transactional writes. It does not migrate standalone triggers, grants, or cross-table dependencies. Complex DDL, incompatible types, and unsupported identity handling fail explicitly. Use the Data Transfer wizard for database or multi-object migrations.
|
||
|
||
## Wizard Workflow
|
||
|
||
<Steps>
|
||
<Step>
|
||
### Select the Source
|
||
|
||
Choose the source connection, catalog when applicable, database, and schema. DBX lists transferable base tables; MongoDB paths list collections.
|
||
|
||
</Step>
|
||
<Step>
|
||
### Select the Target
|
||
|
||
Choose another connection, database, or schema. Source and target cannot be the exact same location.
|
||
|
||
</Step>
|
||
<Step>
|
||
### Choose Tables
|
||
|
||
Select all, search, or choose a subset. A single request can only contain tables from one source context.
|
||
|
||
</Step>
|
||
<Step>
|
||
### Configure Options
|
||
|
||
Review target-table creation, write mode, target-name case, and batch size.
|
||
|
||
</Step>
|
||
<Step>
|
||
### Run in Background and Review
|
||
|
||
The task keeps running after the dialog closes. The progress center reports the current table, table index, rows, duration, failed tables, and cancellation state.
|
||
|
||
</Step>
|
||
</Steps>
|
||
|
||
## Transfer Options
|
||
|
||
| Option | Details |
|
||
| --- | --- |
|
||
| Create target tables | Generates target structure from source metadata when the table does not exist and the path supports it |
|
||
| Rebuild structure and data | Renames existing target tables to backup (`__dbx_bak_` prefix), rebuilds from source structure, inserts data, then drops backup on success. Any failure preserves the backup table for manual recovery. Requires confirmation on production databases. Available for MySQL, PostgreSQL, SQL Server, KingBase, GaussDB, OpenGauss, KWDB, GoldenDB, SQLite, DuckDB, and Cloudflare D1. |
|
||
| Append | Keeps existing target rows and inserts source rows |
|
||
| Overwrite | Clears each target table before writing its source rows |
|
||
| Upsert | Updates or inserts by key when the target path supports it and a usable key is available |
|
||
| Target table name case | Preserve, lowercase, or uppercase; transformed target names must remain unique |
|
||
| Batch size | Controls paged reads and batched writes; the UI default is 1000 |
|
||
| Operation timeout | Inherits the source and target connection query timeouts by default; override this transfer with `0`–`3600` seconds, where `0` means unlimited |
|
||
|
||
If table creation is disabled, compatible target tables must already exist. Even with table creation enabled, cross-engine DDL may require manual changes when the target does not support a source type, default, constraint, or object.
|
||
|
||
The operation timeout does not cap the overall transfer duration. Drivers with incremental result streams treat it as an inactivity window that resets whenever data arrives; drivers that return a result all at once apply it to the total duration of each database operation. A custom value affects only this transfer and its saved task, not the connection or query-editor timeout settings.
|
||
|
||
<Callout type="warn">**Rebuild structure and data** replaces existing target tables and their indexes, triggers, constraints, comments, and object permissions with source structure. The original table is renamed to a backup first, then dropped after a successful transfer. If any step fails, the backup table is retained and can be manually restored.</Callout>
|
||
|
||
## Structure SQL Preview
|
||
|
||
Structure-only transfers do not change the target silently: DBX plans the structure statements first and shows them in a read-only confirmation dialog (syntax highlighted, copyable, wrapped) before anything runs. Cancel there and nothing is executed.
|
||
|
||
Above the SQL Preview, DBX shows a **Structure transfer plan** with operation counts and an object-level list, including planned table creation, explicit skips for existing targets, rebuilds, and structured indexes, foreign keys, sequences, or comments where available. Operations come from the same planning decisions as the SQL; DBX does not parse SQL to guess what will happen.
|
||
|
||
This summary describes the transfer plan, not a full source/target schema diff. It does not compare column types, defaults, or nullability, detect target-only objects, or claim that an existing target is schema-identical. For example, `SKIP table users — target already exists` means no table structure DDL is planned for that table; it does not assert that the source and target tables match.
|
||
|
||
The preview covers the table-level structure operations the transfer is about to run:
|
||
|
||
- `CREATE TABLE` (and the inline PostgreSQL-compatible schema, when one has to be created)
|
||
- `COMMENT` statements
|
||
- Deferred MySQL-family `ALTER TABLE ... ADD CONSTRAINT ... FOREIGN KEY` statements
|
||
- PostgreSQL-compatible indexes and foreign keys
|
||
- PostgreSQL sequence statements implied by the source structure
|
||
- plus the rebuild rename/backup and cleanup statements when rebuild is enabled.
|
||
|
||
It is a preview, not a frozen script. Starting the transfer re-reads source and target metadata and re-runs the existing safety checks, so a target changed in the meantime can make execution differ from the preview — DBX re-validates the structure rather than replaying the shown statements blindly.
|
||
|
||
Selected non-table objects (views, routines, triggers, standalone sequences, and similar) are still transferred, but their generated DDL is not expanded in the preview; the dialog says so explicitly.
|
||
|
||
Structure and data transfers keep their current confirmation flow in this phase.
|
||
|
||
## PostgreSQL Structure and Ownership
|
||
|
||
For PostgreSQL-to-PostgreSQL transfers with table creation enabled, DBX handles supported sequences, indexes, foreign keys, and selected schema-level objects and dependencies. If source owners do not exist on the target instance, DBX prompts you to:
|
||
|
||
- Skip the affected ownership metadata
|
||
- Reassign missing owners to the current target user
|
||
- Cancel and create the required target roles first
|
||
|
||
This is not a full replacement for `pg_dump` and `pg_restore`. Use native PostgreSQL tooling when extensions, grants, policies, triggers, or dependency graphs are complex.
|
||
|
||
## Failure Handling
|
||
|
||
A multi-table task does not discard the entire queue after the first table failure. DBX records the failure, attempts later tables, and finishes either successfully or with a terminal error summary.
|
||
|
||
- Progress keeps per-table failure details
|
||
- When failures exceed the display limit, the UI reports how many entries were omitted
|
||
- A new task is rejected if another active transfer already targets the same table
|
||
- Cancellation stops later reads and writes, but committed target data is not automatically reverted
|
||
|
||
As a result, a finished task does not necessarily mean every table succeeded. Check the terminal state, failure list, target row counts, and key fields.
|
||
|
||
Oracle and OceanBase Oracle standalone routine transfers also read back the exact object, full definition, and compiler result. An acknowledged DDL with an `INVALID` object, failed readback, or mismatched definition does not count as a successful transfer. Errors retain compiler coordinates or the verification failure. Existing targets are still skipped; new routines do not use `OR REPLACE` to overwrite a competing name.
|
||
|
||
## Database Coverage
|
||
|
||
DBX filters source and target connections through the driver capability manifest. The implementation includes SQL-to-SQL, MongoDB-to-MongoDB, and selected MongoDB/SQL conversion paths. Not every connectable database supports Data Transfer.
|
||
|
||
Capabilities also vary by combination:
|
||
|
||
- Some engines transfer rows but cannot reuse source DDL
|
||
- Upsert depends on the target engine and discoverable keys
|
||
- Catalogs, schemas, identity columns, arrays, JSON, and binary values are engine-specific
|
||
- Specialized search, vector, message queue, and configuration-center connections do not use this generic table-transfer entry point
|
||
|
||
If a connection is absent from the selector, its current driver has not declared Data Transfer support; it is not merely a hidden UI action.
|
||
|
||
## Before Transferring
|
||
|
||
- Check whether the target connection has read-only protection; read-only targets are blocked
|
||
- Confirm source read privileges and target create/write privileges
|
||
- Check for target-name collisions after case conversion
|
||
- Review primary keys, unique constraints, foreign keys, triggers, and required fields
|
||
- Decide whether production data must be masked before entering a lower environment
|
||
- Use native export/import or online migration tools when the dataset is too large for an interactive transfer
|
||
|
||
## Related Features
|
||
|
||
<Cards>
|
||
<Card title="Table Import" href="/en/docs/table-import">
|
||
Import files into an existing table or create new tables in batches.
|
||
</Card>
|
||
<Card title="Database Export" href="/en/docs/database-export">
|
||
Produce reviewable SQL files for offline transfer.
|
||
</Card>
|
||
<Card title="Production Safety" href="/en/docs/production-safety">
|
||
Understand read-only protection, production protection, and privilege boundaries.
|
||
</Card>
|
||
</Cards>
|