228 lines
4.4 KiB
Markdown
228 lines
4.4 KiB
Markdown
# suzhu kgrag migration steps
|
|
|
|
This flow is the safer staged migration path:
|
|
|
|
1. export data inside the source `kgrag` container
|
|
2. `docker cp` the archive to the source host
|
|
3. `scp` the archive to `192.168.1.108`
|
|
4. extract it under `/home/workspace/kgrag`
|
|
5. import Neo4j snapshot, rebuild indexes, update URLs, verify
|
|
|
|
It avoids directly running cross-host rsync against the live target directory.
|
|
|
|
## Hosts
|
|
|
|
Source host:
|
|
|
|
```text
|
|
192.168.0.164 root Echo@456
|
|
```
|
|
|
|
Target host:
|
|
|
|
```text
|
|
192.168.1.108 root zdht123@
|
|
```
|
|
|
|
Target migration directory:
|
|
|
|
```text
|
|
/home/workspace/suzhu-kgrag_migration
|
|
```
|
|
|
|
Target kgrag app directory:
|
|
|
|
```text
|
|
/home/workspace/kgrag
|
|
```
|
|
|
|
Inside the target `kgrag` container, the mounted script directory is:
|
|
|
|
```text
|
|
/app/suzhu-kgrag_migration
|
|
```
|
|
|
|
## 1. Copy export script from target to source
|
|
|
|
Run on `192.168.1.108`:
|
|
|
|
```bash
|
|
scp /home/workspace/suzhu-kgrag_migration/00_export_from_source_container.sh \
|
|
root@192.168.0.164:/home/workspace/
|
|
```
|
|
|
|
## 2. Export data on source host
|
|
|
|
Run on `192.168.0.164`:
|
|
|
|
```bash
|
|
bash /home/workspace/00_export_from_source_container.sh
|
|
```
|
|
|
|
By default it exports from:
|
|
|
|
```text
|
|
container: kgrag
|
|
container dir: /app
|
|
```
|
|
|
|
It packages these items if they exist:
|
|
|
|
```text
|
|
files
|
|
kg_output
|
|
kg_snapshot.json
|
|
fault_wenjian
|
|
entity_registry.json
|
|
tree_data.json
|
|
```
|
|
|
|
The archive will be created on the source host under:
|
|
|
|
```text
|
|
/home/workspace/kgrag_export/kgrag_data_YYYYmmddHHMMSS.tar.gz
|
|
```
|
|
|
|
If the source container name is not `kgrag`, run:
|
|
|
|
```bash
|
|
KGRAG_CONTAINER=your_container_name bash /home/workspace/00_export_from_source_container.sh
|
|
```
|
|
|
|
## 3. Copy archive back to target
|
|
|
|
Run on `192.168.1.108`:
|
|
|
|
```bash
|
|
mkdir -p /home/workspace/kgrag_import
|
|
scp root@192.168.0.164:/home/workspace/kgrag_export/kgrag_data_*.tar.gz \
|
|
/home/workspace/kgrag_import/
|
|
```
|
|
|
|
Check the archive:
|
|
|
|
```bash
|
|
ls -lh /home/workspace/kgrag_import/kgrag_data_*.tar.gz
|
|
tar -tzf /home/workspace/kgrag_import/kgrag_data_*.tar.gz | sed -n '1,40p'
|
|
```
|
|
|
|
## 4. Extract archive into target kgrag directory
|
|
|
|
Run on `192.168.1.108`:
|
|
|
|
```bash
|
|
cd /home/workspace/kgrag
|
|
tar -xzf /home/workspace/kgrag_import/kgrag_data_*.tar.gz
|
|
```
|
|
|
|
Check expected files:
|
|
|
|
```bash
|
|
ls -ld /home/workspace/kgrag/files \
|
|
/home/workspace/kgrag/kg_output \
|
|
/home/workspace/kgrag/fault_wenjian 2>/dev/null || true
|
|
ls -lh /home/workspace/kgrag/kg_snapshot.json \
|
|
/home/workspace/kgrag/entity_registry.json \
|
|
/home/workspace/kgrag/tree_data.json 2>/dev/null || true
|
|
```
|
|
|
|
## 5. Import Neo4j snapshot
|
|
|
|
Run on `192.168.1.108`:
|
|
|
|
```bash
|
|
cd /home/workspace/suzhu-kgrag_migration
|
|
bash 02_load_neo4j.sh
|
|
```
|
|
|
|
This reads:
|
|
|
|
```text
|
|
/home/workspace/kgrag/kg_snapshot.json
|
|
```
|
|
|
|
and imports it inside the target `kgrag` container through:
|
|
|
|
```text
|
|
/app/kg_snapshot.json
|
|
```
|
|
|
|
## 6. Rebuild Neo4j indexes
|
|
|
|
Run on `192.168.1.108`:
|
|
|
|
```bash
|
|
cd /home/workspace/suzhu-kgrag_migration
|
|
bash 02b_rebuild_index.sh
|
|
```
|
|
|
|
## 7. Update old source URLs in Neo4j
|
|
|
|
Run on `192.168.1.108`:
|
|
|
|
```bash
|
|
cd /home/workspace/suzhu-kgrag_migration
|
|
bash 03_run_in_container.sh
|
|
```
|
|
|
|
This executes inside the target container:
|
|
|
|
```text
|
|
/app/suzhu-kgrag_migration/03_update_neo4j_urls.py
|
|
```
|
|
|
|
It replaces URLs like:
|
|
|
|
```text
|
|
http://192.168.0.164:9085
|
|
```
|
|
|
|
with:
|
|
|
|
```text
|
|
http://192.168.1.108:9085
|
|
```
|
|
|
|
## 8. Verify migration
|
|
|
|
Run on `192.168.1.108`:
|
|
|
|
```bash
|
|
cd /home/workspace/suzhu-kgrag_migration
|
|
bash 05_run_in_container.sh
|
|
```
|
|
|
|
This executes inside the target container:
|
|
|
|
```text
|
|
/app/suzhu-kgrag_migration/05_verify_migration.py
|
|
```
|
|
|
|
## Full command sequence
|
|
|
|
```bash
|
|
# On 192.168.1.108
|
|
scp /home/workspace/suzhu-kgrag_migration/00_export_from_source_container.sh root@192.168.0.164:/home/workspace/
|
|
|
|
# On 192.168.0.164
|
|
bash /home/workspace/00_export_from_source_container.sh
|
|
|
|
# On 192.168.1.108
|
|
mkdir -p /home/workspace/kgrag_import
|
|
scp root@192.168.0.164:/home/workspace/kgrag_export/kgrag_data_*.tar.gz /home/workspace/kgrag_import/
|
|
cd /home/workspace/kgrag
|
|
tar -xzf /home/workspace/kgrag_import/kgrag_data_*.tar.gz
|
|
|
|
cd /home/workspace/suzhu-kgrag_migration
|
|
bash 02_load_neo4j.sh
|
|
bash 02b_rebuild_index.sh
|
|
bash 03_run_in_container.sh
|
|
bash 05_run_in_container.sh
|
|
```
|
|
|
|
## Notes
|
|
|
|
- Do not run `01_copy_files.sh` for this staged flow. That script directly pulls files with `rsync` from the source host.
|
|
- `02_load_neo4j.sh`, `02b_rebuild_index.sh`, and `03_run_in_container.sh` modify the target Neo4j data/indexes.
|
|
- Keep the archive under `/home/workspace/kgrag_import` until verification passes.
|