Commit 6e338f
2026-04-21 11:16:04 Anonymous: Raid disk migration| /dev/null .. storage setup overview/migration process and incident analysis.md | |
| @@ 0,0 1,111 @@ | |
| + | # Migration Process and Incident Analysis |
| + | |
| + | 1. Objective of migration |
| + | |
| + | The goal was to: |
| + | |
| + | Replace existing RAID storage with larger disks |
| + | Preserve all media data and permissions |
| + | Minimize downtime for Jellyfin media services |
| + | 2. Migration approach |
| + | Step 1 — New RAID creation |
| + | Two new disks were configured in PERC as RAID1 |
| + | The system exposed them as /dev/sdd |
| + | Step 2 — Filesystem creation |
| + | A GPT partition table was created |
| + | ext4 filesystem was applied to the new volume |
| + | Step 3 — Initial data transfer |
| + | |
| + | Data was copied using: |
| + | |
| + | rsync -aHAX --numeric-ids --progress |
| + | |
| + | From: |
| + | |
| + | /mnt/Media |
| + | |
| + | To: |
| + | |
| + | /mnt/newMedia |
| + | 3. Secondary migration |
| + | |
| + | A separate dataset was migrated: |
| + | |
| + | /mnt/BonusDisk/JellyfinMedia/Motorvakantie 2024 |
| + | → /mnt/Media/JellyfinMedia/Motorvakantie 2024 |
| + | |
| + | This was done using rsync with preservation flags to maintain: |
| + | |
| + | Ownership |
| + | Permissions |
| + | ACLs |
| + | Extended attributes |
| + | 4. Issue encountered during migration |
| + | Symptom |
| + | |
| + | Two MP4 files became unreadable after migration. |
| + | |
| + | Observed error |
| + | moov atom not found |
| + | Invalid data found when processing input |
| + | 5. Investigation outcome |
| + | Key findings |
| + | Files existed in both source and destination |
| + | File sizes were consistent in some cases |
| + | ffprobe failed on both copies |
| + | Original playback still worked in media players |
| + | Root cause |
| + | |
| + | The issue was not filesystem corruption but: |
| + | |
| + | MP4 container metadata inconsistency |
| + | Likely incomplete or non-finalized file state during copy |
| + | rsync faithfully copied an already problematic file state |
| + | 6. Technical explanation |
| + | |
| + | MP4 files rely on a moov atom containing index metadata. |
| + | |
| + | Possible scenarios: |
| + | |
| + | moov located at end of file (camera-style encoding) |
| + | file copied during incomplete write/finalization |
| + | metadata not properly finalized before transfer |
| + | |
| + | This causes: |
| + | |
| + | ffprobe failure |
| + | but partial playback may still succeed in tolerant players |
| + | 7. Resolution approach |
| + | |
| + | For affected files: |
| + | |
| + | Attempt container rebuild using ffmpeg: |
| + | |
| + | ffmpeg -i input.mp4 -c copy -movflags +faststart output.mp4 |
| + | If unsuccessful: |
| + | re-copy from true original source |
| + | or re-ingest from camera/archive |
| + | 8. Lessons learned |
| + | 1. Rsync preserves problems |
| + | |
| + | Rsync does not validate media integrity; it replicates file state exactly. |
| + | |
| + | 2. Media must be “finalized” before migration |
| + | |
| + | Files actively written or not fully closed can copy as corrupted artifacts. |
| + | |
| + | 3. Verification requires more than size checks |
| + | Tools like ffprobe or checksum validation are required for media integrity |
| + | 4. Two-pass rsync is essential for live systems |
| + | Initial sync during operation |
| + | Final sync after stopping writes |
| + | 9. Best-practice migration workflow (refined) |
| + | Create destination RAID and filesystem |
| + | Perform initial rsync pass |
| + | Stop all write processes |
| + | Perform final rsync pass |
| + | Validate with: |
| + | rsync --dry-run |
| + | media probing (ffprobe) |
| + | Switch mounts via UUID in /etc/fstab |
| + | Monitor system before removing old storage |