ZFS: preparing for future drive additions

Mirroring is the way to go here. It'll let you easily add additional mirrored pairs to extend your existing volume. No downtime, no tricks, just add more disks and you're on your way. Unless you really need the 6TB of 4x2TB in RAIDZ, the 4TB of 4x2TB mirrored is a better bet. Mirroring also has the benefit of lowered CPU overhead, better performance and easier recovery in case of failure.

But seeing as people prefer to live on the edge and save a couple bucks here's how migrate your data from a two drive mirror to a four drive raidz setup:

  • Create a 2 drive mirror zpool create mirror cXt1d0 cXt2d0 yourPool
  • Happily fill your drives with data
  • Buy drives 3, 4 (same size as drives 1, 2)
  • Take a deep breath, you're about to forgo redundancy cause you're cheap.
  • Break the mirror removing drive 2: zpool detach cXt2d0 yourPool
  • Create a sparse file the same size as your drives: mkfile -n 500GB /path/file.img
  • Create a four drive raidz pool using drives 2, 3, 4 and the sparse file: zpool create raidz cXt2d0 cXt3d0 cXt4d0 /path/file.img newPool
  • Degrade the raidz array by detaching the sparse file: zpool detach /path/file.img newPool
  • Copy data from yourPool to newPool with zfs send/zfs recv (or just rsync/cp it)
  • Destroy the original mirrored pool: zpool destroy yourPool
  • Attach drive1 to the raidz pool: zpool attach cXt1d0 newPool
  • Wait for ZFS to resilver the drive (it'll take hours, don't be fooled by the status % not moving, there's a bug. Just let it grind.)
  • Exhale that deep breath, you again have a redundant pool.