Windows Server 2012, Hyper-V, Ubuntu+ZFS VM for Backups

I set this up only to test ZFS with deduplication as an alternative to using a standard hardware RAID configuration for storing SQL backups.

Hardware used:

  • Dell R720xd
  • Dell Perc H800 RAID controller
  • Dell PowerVault MD1220
  1. Configure RAID on the Dell R720xd / Perc H800 controller. All 24 disks (2.5″ 900GB SAS in my case) as an independent RAID-0 logical volume since the H800 (as far as I can tell) doesn’t support an easy JBOD option.
    Untitled3
    Untitled1
    Untitled2
  2. In Windows Server 2012 Computer Management, I set each of the 24 volumes to GPT partition table when asked, and then set all 24 volumes “offline”. Doing so allows Hyper-V to access each disk directly, so that Windows cannot.
    Untitled5
  3. Install the Hyper-V role and rebooted.
  4. I went into the VMs settings. First I added a SCSI controller, since IDE Controllers are limited to 2 devices, while SCSI can support up to 64 devices. Then I added the 24 volumes as “physical hard disks”, matching the SCSI location number 0 – 23 to the 24 volume target number 0 – 23.
    Untitled4
  5. Also, make sure to install a virtual switch and configure your network interface for the Ubuntu VM.
  6. Downloaded Ubuntu Server 12.04 x64, installed Ubuntu with OpenSSH and Samba features. Started the Ubuntu VM.
  7. Verify Ubuntu can see the 24 volumes:
    sudo lshw -C disk

    Untitled6

  8. sudo apt-get update
  9. sudo apt-get install python-software-properties software-properties-common -y
  10. sudo add-apt-repository ppa:zfs-native/stable -y
  11. sudo apt-get update && sudo apt-get dist-upgrade -y
  12. sudo apt-get install ubuntu-zfs -y
  13. dmesg | grep ZFS
    [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-15-generic root=/dev/mapper/DPMHOST--ZFS--vg-root ro
     [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-15-generic root=/dev/mapper/DPMHOST--ZFS--vg-root ro
     [    8.529432] Adding 4190204k swap on /dev/mapper/DPMHOST--ZFS--vg-swap_1.  Priority:-1 extents:1 across:4190204k SSFS
  14. sudo vim /etc/modules
  15. Add these lines:
    spl
    zavl
    znvpair
    zunicode
    zcommon
    zfs
  16. Incorporate new modules into the boot files:
    sudo update-initramfs -u
  17. sudo reboot
  18. I created a ZFS pool called “zfs0” using raidz3 which can lose up to 3 disks using all 24 volumes:
    sudo zpool create zfs0 raidz3 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo /dev/sdp /dev/sdq /dev/sdr /dev/sds /dev/sdt /dev/sdu /dev/sdv /dev/sdw /dev/sdx /dev/sdy -f
  19. sudo zpool status
    pool: zfs0
     state: ONLINE
     scan: none requested
     config:
    
    NAME        STATE     READ WRITE CKSUM
     zfs0      ONLINE       0     0     0
     raidz3-0  ONLINE       0     0     0
     sdb     ONLINE       0     0     0
     sdc     ONLINE       0     0     0
     sdd     ONLINE       0     0     0
     sde     ONLINE       0     0     0
     sdf     ONLINE       0     0     0
     sdg     ONLINE       0     0     0
     sdh     ONLINE       0     0     0
     sdi     ONLINE       0     0     0
     sdj     ONLINE       0     0     0
     sdk     ONLINE       0     0     0
     sdl     ONLINE       0     0     0
     sdm     ONLINE       0     0     0
     sdn     ONLINE       0     0     0
     sdo     ONLINE       0     0     0
     sdp     ONLINE       0     0     0
     sdq     ONLINE       0     0     0
     sdr     ONLINE       0     0     0
     sds     ONLINE       0     0     0
     sdt     ONLINE       0     0     0
     sdu     ONLINE       0     0     0
     sdv     ONLINE       0     0     0
     sdw     ONLINE       0     0     0
     sdx     ONLINE       0     0     0
     sdy     ONLINE       0     0     0
    
    errors: No known data errors
  20. sudo zfs list
    NAME   USED  AVAIL  REFER  MOUNTPOINT
     zfs0   297K  16.7T  89.8K  /zfs0
  21. df -h
    Filesystem                         Size  Used Avail Use% Mounted on
     /dev/mapper/DPMHOST--ZFS--vg-root   15G  1.5G   13G  11% /
     udev                               2.0G  4.0K  2.0G   1% /dev
     tmpfs                              790M  668K  789M   1% /run
     none                               5.0M     0  5.0M   0% /run/lock
     none                               2.0G     0  2.0G   0% /run/shm
     /dev/sda1                          236M   32M  192M  14% /boot
     zfs0                                17T  128K   17T   1% /zfs0
  22. Configure Samba:
    sudo vim /etc/samba/smb.conf
  23. sudo zfs set sharesmb=on zfs0/backuptest1
  24. sudo chmod 0777 /zfs0/backuptest1
  25. sudo service smbd restart
  26. sudo zfs get sharesmb,sharenfs
    NAME              PROPERTY  VALUE     SOURCE
    zfs0              sharesmb  off       default
    zfs0              sharenfs  off       default
    zfs0/backuptest1  sharesmb  on        local
    zfs0/backuptest1  sharenfs  off       default
  27. I set compression as LZ4, which does wonders for raw SQL files:
    sudo zfs set compression=lz4 zfs0/backuptest1

Enjoy!

Advertisements

Some awesome documentaries

My ex boyfriend really enjoyed watching documentaries (and he even got me to pronounce the word correctly!) so I’m going to start a new “documentary” section for my blog, so maybe I’ll spend more time watching them and sharing them here. A good place I’ve found to look for some good ones: https://www.reddit.com/r/Documentaries/

Me watching the North Korean documentary
Me watching the North Korean documentary

 

 

 

The very thought provoking things I’ve watched lately:

Vandana Shiva: Food, Ethics, and Sustainability

(start on 24:50. from here: http://www.yesmagazine.org/about/vandana-shiva-speaks-at-seattle-town-hall)

This Is What Democracy Looks Like (Seattle 1999 WTO)

All Wars Are Bankers’ Wars

North Korea Exposes the Western Propaganda

New PGP key for email

Heya

Used Seahorse today to manage my new PGP key for my email address yawnbox@gmail.com. My really good password was generated using passwordsgenerator.net and securely stored.

A531D73D

Other (older) keys are not valid (EBF3C1FD and 2FB08023), so please do not use them.

I followed RiseUp’s Howto Setup OpenPGP Keys. I then made a backup of my private key in a Tomb.

Exported to pgp.mit.edu for ease of use:

gpg --list-keys
gpg --export -a A531D73D > mypubkey.asc
gpg --send-keys --keyserver pgp.mit.edu A531D73D

Verifying output:

gpg: sending key A531D73D to hkp server pgp.mit.edu

Happy encrypting!