Windows Server 2012, Hyper-V, Ubuntu+ZFS VM for Backups

I set this up only to test ZFS with deduplication as an alternative to using a standard hardware RAID configuration for storing SQL backups.

Hardware used:

  • Dell R720xd
  • Dell Perc H800 RAID controller
  • Dell PowerVault MD1220
  1. Configure RAID on the Dell R720xd / Perc H800 controller. All 24 disks (2.5″ 900GB SAS in my case) as an independent RAID-0 logical volume since the H800 (as far as I can tell) doesn’t support an easy JBOD option.
    Untitled3
    Untitled1
    Untitled2
  2. In Windows Server 2012 Computer Management, I set each of the 24 volumes to GPT partition table when asked, and then set all 24 volumes “offline”. Doing so allows Hyper-V to access each disk directly, so that Windows cannot.
    Untitled5
  3. Install the Hyper-V role and rebooted.
  4. I went into the VMs settings. First I added a SCSI controller, since IDE Controllers are limited to 2 devices, while SCSI can support up to 64 devices. Then I added the 24 volumes as “physical hard disks”, matching the SCSI location number 0 – 23 to the 24 volume target number 0 – 23.
    Untitled4
  5. Also, make sure to install a virtual switch and configure your network interface for the Ubuntu VM.
  6. Downloaded Ubuntu Server 12.04 x64, installed Ubuntu with OpenSSH and Samba features. Started the Ubuntu VM.
  7. Verify Ubuntu can see the 24 volumes:
    sudo lshw -C disk

    Untitled6

  8. sudo apt-get update
  9. sudo apt-get install python-software-properties software-properties-common -y
  10. sudo add-apt-repository ppa:zfs-native/stable -y
  11. sudo apt-get update && sudo apt-get dist-upgrade -y
  12. sudo apt-get install ubuntu-zfs -y
  13. dmesg | grep ZFS
    [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-15-generic root=/dev/mapper/DPMHOST--ZFS--vg-root ro
     [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-15-generic root=/dev/mapper/DPMHOST--ZFS--vg-root ro
     [    8.529432] Adding 4190204k swap on /dev/mapper/DPMHOST--ZFS--vg-swap_1.  Priority:-1 extents:1 across:4190204k SSFS
  14. sudo vim /etc/modules
  15. Add these lines:
    spl
    zavl
    znvpair
    zunicode
    zcommon
    zfs
  16. Incorporate new modules into the boot files:
    sudo update-initramfs -u
  17. sudo reboot
  18. I created a ZFS pool called “zfs0” using raidz3 which can lose up to 3 disks using all 24 volumes:
    sudo zpool create zfs0 raidz3 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo /dev/sdp /dev/sdq /dev/sdr /dev/sds /dev/sdt /dev/sdu /dev/sdv /dev/sdw /dev/sdx /dev/sdy -f
  19. sudo zpool status
    pool: zfs0
     state: ONLINE
     scan: none requested
     config:
    
    NAME        STATE     READ WRITE CKSUM
     zfs0      ONLINE       0     0     0
     raidz3-0  ONLINE       0     0     0
     sdb     ONLINE       0     0     0
     sdc     ONLINE       0     0     0
     sdd     ONLINE       0     0     0
     sde     ONLINE       0     0     0
     sdf     ONLINE       0     0     0
     sdg     ONLINE       0     0     0
     sdh     ONLINE       0     0     0
     sdi     ONLINE       0     0     0
     sdj     ONLINE       0     0     0
     sdk     ONLINE       0     0     0
     sdl     ONLINE       0     0     0
     sdm     ONLINE       0     0     0
     sdn     ONLINE       0     0     0
     sdo     ONLINE       0     0     0
     sdp     ONLINE       0     0     0
     sdq     ONLINE       0     0     0
     sdr     ONLINE       0     0     0
     sds     ONLINE       0     0     0
     sdt     ONLINE       0     0     0
     sdu     ONLINE       0     0     0
     sdv     ONLINE       0     0     0
     sdw     ONLINE       0     0     0
     sdx     ONLINE       0     0     0
     sdy     ONLINE       0     0     0
    
    errors: No known data errors
  20. sudo zfs list
    NAME   USED  AVAIL  REFER  MOUNTPOINT
     zfs0   297K  16.7T  89.8K  /zfs0
  21. df -h
    Filesystem                         Size  Used Avail Use% Mounted on
     /dev/mapper/DPMHOST--ZFS--vg-root   15G  1.5G   13G  11% /
     udev                               2.0G  4.0K  2.0G   1% /dev
     tmpfs                              790M  668K  789M   1% /run
     none                               5.0M     0  5.0M   0% /run/lock
     none                               2.0G     0  2.0G   0% /run/shm
     /dev/sda1                          236M   32M  192M  14% /boot
     zfs0                                17T  128K   17T   1% /zfs0
  22. Configure Samba:
    sudo vim /etc/samba/smb.conf
  23. sudo zfs set sharesmb=on zfs0/backuptest1
  24. sudo chmod 0777 /zfs0/backuptest1
  25. sudo service smbd restart
  26. sudo zfs get sharesmb,sharenfs
    NAME              PROPERTY  VALUE     SOURCE
    zfs0              sharesmb  off       default
    zfs0              sharenfs  off       default
    zfs0/backuptest1  sharesmb  on        local
    zfs0/backuptest1  sharenfs  off       default
  27. I set compression as LZ4, which does wonders for raw SQL files:
    sudo zfs set compression=lz4 zfs0/backuptest1

Enjoy!

Advertisements

Some awesome documentaries

My ex boyfriend really enjoyed watching documentaries (and he even got me to pronounce the word correctly!) so I’m going to start a new “documentary” section for my blog, so maybe I’ll spend more time watching them and sharing them here. A good place I’ve found to look for some good ones: https://www.reddit.com/r/Documentaries/

Me watching the North Korean documentary
Me watching the North Korean documentary

 

 

 

The very thought provoking things I’ve watched lately:

Vandana Shiva: Food, Ethics, and Sustainability

(start on 24:50. from here: http://www.yesmagazine.org/about/vandana-shiva-speaks-at-seattle-town-hall)

This Is What Democracy Looks Like (Seattle 1999 WTO)

All Wars Are Bankers’ Wars

North Korea Exposes the Western Propaganda

New PGP key for email

Heya

Used Seahorse today to manage my new PGP key for my email address yawnbox@gmail.com. My really good password was generated using passwordsgenerator.net and securely stored.

A531D73D

Other (older) keys are not valid (EBF3C1FD and 2FB08023), so please do not use them.

I followed RiseUp’s Howto Setup OpenPGP Keys. I then made a backup of my private key in a Tomb.

Exported to pgp.mit.edu for ease of use:

gpg --list-keys
gpg --export -a A531D73D > mypubkey.asc
gpg --send-keys --keyserver pgp.mit.edu A531D73D

Verifying output:

gpg: sending key A531D73D to hkp server pgp.mit.edu

Happy encrypting!

looking at my heart

I have not blogged about personal stuff in a long time. Sometimes the only thing that will listen in the way that I want to be heard is with a pen and a piece of paper.

It is clear to me that I am an introvert. When it comes to personal matters, I like being by myself and working out my own problems. With that comes cut-throat prioritization and intense moral struggle.

The exception to my independence is when I fall in love with another person. The act of falling in love happens rarely, because it takes a long time for me to identify someone with a mix of characteristics and mannerisms that I thoroughly and thoughtfully enjoy.

When I do fall in love, I fall quickly, and I fall hard.

When I find this person, I think that I know what to do. I actively demonstrate care, humility, and respect. I become super attentive and reactive. I do all of these things because it is what I want:

  • It is how I want to be mutually treated by this person.
  • Because it is an outlet for positive emotion, one that I understand, which makes me happy.
  • And because I need to create feedback loops to–in my opinion, help–verify the integrity of our feelings.

Sometimes one or more of these things scares people. It makes them question their own emotional depth, which creates uncomfortable rifts. I then get frustrated because these people do not seem to be patient with themselves or with me–or is it truly irreversible?

Cities: adopt these privacy laws

To follow up on a my previous post about enacting local initiatives for the people’s right to privacy:

  • A publicly-accessible warrant must be provided to monitor or capture any private voice communication or digital data, at rest or in motion.
  • Only the entities that privately sign digital certificates for the purpose of encrypting voice communication or digital data, at rest or in motion, can decrypt said voice communication or digital data.
  • Only the original creator of a private certificate may own and use, in such a way, that allows said entity to view or record decrypted voice communication or digital data.
  • It is illegal for any entity to attempt to break or subvert any voice or data encryption mechanism.

I foresee some business impacts to these, so some of them probably need to change. Discuss!

Ideas to support the Tor Project: Wikipedia IdeaLab proposal

Special thanks to my open-access comrade-in-arms Lane Rasberry.

Lane emailed me this morning asking for my input on a current proposal that’s on Jimmy Wales very own Wikipedia talk page.

After CC’ing Runa Sandvik from the Tor Project to verify the factuality of my feedback for the Wikipedia community, I posted my comments.

The ongoing issue, that Jacob Appelbaum repeatedly vocalizes, is that Tor users, Jacob included, is not able to protect his identity and contribute to the knowledge base that exists on Wikipedia.

Political activists and dissidents create a critical feedback loop into the controversial dialogue that is only made possible through the Internet and social media. Not only are these people self-empowering, they are the ones most likely to seek out the truth.

From Lane:

If you would be willing to write a brief set of proposals about what Wikipedia should do with Tor, then [Lane] would format those with you in the IdeaLab. This is a space where ideas are stored on Wikipedia so that they would always be found if anyone ever wanted them. I think it would be a good idea just to establish the conversation.

https://meta.wikimedia.org/wiki/Grants:IdeaLab

[If] it is of interest to you, I would help you start a proposal, format it properly, publicize it, and if you know anyone in the Tor community that might want to make a grant proposal for funding to establish and document the relationship between Tor and Wikipedia, then I might be able to advise on how to do that also.

This conversation is happening now live and it does have Jimbo Wales’ attention. It would be awesome to get input from established Tor supporters.

If you would like to create a proposal and have the support of a Wikipedia veteran, please contact Lane directly, and ask for other peoples input! I’m also extremely interested in supporting, I just don’t know what an ideal proposal would look like, and I don’t want to speak on behalf of Tor Project.

Thank you!

Developing an Open Educational Resource on Encryption

Encryption works. Properly implemented strong crypto systems are one of the few things that you can rely on. Unfortunately, endpoint security is so terrifically weak that NSA can frequently find ways around it.

— Edward Snowden, answering questions live on the Guardian’s website

Society needs an educational resource, covering the complex topics involved with information encryption, that is modular, openly accessible, and freely remixable. This is my proposal to create such a resource.

Open Educational Resources (OER) are freely accessible, openly licensed documents and media that are useful for teaching, learning, educational, assessment and research purposes. The development and promotion of open educational resources is often motivated by a desire to curb the commodification of knowledge[1] and provide an alternate or enhanced educational paradigm.

Utilizing Creative Commons licensing, an OER can be created on oercommons.org, where it will be maintained by a single authority, yet anyone in the world will be able to adapt and create their own work from ours. Oercommons.org provides a long-term support platform for maintaining these resources.

I started publicly asking for help in June of 2013–and I received a very warm welcome. You don’t have to look far to see why.

2013-06-24

August 2013:

2013-08-23 2013-08-23-2

October 2013: KEYNOTE: Journalism in the Age of Surveillance, Threat Modeling: Determining Digital Security for You, [For Journalism] Keeping Under the Security Radar, Improving Your Digital Hygiene

December 2013: United We Stand — and Encrypt by Josh Sterns2013-12-21

December 2013: Arab journalists need training for civil unrest and wars — referencing the CPJ’s Journalist Security Guide

January 2014: A Modest Proposal for Encrypting the Work of Activists by Kate Krauss

2014-01-20

It is clear that a diversity of educational resources are needed. While my original proposal was going to be supported by the United States Open Knowledge Foundation, OKFNUS has since back peddled due to lack of support from central-OKF. I am hoping that the many people behind Crypto.is are interested in spearheading the development of this OER. If they are not, and no other organization is, I will shortly be registering my own domain name to create a project launch page.

The initial launch of the OER can be created using Micah Lee‘s work, of the Freedom of the Press Foundation, Encryption Works: How to Protect Your Privacy (And Your Sources) in the Age of NSA Surveillance. Micah and the Freedom of the Press Foundation graciously licensed this work as CC-BY, allowing us, and even Wikipedia to reuse the work with attribution. I am hoping that Micah, himself, will want to be included in this project.

The target audience, initially, will be journalists, whistle blowers, activists, and dissidents. While these groups are the extreme, their example proves useful for the rest of society.

Please comment on this post, or tweet me, or email me your feedback.