from Hacker News

Highly Available Block Storage

by dineshp2 on 7/13/16, 1:16 PM with 140 comments

  • by wiremine on 7/13/16, 1:58 PM

    Spun one up and ran some quick numbers on a 100GB volume:

    root@ubuntu-1gb-nyc1-01:~# time dd if=/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 of=test.dat bs=1024 count=10000000 10000000+0 records in

    10000000+0 records out

    10240000000 bytes (10 GB) copied, 58.0655 s, 176 MB/s

    real 0m58.248s

    user 0m2.608s

    sys 0m41.604s

    Some quick observations:

    * Easy to add one when creating a droplet; by default they let you create volumes with these sizes: 100GB, 250GB, 500GB, 1000GB, 1.95TB; it's also really easy to create your own size.

    * You can resize in any increments; took about 4 seconds to go from 100GB to 110GB with no downtime; you obviously need to resize/manage the mounted volume yourself.

    * [Edit 1] Deleting the droplet does NOT destroy the volume. Worth keeping in mind when you spin them up/down.

    * [Edit 2] Remounting an existing volume to a new droplet was quick and painless.

  • by bjacobel on 7/13/16, 2:18 PM

    Reminder that just a few weeks ago DigitalOcean rolled over on one of their customers and took down 38,000 websites after receiving a claim of infringement from the NRA against a parody site hosted on surge.sh:

    http://motherboard.vice.com/read/nra-complaint-takes-down-38...

  • by dastbe on 7/13/16, 1:38 PM

    Don't be confused: the article makes the mistake of comparing DOs new block storage service with other companies object stores. EBS is the competitor to this, not S3. Same for gce persistent disks and azure drives.

    Unfortunately this means the pricing comparison is just wrong.

  • by mwcampbell on 7/13/16, 2:54 PM

    I think this might be a mistake. Ever since Joyent's commentary on one of the big Amazon EBS failures in 2011 [1] [2] [3], I've been suspicious of all network-attached block storage. Then again, I haven't heard of any big EBS failures recently; I wonder what changed.

    [1]: https://www.joyent.com/blog/on-cascading-failures-and-amazon...

    [2]: https://www.joyent.com/blog/magical-block-store-when-abstrac...

    [3]: https://www.joyent.com/blog/network-storage-in-the-cloud-del...

  • by Mister_Snuggles on 7/13/16, 2:26 PM

    This is EXACTLY the thing that I need for one of my droplets! I love how there is nothing "special" about it - it's just a disk that you can attach to a droplet. I'm sure that under the hood there's some kind of magic going on, but it looks like it's nicely abstracted away. This is what I hoped block storage would turn out to be - here's a block device, use it like one.

    As soon as this rolls out to the region I've got that droplet in, I'm going to pull the trigger on it. I might even spend the effort to migrate my droplet to a supported region just to get this.

  • by 3pt14159 on 7/13/16, 2:33 PM

    I have been asking for non-SSD on DO for a long time now. My heart jumped when I saw the HN title, only to be dashed on the rocks.

    What are us data nerds supposed to do? We want to take 10 terabytes, run a batch process on it, keep the 20TB, then continue with about 5GB of working data until the next month's terabyte comes in, then we want to batch through the 21TB. Right now the price slider doesn't even go up to 21TB, and clicking on the "need more storage button" doesn't go anywhere, but I'm assuming it would be $2100 / month which is more than 3x as expensive than vanilla S3.

  • by misframer on 7/13/16, 2:27 PM

    It's the same price as AWS's General Purpose SSD EBS volumes.

    https://aws.amazon.com/ebs/pricing/

  • by andybak on 7/13/16, 1:52 PM

    This helps me with a nicer deployment setup. I was always keen on 'rebuild from scratch' rather than 'update stuff and hope you're idempotent and have captured all changes' but transitory data was always the problem. Now I can start building a new updated droplet and the only downtime will be that needed to detach and reattach the block storage containing the db etc.

    Anyone see a flaw in this? (I know there are other ways to achieve similar benefits - my files could be on S3 and the database could be a separate droplet etc but these introduced various drawbacks and added complexity)

  • by skrowl on 7/13/16, 1:31 PM

    I like their straight forward pricing. $0.10 USD per GB per month. No IOPS limits.

    That said, how do you prevent a rogue droplet from going crazy and hogging up all of the SSD I/O?

  • by happyslobro on 7/13/16, 1:54 PM

    Setup: find it's name, format and mount it, as if you were adding an SSD to a desktop.

    https://www.digitalocean.com/community/tutorials/how-to-use-...

  • by koolba on 7/13/16, 1:44 PM

    This has been a much requested feature and I'm sure it will be very popular. I'm still reminded of this quote though:

    "He was a bold man that first ran a production database on a brand new block storage service!"

  • by johnwheeler on 7/13/16, 2:15 PM

    I love how DO focuses on what matters the most: Inexpensive VMs and scalable block storage.

    If I had to pick two, those would be them!

  • by aibottle on 7/13/16, 11:24 PM

    Thank god! Highly Available Block Storage. From Digital Ocean. Great! Now I can finally store all the 300mb/s streaming in on my server. Oh wait. I cannot, because DO cancelled the service again. Bummer.

  • by cgag on 7/13/16, 5:36 PM

    Sweet. This makes digitalocean much more appealing as a potential substrate for a kubernetes cluster.

  • by Mister_Snuggles on 7/13/16, 2:38 PM

    I can't wait for this to roll out to more regions.

    This is EXACTLY the thing I need for some stuff I'm working on!

  • by ozy23378 on 7/13/16, 5:53 PM

    Going to perform some very basic DD i/o benchmarks using: https://haydenjames.io/web-host-doesnt-want-read-benchmark-v...

    Will post results.

  • by scurvy on 7/13/16, 2:07 PM

    What's the backend? Ceph?

  • by mrmondo on 7/17/16, 4:06 PM

    How is it taking huge cloud providers so long to catch up with things we do self hosted every day? It obviously has to be well engineered, yet it's relatively simple. Woefully poor performance too.

  • by simos on 7/13/16, 2:01 PM

    Some early benchmarks about the new block storage, https://simos.info/blog/trying-out-lxd-containers-on-ubuntu-...

    I did not get good speeds and I am wondering why that may be...

  • by drtse4 on 7/13/16, 1:47 PM

    A bit pricey for the long term but great if you just need to add some disk space to your vm and don't need the other improvements more expensive vms give you.

    I use DO mostly to compile stuff on Linux when i don't have access to a physical server, and storage size is always a problem.

  • by daveguy on 7/13/16, 1:36 PM

    Edit3: Remountable/movable flexible storage for DO instances is what this gives you and it's kind of pricey. The comparison to B2 is not valid. Leaving the original mess for posterity.

    ---

    TWENTY times (edit) the price as B2 from Backblaze ($.10 vs $.005 per GB per month). It is one of the more expensive ones. But that gets you two things:

    * (moot See edit 2) SSD! (significant iops improvement)

    * (moot See edit 2) No transactional costs! (not sure if just between digital ocean instances, but they say none)

    Improved performance and no transaction costs MAY (edit) be worth it for some applications.

    Edit1: made it an order of magnitude cheaper in my head after looking at backblaze. It is no where close to the same price. Thank you for the catch, scq!

    Edit2: and I'm just all kinds of off on this. Block not object storage. Essentially storage you can mount and move between digitalocean instances. That makes no transaction costs moot. You still have to get data out of the instance.

    Thank you all for quickly catching how backwards this post was! I need coffee.

  • by happyslobro on 7/13/16, 1:56 PM

    I don't suppose DO or anyone else is already working on an Ansible addon for this?

  • by lamarkia on 7/13/16, 1:34 PM

    I benchmarked the new block storage and it is not that faster than the virtio disk.

  • by fweespeech on 7/13/16, 2:32 PM

    $.10/GB block storage is too expensive.

    https://www.online.net/en/dedicated-server/rpn-san

    There are several places you can get ~5 TB for +/-10% of the 1TB price at DO.

    DO is offering a SAN at Object Store prices. :/