Accelerate Storage Spaces with SSDs in Windows 10 Storage Pool tiers

Create Storage Spaces in Windows 10

Windows Server O/S contains Storage Spaces support for Server Spaces tiered storage. You can front slower spinning disks with smaller faster SSDs. Windows 10 has a Storage Spaces GUI Control Panel that does not include the tiered storage GUI. This means Powershell must be used for all configuration.
Physical Disks
https://github.com/freemansoft/win10-storage-spaces contains scripts that create tiered storage pools that integrate SSDs as caching drives and HDDs as storage drives. They assume you have at least one SSD and one HDD.
  • The scripts automatically find all raw drives and add them to the pool.
  • Some HDDs have their types incorrectly identified. The script can coerce them to be MediaType:HDD
  • The entire virtual drive is added to the system as a single large volume
  • You need at least
    • 1 SSD and 1 HDD to run cached storage / Simple resiliency
    • 2 SSD and 2 HDD to run cached storage / Mirror resiliency /
    • 1 SSD and 2 HDD to run cached storage / Simple resiliency / striped storage (sum of HDD space)
Simple Mirrored Mirrored and Striped

Simple vs Mirror

The "Mirror" resiliency level attempts to mirror both SSD and HDD tiers so you would need 4 drives run mirror, to mirror both tiers

Scripts

new-storage-space.ps1

Creates a tiered storage pool and allocates all the disk space to a single drive
  • You can change the drive letter and label by editing the variables at the top.
  • the script can auto size the drive and cache. That didn't work for me so the script supports manual sizing.

remove-storage-space

Removes the virtual drive, the storage tiers and then the storage pool.
  • All drives are returned the Primordial pool

Sample configuration

new-storage-space.ps created a Virtual drive from 3 physical drives
Physical DrivesStorage Space Virtual Drive
two 2TB HDDsingle 3.6TB data volume striped across my two HDD
one 200GB SSD.with a 200GB read/write cache
Simple resiliency with striped drives

Win 10 Pro Storage Spaces Control Panel

The control panel does not display or manipulate tiers

Video


Meaningless Benchmark

All Storage Pool drives connected to 3Gb/s SATA. The write-back cache is not used with sequential writes over 256KB
[Read]                           *Single 2TB no cache*           *Two 2TB mirrored with 200GB cache*
Sequential 1MiB (Q=  8, T= 1):   160.497 MB/s [    153.1 IOPS]   282.983 MB/s [    269.9 IOPS]
Sequential 1MiB (Q=  1, T= 1):   156.766 MB/s [    149.5 IOPS]   254.605 MB/s [    242.8 IOPS]
    Random 4KiB (Q= 32, T=16):     1.748 MB/s [    426.8 IOPS]   175.272 MB/s [  42791.0 IOPS]
    Random 4KiB (Q=  1, T= 1):     0.527 MB/s [    128.7 IOPS]    21.189 MB/s [   5173.1 IOPS]

[Write]                          *Single 2TB no cache*           *Two 2TB mirrored with 200GB cache*
Sequential 1MiB (Q=  8, T= 1):   153.896 MB/s [    146.8 IOPS]   226.825 MB/s [    216.3 IOPS]
Sequential 1MiB (Q=  1, T= 1):   154.147 MB/s [    147.0 IOPS]   230.149 MB/s [    219.5 IOPS]
    Random 4KiB (Q= 32, T=16):     2.033 MB/s [    496.3 IOPS]   149.000 MB/s [  36377.0 IOPS]
    Random 4KiB (Q=  1, T= 1):     1.706 MB/s [    416.5 IOPS]    38.790 MB/s [   9470.2 IOPS]

Credits


Created 4/2020

Comments

  1. Will (Militant Liberal)September 1, 2020 at 9:14 PM

    The script is great - I'm having problems with it, however.

    If I change the $DriveTierResiliency = "Simple" to $DriveTierResiliency = "Parity", the script fails at line 83 and won't initialize the disk. What am I doing wrong?

    I have a large drive set-up - 4 SSDs, 12 HDDs. Please help!

    ReplyDelete
  2. Will (Militant Liberal)September 1, 2020 at 9:25 PM

    I've been planning on using this script to replace my NAS drive - I have 4 SSDs and 12 HDDs.

    When I change the script line 15 of the script from:

    $DriveTierResiliency = "Simple"

    to

    $DriveTierResiliency = "Parity"

    The script fails at line 83, saying Initilaize-Disk : Failed.

    The details below it say:
    + CategoryInfo : NotSpecified: (StorageWMI:Root/Microsoft/Windows/Storage/MSFT_Disk) [Initalize-Disk], CimException
    + FullyQualifiedErrorID:StorageWMI 4,Initialize-Disk

    Any thoughts?

    ReplyDelete
  3. PawlOctober 4, 2020 at 2:29 AM

    Your https://github.com/freemansoft/win10-storage-spaces link goes to blogger.com

    ReplyDelete
  4. DeloziedDecember 6, 2020 at 10:19 PM

    Trying this later this week. Do you know if you can Stripe 2 HDD (simple) and use 2 SSD for the cache. I want the larger size, have 2 SSDs and a good backup strategy to NAS+cloud+external drive.

    ReplyDelete
  5. Roman SheinJanuary 11, 2021 at 3:20 AM

    With all respect the tiering is not the same with the caching.
    Clinet Win10 has no caching in storage spaces, only tiering is available.

    ReplyDelete

Post a Comment

Popular posts from this blog

Installing the RNDIS driver on Windows 11 to use USB Raspberry Pi as network attached

Image
I do a lot of my development and configuration via ssh into my Raspberry Pi Zero over the RNDIS connection. Some models of the Raspberry PIs can be configured with  gadget drivers  that let the Raspberry pi emulate different devices when plugged into computers via USB. My favorite gadget  is the network profile that makes a Raspberry Pi look like an RNDIS-attached network device.  All types of network services travel over an RNDIS device without knowing it is a USB hardware connection. A Raspberry Pi shows up as a Remote NDIS (RNDIS) device when you plug the Pi into a PC or Mac via a USB cable. The gadget  in the Windows Device Manager picture shows this RNDIS Gadget connectivity between a Windows machine and a Raspberry Pi. The Problem Windows 11 and Windows 10 no longer auto-installs the RNDIS driver that makes magic happen. Windows recognizes that the Raspberry Pi is some type of generic USB COM device. Manually running W indows Update  or  Update Driver  does  not  install the RNDI
Read more

Understanding your WSL2 RAM and swap - Changing the default 50%-25%

Image
The Windows Subsystem for Linux operates as a virtual machine that can dynamically grow the amount of RAM to a maximum set at startup time.  Microsoft sets a default maximum RAM available to 50% of the physical memory and a swap-space that is 1/4 of the maximum WSL RAM.  You can scale those numbers up or down to allocate more or less RAM to the Linux instance. The first drawing shows the default WSL memory and swap space sizing. The images below show a developer machine that is running a dev environment in WSL2 and Docker Desktop.  Docker Desktop has two of its own WSL modules that need to be accounted for.  You can see that the memory would actually be oversubscribed, 3 x 50% if every VM used its maximum memory.  The actual amount of memory used is significantly smaller allowing every piece to fit. Click to Enlarge The second drawing shows the memory allocation on my 64GB laptop. WSL Linux defaults to a maximum RAM size of 5
Read more

Almost PaaS Document Parsing with Tika and AWS Elastic Beanstalk

Image
The Apache Tika project provides a  library capable of parsing and extracting data and meta data from over 1000 file types.  Tika is available as a single jar file that can be included inside applications or as a deployable jar file that runs Tika as a standalone service. This blog describes deploying the Tika jar as an auto-scale service in Amazon AWS Elastic Beanstalk.  I selected Elastic Beanstalk because it supports jar based deployments without any real Infrastructure configuration. Elastic Beanstalk auto-scale should take care of scaling up and down for for the number of requests you get. Tika parses documents and extracts their text completely in memory. Tika was deployed for this blog using EC2 t2.micro instances available in the AWS free tier. t2.micro VMs are 1GB which means that you are restricted in document complexity and size. You would size your instances appropriately for your largest documents.   Preconditions An AWS account. AWS access id and secret key.
Read more