Our ebay development S2D cluster

Share this post:

During this summer i decided i wanted to test out Storage Spaces Direct. TP5 was out and i was quite eager to test it out. Now it’s been upgraded to RTM with cluster rolling upgrade. Rember to run Update-ClusterFunctionalLevel after.

So i look arround on ebay for some servers and other items i needed to buy. I ended up with the list under all in 4x

  • HP DL380 G6 16 bay 128GB mem, 2x4core Intel CPU HP P420
  • HP H220
  • MellanoX ConnectX3 MCX312A-XCBT
  • Intel 750 NVME PCIe
  • 2x Kingston SSDNow V310 for caching(Replacing with Samsung SM863)
  • 6x WD Red NAS 1tb 2.5″
  • Dell Force Ten S4810P (Already had)

The price for all this without the switches and cables was about 90 000 nok, about 10 000 USD.

I cabled the HP H220 to controll the 2nd 8 bay HDD cage so give the HBA mode for the SSD and HDD. And 2 HP drives for OS in raid 0

I followed the guides from

For S2D setup


For RDMA setup on S4810P


I setup managment over the 4 1GB netcards on the server. I will just post the powershell commands directly.

New-NetLbfoTeam -name Team1 -TeamMember NIC1,NIC2,NIC3,NIC4 -TeamingMode SwitchIndependent -LoadBalancingAlgorithm HyperVPort
New-VMSwitch “VSwitch” -MinimumBandwidthMode Weight -NetAdapterName “Team1" -AllowManagementOS 0
Set-VMSwitch “VSwitch” -DefaultFlowMinimumBandwidthWeight 50
Add-VMNetworkAdapter -ManagementOS -Name “Management” -SwitchName “VSwitch”
Set-VMNetworkAdapterVlan -ManagementOS -VMNetworkAdapterName “Management” -Access -VlanId "VLANID"



For the config on the server for RDMA and so on use these powershell commands

#Install DCB on the hosts
Install-WindowsFeature Data-Center-Bridging
#Mellanox/Windows RoCE drivers don\'t support DCBx (yet?), disable it.
Set-NetQosDcbxSetting -Willing $False
#Make sure RDMA is enable on the NIC (should be by default)
Enable-NetAdapterRdma –Name 10GNIC1
Enable-NetAdapterRdma –Name 10GNIC2

#SMB Direct traffic to port 445 is tagged with priority 4
New-NetQosPolicy "SMBDIRECT" -netDirectPortMatchCondition 445 -PriorityValue8021Action 4
#Anything else goes into the "default" bucket with priority tag 1
New-NetQosPolicy "DEFAULT" -default -PriorityValue8021Action 1
#Enable PFC (lossless) on the priority of the SMB Direct traffic.
Enable-NetQosFlowControl -Priority 4
#Disable PFC on the other traffic (TCP/IP, we don\'t need that to be lossless)
Disable-NetQosFlowControl 0,1,2,3,5,6,7
#Enable QoS on the RDMA interface
Enable-NetAdapterQos -InterfaceAlias 10GNIC1
Enable-NetAdapterQos -InterfaceAlias 10GNIC2
#Set the minimum bandwidth for SMB Direct traffic to 90% (ETS, optional)
#No need to do this for the other priorities as all those not configured
#explicitly goes in to default with the remaining bandwith.
New-NetQoSTrafficClass "SMBDirect" -Priority 4 -Bandwidth 90 -Algorithm ETS
#Configure VMSwitch and setup VMNetworkAdapters
New-VMSwitch -Name SETswitch -NetAdapterName "10GNIC1","10GNIC2" -EnableEmbeddedTeaming $true
Add-VMNetworkAdapter -SwitchName SETswitch -Name Cluster -managementOS
Add-VMNetworkAdapter -SwitchName SETswitch -Name LiveMigration -managementOS
Add-VMNetworkAdapter -SwitchName SETswitch -Name Storage -managementOS
Set-VMNetworkAdapterVlan -VMNetworkAdapterName "Cluster" -VlanId 5 -Access -ManagementOS
Set-VMNetworkAdapterVlan -VMNetworkAdapterName "LiveMigration" -VlanId 6 -Access -ManagementOS
Set-VMNetworkAdapterVlan -VMNetworkAdapterName "Storage" -VlanId 99 -Access -ManagementOS
Enable-NetAdapterRDMA "vEthernet (Cluster)","vEthernet (LiveMigration)","vEthernet (Storage)"




Test-Cluster -Node "SRV01", "SRV02", "SRV03", "SRV04" -Include "Storage Spaces Direct",Inventory,Network,"System Configuration"

New-Cluster -Name HyperV-S2D -Node "SRV01", "SRV02", "SRV03", "SRV04" -NoStorage -StaticAddress x.x.x.x


Get-StoragePool S2D* | FT FriendlyName, FaultDomainAwarenessDefault, OperationalStatus, HealthStatus -autosize

Get-StorageTier | FT FriendlyName, ResiliencySettingName, MediaType, PhysicalDiskRedundancy -autosize


Check to see if HDD has been set as capacity tier

If not run this command

Set-StorageTier -FriendlyName Capacity -MediaType HDD


Time to set volume

New-Volume -StoragePoolFriendlyName S2D* -FriendlyName MRV -FileSystem CSVFS_REFS -StorageTierFriendlyName Performance, Capacity -StorageTierSizes 1800GB, 11000GB


enable CSV cache

(Get-Cluster). BlockCacheSize = 1024



This is how it will look in failover cluster manager


Subscribe to our newsletter

Get the inside scoop! Sign up for our newsletter to stay in the know with all the latest news and updates.

Don’t forget to share this post!

6 thoughts on “Our ebay development S2D cluster”

    1. Pretty good. Have not run a vmfleet test on it. But have run 8 vm’s with diskpeed. And gotten about 550k iops. Så not to dissapointed. Old san maxed out at 2000 iops 😉

    1. Pretty good. Have not run a vmfleet test on it. But have run 8 vm’s with diskpeed. And gotten about 550k iops. Så not to dissapointed. Old san maxed out at 2000 iops 😉

Leave a Comment

Scroll to Top