The SSD Review The Worlds Dedicated SSD Education and Review Resource | 
REPORT SUMMARY AND FINAL THOUGHTS
NVMe SSDs are becoming very popular, and the ability of pushing 2-3GB/s of throughput and a half million IOPS through a storage medium the size of a stick of gum is amazing. Equally as impressive is the fact that, with this performance, comes a zero footprint on the motherboard. As with our demonstrations today, the only concern with RAIDING three NVMe SSDs is the fact that, depending on your needs, lanes become very precious. As appealing as the thought of having three M.2 NVMe PCIe X4 SSDs running under three graphics cards is, a bit of investigation might be necessary as to chipset or CPU lane travel and route chosen by the manufacturer. The Z170’s 20 lanes dedicated to PCH and 16 lanes dedicated to the CPU still isn’t enough for the true geek at heart.
Just as important is the need to do a bit of reading before you run out and buy an NVMe SSD, or decide upon your build for a new NVMe SSD contained system. There have been more than a few motherboards returned and more than a few Samsung 950 Pro SSDs returned because preparation wasn’t there before purchase. Again…the cart before the horse. Having said that, this is what we do and whenever hesitate to answer site comments, e-mails or Forum threads requesting assistance. All we ask is to do a bit of reading first (Google is your friend) to help us to help you.
It is also amusing how we are always pushing for faster speeds without realizing how far this technology has grown so soon. We can pull off 3GB/s read and write speeds without a heat or space concern today, whereas I remember only a few years ago when a certain company sent their newest PCIe Add in Card capable of 2.5GB/s and valued at somewhere around $1400. Yup, I toasted it within 15 minutes not realizing I needed to throw a few fans on it to provide for “active cooling”. We were all stunned when they got another to us within 24 hours (California to Niagara, Canada) without a complaint. Today we are pushing in excess of 3GB/s from a zero footprint; who would have figured?
Our recommendation for any M.2 SSD setup simply has to go with the ASRock Z170 Extreme 7+ motherboard and three Samsung 950 Pros SSDs. What a sweet combination for a media professional!
Les, I am one of those fool hearty individuals who just bought the ASRock motherboard and two of the 950 drives that I want to mirror and use as a database server for my small business. However, my database requires Windows Server 2012 R2. I haven’t even opened up the boxes yet. Do you think I will be able to configure the raid 1 in bios and then install WS 2012 R2 or is that a crazy idea will not work?
When you are in the the UEFI RST, you can select RAID 0, 1 or recovery, along with several stripe levels…..so yes! Great selections…you have a motherboard that supports NVMe, an OS that supports it and two excellent drives! Should be a piece of cake.
Thanks. That’s good news.
But what about adding a 3rd and 4th 950 drive to do raid 10? The 3rd through the m.2 slot but the 4th by adding a m.2 pcie adapter into one of the slots? Do you think that would work?
No. The BIOS is only set up for creating a RAID with the 3 M.2 slots remapped. It also only enables RAID 0 or 1.
Good to know. Thanks.
So, what support or development needs to happen to eventually be able to boot off of any NVMe drive and RAID more drives? Is it enhancement or development of the bios? Is it new technology added to the Intel chipset platforms? New Raid cards?
It would be a BIOS alteration. You could probably go through the OS and create such a RAID but it would not be bootable.
(1) 4 or more U.2 ports, replacing motherboard SATA ports
(2) a wider DMI link e.g. x16 lanes for adequate upstream bandwidth
(3) x16 expansion slot(s) wired directly to the CPU
(4) a PLX-type switch to support four NVMe devices on one x16 slot
(5) NVMe RAID controllers with x16 edge connectors
(6) (add your wish list here 🙂
p.s. PCIe 4.0 will oscillate at 16 GHz, so a DMI 4.0 link
only increases upstream bandwidth to 8 GB/second.
The DMI link needs to be wider to support upcoming
non-volatile DRAM e.g. Intel Optane, Everspin’s ST-MRAM etc.
x16 PCIe 4.0 lanes @ 2 GB/sec = 32 GB/second.
if you’re doing this on a server, don’t forget that for every M.2 slot you use on z170, you have to deactivate more sata ports. The sata ports need to use the same PCIe lanes to work. You’ll need to check what gets disabled when using each specific M.2 slot in the manual.
In addition to what Les said on that the bios features need to be changed in order to get raid working on more drives, is probably a new chipset, the software-like hardware raid is probably done by the PCH, which on z170 supports a maximum of 3 M.2 slots. any M.2 slots added to the regular PCIe slots will be connected directly to the CPU and will not be hardware raidable with M.2 slots on the PCH.
I wish I could slap one of these bad boys in my iMac retina! Darn. Great write up Les.
Have an iMac as well…feel your pain.
Nice one Les………
Might have been worth while going over the difference between M2 Sata and NVME
again on Page 1 for those that don’t know……………
Wouldn’t mind a $ for everyone who rushes out to buy these and only has M2 Sata slot…haha
Never thought of that…good point.
It should really be nvme vs ahci, since those are the two communication protocols/interfaces (software). (Hint, nvme has lower overhead and has some increase in throughput and a bigger improvement in latency.) You could also talk about hardware interfaces, i.e. SATA (1,2,3) vs PCIe. (Hint, SATA 3, the current fast version is overwhelmed by the speed of flash drives and limits them. PCIe lets them fly.) You might also mention what Intel said about M.2 being limited for bigger drives because of temperature and available power. Certainly their PCIe drives are noticably faster and available at bigger sizes. Less clear (to me;) is how the M.2 -> U.2 interfaces affect this. If power is the limitation, then getting your power via M.2 wouldn’t fix the problem. Certainly, no one is doing larger (bigger than 512MB) M.2 drives. Is it true what Intel says that it just won’t work?
Of course M.2 can be connected via SATA 3.0 and PCIe. PCIe is a lot faster.
Hello guys
I have purchased laptop which has m.2 ssd and it supposed to be connected via pcie x4 but instead it is via ancient ahci..
don’t know why they didn’t put it via pcie
Does that mean , that this nvme works that way I have to split my M.2 SSD and system will put it again together but in RAID ?
My SSD is TOSHIBA THNSNJ128G8NU
Your SSD will work in there as it is a SATA 3 SSD. With respect to the connection, it is still most likely PCIe x4 which means it can house a much faster Samsung XP941 pr Samsung SM951 (both AHCI) but it most likely wont yet be able to work with the 950 pro as it is NVMe…
aah so it is not working because mine m.2 Toshiba does not support NVMe right ?
Only Samsung 950 supports NVMe ?
EDIT : just found ; The Samsung MZVPV supports NVMe
OK thanx
The 950s are really beginning to make their presence known. The prices on the Intel 750s are dropping substantially.
Can you please confirm or deny Windows 7 support for this configuration on the new system? Is this even possible?
What about linux support?
We won’t confirm or deny such as we haven’t created those builds sorry…
linux has nvme support in 3.3 kernel and later, it ships with all the distributions and just works.
I think putting NVMe SSDs into RAID 0 is pointless..
Until you’re a content creator trying to work on 4 (or more) uncompressed 4K video streams at the same time and mux them all together @ 400 MB/s per stream or more, depending on how they were recorded.
And M.2 raid like this is the cheapest option at the moment for this kind of speed in the retail consumer tier. Plus (I don’t know but I would expect so) you could likely make a bootable Intel Matrix RAID Raid-5 out of 3 of these right on the motherboard. Which, this option traditionally in the past, has been way too slow to even be remotely viable. But with these speed of these babies, it might be a real option now.
Any guess what performance would be like using the two 512GB sticks? I always try to avoid mixed-size RAID. The 2X RAID test makes an interesting case to save money on the 3rd stick. The proverbial point of diminishing returns. Did the 2X test use the UEFI or W10 RAID?
Hey guyz I just installed NVMe Samsung 256GB and had real problems at beginning after win 10 install very bad speeds but now it corrected.. It needs to have samsung driver controller installed + DON’T install unnecessary Intel RST software or driver.. it can conflict!
it won’t even detect NVMe it is just for AHCI and RAID…
And bythe way doing RAID 0 with two or NVMe SSDs is pointless as it goes through AHCi again bcos you have to setup 2 SSDs in IRST in BIOS which enables AHCi then NVMe is off
Here after few minutes even better
Guys u need to download samsung driver controller and uninstall unecessary Intel RST program.. twill dedect NONE storage! that is just for AHCI ..
check pic..
[URL=https://s1295.photobucket.com/user/Ethos-Evoss/media/NVMe%20SSD%20performance_zps52lcfwtb.png.html][IMG]https://i1295.photobucket.com/albums/b633/Ethos-Evoss/NVMe%20SSD%20performance_zps52lcfwtb.png[/IMG][/URL]
We addressed this in our original report f this SSD; the drivers are necessary for better performance. Removal of RST is not, however, and recognition of your drive occurs in BIOS before OS and software affectation.
Hello!
I managed through the whole article but I feel only more confused after reading it.
I even used the tool for my mobo to contact the manufacturer (Asrock) and they replied, but in a language I do not recognize:
Dear customer,
Thank you for response.
Z97X Killer supports Samsung 950 Pro M.2 NVMe type SSD.
notice:
NVMe type, it should be use on Win 8.1/ Win 10 platform.
(If you would like to install SM950 NVMe, please install OS under GPT mode. )
If you would like to install OS in the M.2 disk, please follow the steps to install OS with GPT mode.
1.Please set Storage OpROM policy to UEFI only in BIOS”Boot -> CSM(Compatibility Support Module)”
2.Please press “F11” enter to Boot menu.
3.Please select “uefi: USB/DVD OS Image Drive” to install OS with GPT mode
4.When you finished install OS, please reboot the system and press “Delete” to UEFI menu.
5.Please select Boot to Boot Option, and choose Boot Option#1 then select M.2 storage.
My question is simple:
Can someone please translate this into more user friendly text?
I have Asrock fatality z97x killer and would like to know if my M.2 socket is compatible with samsung 950 pro. Will I be able to install OS on that drive and will it be limited in any way, regarding read/write speeds etc. Also, I have R9 290 Sapphire vapor-x in pciex16 installed, will it not collide with the samsung SSD right underneath the GPU?
Truly sorry for that ellaborate, but I lack knowledge in this matter.
Best regards,
Peter
I’m sorry but how can you not understand this? It’s perfectly clear in plain english words. It tells you exactly what to do step by step in BIOS. It doesn’t need translating, that’s english right there, all of it.
Is there anyway to make these bootable on an X99, like the Rampage v extreme, with an add in raid card, or is the best option putting grub on a small boot drive that launches the raid?
No…not that I am aware of. the difficulty will always be having the boot files execute at or before the bios.
I read the title of the article wrong; I thought this would be about boot times with the RAID 950. But on that topic, I always want to know how fast my system would boot up with a pair or more of these bad boys. Any results?
The DMI 3.0 connection between the processor and z170 chipset is limited to about 4GBps, that could be a bottleneck and the reason you got similar numbers for the 2-device and 3-device arrays. Since the m.2 4x PCIe 3.0 interface can also do about 4GBps, I’m thinking just one drive will be enough to saturate that bottleneck in the not-too-distant future.
The DMI 3.0 bottleneck is very real. I’ve got a Raid0 of 2x Samsung 950 Pro through 4x PCIe 3.0 and they read 5 GB/s. Connecting more than 1x NVMe M.2 to the motherboard is useless.
I hear you and feel your pain. The DMI 3.0 bandwidth is shared among a lot of things, all the chipset SATA, USB, and PCIe. There will be some benefit since more that one drive won’t be accepting data or returning data at the same time, but it is a dumb way to go. I want to do RAID5 with this system which from 3 drives gets me 2 drives worth of space with data security. Shame that I won’t get the throughput I want because of the Z170 chipset DMI 3.0 bridge, but what can you do. Using each of the m.2 connectors disables available SATA3 connectors, and if you use all three, you lose all of the chipset SATA3. AsRock include another four. I don’t know where they get their bandwidth. It could be the chipset which also has other serial connectors or a connection to the CPU directly which also has other serial connectors. They do not say in their specs, but I sure wish they did. If it is the chipset, then it will be more contention for the DMI 3.0 bandwidth. (DMI 3.0, released in August 2015, allows 8 GT/s transfer rate per lane, for a total of four lanes and 3.93 GB/s for the CPU–PCH link.) That’s less than theoretical for one Samsung 950 M.2 drive.
Where’s the 5 gbps coming from? Like Eric said.. the DMI is limited to PCIe 3.0 x4 which is a max of 4gbps not 5gbps… ?
The 5GB/s come from the Raid0. I’ve got 2x 950 Pro connected to 2 different 4x PCIe 3.0 lanes which go straight to the CPU (and not through the chipset / DMI 3.0 bus ). This configuration reads 5GB/s. Plugging the 2x SSDs to the chipset m.2 ports achieves 3GB/s read bandwidth.
Wich adapter did you use?
Adapter?
do both of the PCIe x4 3.0 go straight to the CPU and the M.2 through PCH?
Yes I came here to ask that question. We can see the 3 device arrays triple the write speeds but the read speeds seem to be capped at 3.2GBps… is this because the DMI is making use of something else and can’t reach the 4GBps limit? In theory a 3 device array can reach read speeds of 4.8GBps
Help please. ASRock Extreme7+ and (2) 950 Pro’s. Followed instructions and in UEFI created RAID1 disk, then boot under UEFI DVD drive, then get Windows Server 2012 R2 setup and enter PK but then at Where to install it can’t see the drive. I’ve quadruple checked everything, but no go. I was able to go back to UEFI and delete raid and switch back to AHCI and then Windows sees my two drives, but I want RAID. Any suggestions?
No suggestions sorry. Once the UEFI RAID is built, it should be recognized as a logical drive as it was ours. Hmmmm fresh install eh?
I used an old HD to install Windows Server but the LAN ports aren’t recognized and Samsung Magician crashes the OS (tried compatibility mode and run as admin, but still crashes). I am giving up on Server and will try installing Win10 and see how that goes. Something just doesn’t feel right with this setup but I don’t know what.
So it isn’t a fresh install?
Yes, it was a fresh install. I just couldn’t install on the 950’s in raid so I put in an old HD so that I could install Windows Server on it. On the old HD I did delete all partitions and formatted it and was able to install Server on that drive. (That way I figured I could get server running and then run the Samsung 950 driver and run Magician to update the 950’s in hope that then I could install Server on the RAID’ed drives). Server did install on the old HD, but didn’t install LAN drivers for either LAN port. The ASRock driver disk wouldn’t install anything because of the Server OS. On another PC I downloaded the LAN driver from the ASRock website and copied it over to the new PC, but it wouldn’t run either. So, I found an article on disabling Server’s strict enforcement of signed drivers and tried to run the LAN installer, but it crashed. I was able to go to device manager and update driver and manually select a driver inside the LAN driver folder and it made one of the LAN ports work but not the other. Windows Server was then able to do a Windows Update for 148 updates, but after a reboot the other LAN port still wasn’t enabled and no driver. So I downloaded Samsung Magician and installed it, but when I run it, it freezes the PC, sometimes hard so that no control-alt-delete even works. I just concluded that this motherboard isn’t supported enough by Windows Server and certainly isn’t supported by ASRock for Server. I can use the PC with Win10 for a while with my old database server, but won’t be able to upgrade to newest FileMaker Server because it requires Windows Server, so I’ll have to do something else when I need to upgrade FileMaker to the newest server. But for now I can use the old FM Server software.
Did you try getting raid drivers off the asrock website and load them in to setup (using a usb flash drive) during your fresh install with the raid configuration on the 950’s ? That’s probably why it didn’t see it: windows didn’t know what it was. It’s pretty standard to have to load raid drivers during windows setup, a normal thing.
I’m certainly no expert, but the article above shows that the RAID is actually created in BIOS via ASRock’s baked in Intel RST so that Windows should “see” it as a valid drive, but something’s wrong in my case. I’m using Windows Server 2012 R2 instead of Les’ Window 10, so that might be the reason. RAID drivers have to do with RAID cards that connect to SATA or SAS drives, not NVMe drives. NVMe is too new.
It’s still created with Intel Matrix Storage Manager (AKA RAID) using the onboard motherboard. It is still considered “RAID” and there are drivers for it provided by intel and Asrock. If the motherboards are newer than the operating system and use new technology since the OS was created, then yes, you will most likely have to load drivers for it. It should be on the asrock website, and if not you can usually get it on intel’s website in the download center. Just go search for it, copy to usb drive, load during setup and you should be on your way, most likely.
Just because it doesn’t appear as a valid drive “without loading anything” during windows setup does not mean it isn’t functioning. It’s probably working fine, windows just needs a little help I would guess.
Here, I’ll even do the work for you: https://downloadcenter.intel.com/download/25165/Intel-Rapid-Storage-Technology-Intel-RST-RAID-Driver
If that doesn’t work then I don’t know what to tell you. But that’s the first thing that came to my mind when you said you configured it and got in windows setup and the thing didn’t show up as an installable drive. So try that.
Thanks. Your instinct was right. I just needed the driver but asrock’s funny naming of the driver as Sata Floppy driver threw me off plus the fact that I completely misunderstood the concept.
Yes, these Asrock boards are shit when it comes to raid, I have a brand new z270 killer sli/ac board and brand new 2x 960Pro 512gb m.2 and same story, I create the array in bios, no issue.. Then I go to install the latest Win10x64 with piles of different drivers from everywhere I look and not ever seen! Garbage, I’ll never own another shit Asrock board ever.
UPDATE: After ASRock tech support, I was able to install Win10 but there’s a few things different that I had to do.
#1 You should update your UEFI firmware to the newest 2.10 but use the ASRock pinned Forum post called “UEFI Update 2.10 Information”. Otherwise you may only get half of the update (it updates both the UEFI firmware to 2.10 and the Intel ME Firmware to 11.0.0.1180). The gist of the post is to be patient and let the update reboot the PC and leave it alone and wait even if it seems like it’s hanging. If you interrupt it you will not get the current Intel ME firmware. The only place to find out what version of the ME firmware you have is in the UEFI under Advanced > Storage Configuration.
#2 You must download and UNZIP and copy to a USB drive the ASRock RAID driver (“SATA Floppy Image”, I know, terrible name that doesn’t sound like a RAID driver). Then when you get to the Windows Installer point where it asks “Where do you want to install Windows” and you don’t see your RAID drive, you can click the button to browse for a driver, and you browse your usb to that driver and after a few minutes of loading, you will see the RAID volume.
#3 When installing your OS, you must boot from the UEFI version of your installation boot drive (USB or Optical Drive, i.e. where your OS installer file is on). Upon boot press F11 to bring up the boot window and select your install drive listing that begins with UEFI: _____.
#4 After installing Windows, you should run the INF and ME updaters and then the LAN drivers, BUT you shouldn’t have any ethernet cable plugged in while updating the LAN drivers (I did and the installer hung the PC. Suspecting the problem, I unplugged the cable and rebooted and then uninstalled the LAN software, rebooted and then installed the LAN software again and then it worked fine.
Later, I will attempt to do install with Windows Server 2012 and see if I can coerce the drivers.
I still don’t know where you went astray but am happy your method worked for you. Our instructions on this website are still the prescribed method and much easier.
My main issue was the need for the raid driver from asrock that I didn’t know existed because they name then so poorly. I’m glad your raid showed up without any other steps, but just in case someone else has trouble like me, I wanted to document what I needed to do to make it work.
Thank you for that. I had and im kinda still having trouble.
For me after i build the RAID i could not choose to boot from it after i copied over the contents from my former OS ssd.
I then tried installing from the windows 10 disk and could again not see the SSD in the Overview. After using the RAID Driver like you described it finally showed up and i could install it fine and boot fine from the m2 SSDs. Interesting fact. The Entry in the Bios says Windows Boot Manager instead of Intel Raid which has to do with the UEFI Installation i guess.
Unfortunately this would mean i would have to start from scratch which im not really fond of so im still trying to find a way to copy my old Windows to the M2 SSDs.
Unfortunately it wont show up in the Bios then. I guess it has to do with being a Legacy installation instead of a UEFI one as the new one.
Hello so will I be able to boot win. in my laptop when I connect two M.2 NVMe or M.2 AHCI SSDs then
I will choose RAID mode.. After restart RST will appear in BIOS .. Do you thing it will boot ?
Thnx
If RST appears in BIOS, the purpose is to enable you to RAID at the BIOS mode so that you can use that RAID as a logical drive and boot from it.
I have an i7 4770k and Asus Maximus VI Hero and Windows 10 Pro 64-bit. Can I use an NVMe SSD with an adapter or wait for Kaby Lake?
Can this be done with a x99 motherboard? Will it work on asrock fatal1ty x99 professional?
You need to email Asrock support and ask them for confirmation of this function with your motherboard. This review was only testing this configuration on the new Intel Skylake platform, not x99.
I have both the X99 and Z170 in the office. This conversation has been very interesting and informative. On return from CES in Las Vegas next week, I will check this out. Are we saying that we will get better performance from the M.2 in the X99 rather than Z170?
Did you try RAID5? ASRock’s website says they support it. If you didn’t, could you?
It is my understanding that I cannot set these up as bootable raid 0 for the x99 chipset, correct. Is there a raid controller that would make this possible? If not, is there any benefit to do this as a non-bootable drive?
Hello, I am still little confused about the lanes for PCI-E…
I already have ASRock Z170 Extreme 7+, and 10*HDD for storage. I already know, it means, I can’t use SATA-Express or M.2 ports, since they are sharing the lanes (1x M.2 = 1x SATA Express = 2 SATA).
What I don’t know is:
Can I use 2x Intel SSD 750 (2×4) + 10 SATA HDD + Graphic card (x16)?
Because I don’t know:
– how many lanes is avaible pro PCI-E?
– how does the maping of PCI-E SSD working?
– can 2x PCI-E SSD be connected same time as 10*SATA HDD to this ASRock?
Thanks for answers.
I found this article while trying to resolve a performance problem with RAID 0 on an ASRock Z170 Professional Gaming i7 motherboard. I got the two Samsung SM951 512GB AHCI drives installed as RAID 0 in the BIOS with no problems. Windows 10 Pro loaded with the IRST driver on the thumb drive. Everything seems good. But when I run benchmarks I get sequential reads speeds lower than the writes. Crystal Disk Mark has them about where the drives would perform without RAID, 2100 MB/s. Writes are 2500 MB/s. AS SSD Benchmark results are even lower at about 1700 MB/s. I had this same setup on an Asus board and got 2800 MB/s sequential reads. I’m thinking of RMAing the board. Any ideas would be appreciated. TIA.
hello ,
My name is Arun and i am using supermirco server 1028R-MCT with 2 x 480 GB Intel SSD’s so its possible to use raid 1 for its possible . kindly revert .
my Email I-D : arunbohani@gmail.com
Using the Extreme 7+ Mainboard and 2x NVME Samsung M2 SSD 256 GB. The Raid works fine, but i wonder i can only get around 1800 and not 2gb+ ? Do i need 3 drives for more speed ? I mean, even in Raid0 now its same as if i use both ssd separatly.
I guess I really got lucky. I am using three bit older AHCI PCIe gen3 x4 drives (Samsung SM951) in windows software RAID 0 (boot from another single M.2 gen3 slot on the motherboard) and I’m getting over 6GB/s sequential read. The 3 SSD’s in the array are installed on three PCIe x4 to M.2 adapters in three of my PCIe slots (I only use one video card). My DMI chip is only v2.7 so I must be bypassing it. Another thing is that my CPU has 40 PCIe lanes (I7-5960X on X99). The I7-6700K used in this article only has 16 PCIe lanes. The M.2 slots on their setup might have been contending for lanes when trying to read the stripes on the drives simultaneously. Perhaps using the Intel graphics in the processor would free up some lanes.
Here’s the PCIe to M.2 adapter I used:
https://www.newegg.com/Product/Product.aspx?Item=12K-017B-00001
The first NVMe RAID adapter (U.2):
https://highpoint-tech.com/USA_new/series-rr3800a-overview.htm
A comparitive test using the IGP is an excellent suggestion.
Just a note that at least with the newer AS-Rock motherboards (Z390 Phantom Gaming 6 in particular), you only need to turn on CSM to enable UEFI, after that, you can turn CSM back off (UEFI stays on), which improves some other areas.
This allowed me to setup up a bootable RAID1 system using NVMe drives. Fast and bulletproof reliable (RAID5 would be cheaper per GB, albeit with slower write speeds, but the Phantom Gaming 6 board only includes 2 M.2 ports, not enough for RAID5).