Two PCIe slots looked the same but gave different benchmark results - PCIe-2 vs PCIe-3

I benchmarked a new-old Intel P3700 PCIe disk drive I got off eBay in my HP Z820 and the P3700 seemed to underperform. I didn't get the performance I expected in a benchmark.  I put the P37000 in the wrong PCIe slot.  The HP Z820 machine has a bunch of PCIe-3, apparently one PCIe-2, and one classic PCI slot. All the connectors looked the same in the dark but one is not like the others. The P3700 was in the wrong slot.

PCIe 3.0 supports a max speed of 8.0 GT/s while PCIe 2.0 supports a max speed of 5.0 GT/s. I have no idea if there was any real-world difference.

PCIe 2.0 vs PCIe 3.0

These are two representative samples with an Intel P3700 in the v2 vs v3 slot.  I ran both CrystalDiskMark tests with the Default setting not the more stressful NVMe configuration. There is a pretty significant boost in sequential performance moving from PCIe-2 to PCIe-3.  The rest of the settings appear to be maxed out because the Default configuration doesn't generate enough load for an SSD of this type.

This was disappointing because only Sequential reads showed any major impact.  We see different results if we change the testing methodology.

Test Configuration Impact - Saturating the Device

CrystalDiskMark 8 has two different test modes, Default and NVMe. The difference is the number of Queues and the number of threads in the middle two tests. The differences in the two modes demonstrate that the NVMe drives really need to be saturated to see the differences in some tests.

Random Operation Differences

These results may explain why people don't see huge differences between running O/S operations on an SSD vs PCIe 2.0 vs PCIe 3.0.  Random operation performance is really pretty similar when the device isn't saturated with parallel operations. 

PCIe 3.0 NVMe drives probably make a huge difference in the data center or cloud where higher IOPs or parallel random activity is more common.

SATA 6Gb/s Comparing NVMe vs Default tests

All of this made me wonder what kind of performance I'd get out of my SATA disk.  Was the PCIe drive really that much faster?  It turns out the test config does have an impact but still leaves the SATA way underperforming the P3700 bus-based disk.

Running the same two configurations against a 6Gb/s SATA shows that the single-threaded random test is probably not saturating the drive for read operations. Changing the test configuration to NVMe increases the thread count resulting in about a 2.5x performance boost for that single benchmark.  

Revision History

Created 2023 02

Updated 2023 05