Performance characteristics

[avefrandsen]

I'm interested in what understanding exists regarding performance, specifically in regards to the host computer specs.

I know:

• A given HBA usually (always?) has a fixed maximum endpoint device limit
• SAS has a bandwidth limit by gen by lane
• PCIe has a bandwidth limit by gen by lane

Questions:

• What are the behaviors of SAS & PCIe overhead at high utilization
• How does bandwidth bottleneck between the whole set of PCIe lanes and the CPU? (are lanes guaranteed bandwidth or overprovisioned?)
• How does nwipe utilize RAM?
• How much computation is nwipe doing on the CPU?
• How well does nwipe parallelize?
• Is there any hardware acceleration/offload? Is it possible to involve a cryptographic accelerator card? Would that matter?

I've only had the chance to notice two things

Originally I tried using Dell Optiplex 5000 and 7000 series desktops with 8th gen i5s & i7s. Even with the same HBA, same bootable media, same PCIe gen, there was severe bottlenecking. I don't have the reasoning anymore but at the time I attributed this to a problem with the PCIe controller. Switched to a multi-CPU workstation running xeons at a slightly lower clock speed but with many more cores, and the problem went away.

After that I increased the system RAM to 64GB and saw a 15~20% improvement in total performance, with high performing SATA drives.

In all 3 cases, the CPUs, RAM, HBAs, and SAS lanes were not maxing out. Drive quantity and performance are definitely my current bottleneck, and I'm working on increasing quantity. I don't love the machine I'm currently using, but I also don't want to give up too much performance as I pick out the new parts