Anandtech has an article explaining how they have built their new testbed for NAS reviews. They explain the reasoning behind designing and building a new testbed for NAS testing:
Followers of our NAS reviews have seen the standard Intel NASPT benchmarks and file transfer test results along with a qualitative coverage of the NAS’s operating system / user interface. The reviews briefly touch upon miscellaneous factors such as power consumption. The feedback from the readers as well as the industry pointed towards some essential NAS aspects such as performance under loading from multiple clients being ignored. Towards the end of 2011, we started evaluating approaches to cover these aspects.
They achieve their goal of simulating multiple clients connected to a NAS by creating multiple virtual machines (VMs) within a powerful computer. With one SSD for each VM - twelve in all, and twelve physical ethernet ports, this is one powerful computer.
Software-wise, the testbed runs Hyper-V on Windows Server 2008R2:
The virtualization infrastructure in our testbed involved running Hyper-V on Windows Server 2008 R2. The benchmarking programs include IOMeter 1.1rc1 / Dynamo and Intel NASPT. While the former is used to test multi-client performance, the latter is used for testing the single client scenario. IOMeter itself runs on the host machine, while Dynamo instances run on each VM. Intel NASPT is run from one of the VMs only.
On this page, they run the Synology DS211+ through its paces using the new testbed. Performance is summarized using a number of graphs and with this conclusion:
The first graph shows that the DS211+ is able to hit as high as 112.65 MBps in 100% sequential reads when five clients are simultaneously accessing the unit. However, for more than 7 clients, the total available bandwidth sees a decrease. In all the four cases, the average response time seems to increase only linearly. However, a look at the maximum response time in the above link shows that under some conditions, clients need to wait for as much as 10 seconds for a transaction to complete.
The Synology DS211+ seems to be a good fit for cases where there are 5 - 10 computers simultaneously accessing the unit, but performance seems to degrade with additional users.
Similar tests are run on the Thecus 4800 with somewhat better results with multiple clients connected:
The Thecus N4800 seems to hold up much better when compared to the Synology DS211+ as the number of clients increases. With just twelve simultaneous clients, we don't see a marked decrease in performance at any given node, unlike what we saw with the Synology DS211+ in the previous section. However, this is only to be expected because of the hardware configuration.
Overall, a very interesting article. I'll be looking to see the results of testing other NAS servers using this method.
Related articles
- Building the 2012 AnandTech SMB / SOHO NAS Testbed (anandtech.com)
- Thecus N2800 Review (nas-pro.com)
( Leave a Comment )