GIGABYTE MZ31-AR0 (SP3 EPYC) Server Motherboard Review

Introduction, Specifications, and Packaging GIGABYTE MZ31-AR0 (SP3 EPYC) Server Motherboard Today is a grand day as we have our first motherboard for AMD's EPYC processors on the test bench. Since we reviewed Intel's new server processors, users have been asking us to show what AMD's EPYC platform can do in comparison. You ask, we deliver! Today we're taking a closer look at GIGABYTE's MZ31-AR0 motherboard (product link) with an SP3 socket for EPYC CPUs. AMD EPYC Processors AMD has truly made a huge comeback with their Zen architecture chips. From the entry-level market APUs over mainstream Ryzen and enthusiast Threadripper to the enterprise-level EPYC processors we're using today. The value you got is uncomparable while compatibility is high thanks to the x86 architecture. You can get processors with up to 32 cores and 64 threads, each supporting 128 PCIe lanes and 2TB 8-channel memory. The possibilities are almost endless with such options. Not only that, the power can be regulated per core and it comes with integrated custom security on the chip. Close to No-Limit Expansion Options There truly aren't many limitations with this platform's options. Both the single- and dual-socket CPUs come equipped with 128x PCIe lanes, providing extensive I/O options. That is a huge improvement in I/O vs. previous x86 offerings. To put that into perspective, it equals 8 PCIe slots with x16 lanes at full speed. Such an increased I/O allows for flexibility in extending every aspect of your system. From your storage, over networking, to computing power. You can take it all to a level previously unattainable with standard server motherboards. In addition, the integrated nature of the processor, which is designed as a system-on-chip (SoC), means that there is no need for a separate PCH. Thereby, less additional switches are needed for the expansion lanes. GIGABYTE MZ31-AR0 Information and Block Diagram GIGABYTE's MZ31-AR0 is an E-ATX motherboard with a size of 305 by 330 mm. The official block diagram gives us a great view on how it's all connected, which is why there isn't a whole lot I can add here. Processor and Memory Support GIGABYTE's MZ31-AR0 support a single CPU of the AMD EPYC 7000 series processor family. Which you pick is completely up to you and your needs as it supports them all: Up to 32-cores and a TDP of up to 180W. So knock yourself out and build the best you can afford. Besides CPU threads and PCIe lanes, memory support is a strong selling point for AMD's EPYC. This motherboard comes with 16 DDR4 DIMM slots and fully supports the 8-channel memory architecture. With support for 64GB modules, both RDIMM and LRDIMM, you can add up to 1TB memory on the MZ31-AR0 motherboard. Memory speeds of 2666 and 2400 MHz are supported, but it will run at 2133 MHz when both 1R and 2R memory modules are fully populated. Networking Options With such strong I/O capabilities, the onboard parts wouldn't need to be plenty nor good - you could easily add what you want. But they are both plenty and great when it comes to GIGABYTE's MZ31-AR0 motherboard. Gigabit is old-school which is why it comes with not one, but two SFP+ 10GbE LAN ports. Those are controlled by one of Broadcom's BCM 57810S. But it doesn't stop here, there's another LAN port. There's a Gigabit-capable RJ45 port for the remote management interface. With this IPMI interface, you can use GIGABYTE's Server Management for easy control of your infrastructure. Storage Options The storage options are just as comprehensive and cleverly designed. The MZ31-AR0 has an onboard M.2 slot with a 32Gbps speed which is an optimal place to insert your OS drive. The M-key slot is PCIe 3.0 x4 and supports 2242, 2260, and 2280 modules. More PCIe-based drives can be installed in the available PCIe x8 and x16 slots. There are plenty of connection options for SATA-based drives too. The motherboard supports up to 16 drives natively, but not via traditional SATA connectors. Instead, Gigabyte added four SlimSAS ports which each supports four drives. This doesn't just save on PCB real-estate, it's also clever for server setups. 16 SATA cables will take up a lot of space and can interfere with the airflow, that's one reason. The other is that most server setups will use a backplane which already delivers power and close-by connectors. Using SFS cables will make for a much cleaner setup. Other Connection Options We've already covered the network options, so enough about that. But what else is there on connection options. Externally, you get two USB 3.0 and two USB 2.0 ports on the rear IO. There's also a VGA port controlled by the onboard Aspeed AST2500 chip. Other than that, you get a standard COM port and an ID button with LED. Internally, you get two CPU fan headers and five system fan headers. There is an additional USB 3.0 and USB 2.0 header too for chassis-ports. You can also attach a second COM port to the motherboard. Last but not certainly not least, there are a lot more headers. Two LAN activity LEDs can be attached, there's an HDD backplane header, TPM header, PMBus connector, and an IPMB connector too. Motherboard Features Server motherboards need to be of great quality as any system downtime can cost a lot in lost profits. GIGABYTE's MZ31-AR0 certainly qualifies in that area as all components have been carefully selected. They didn't go for the cheapest option, they got the best and best-suited parts. That in return guarantees a super stable operating environment where the processors and the memory can deliver their maximum performance. GIGABYTE’s server motherboards are built with ultra long-lasting components for the highest reliability attainable. That's a fact that is shown by GIGABYTE having some of most durable servers in the industry. GIGABYTE’s EPYC server platform is fully compatible with AMD’s new Radeon Instinct MI25, one of the world’s fastest training accelerators. The MI25 can deliver up to 24.6 TFLOPS of FP16 and 12.3 TFLOPS of FP32 peak performance. It also features large BAR (Base Address Register) support for multi-GPU peer to peer communication. Combining AMD’s CPU and GPU solutions together with its ROCm open software platform allows for enhanced synergy and optimization, delivering a powerful compute solution with lower latency to meet the challenges of machine intelligence and deep learning. Motherboard Management GIGABYTE has been in the server business for a long time and they know that the ability to manage all one's systems centrally is as important as the hardware itself. For that, you get GIGABYTE Server Management, or GSM for short. The exclusive GSM utility is designed to provide effortless datacentre and rack-level management. Whether you prefer a browser-based client or a command line interface is your choice. It enables administrators to remotely manage and monitor the GSM Server and GSM Agent modules. This reduces administrative burden further increasing ROI. Additionally, the API programmable client enables administrators to streamline and automate workflows with multiple concurrent sessions. The Aspeed AST2500 management controller is what makes this possible in combination with the Avocent MergePoint IPMI 2.0 web interface. With it, you can manage and get access to Network settings, Network security settings, Hardware information, Users control, Services settings, IPMI settings, Sessions control, LDAP settings, Power control, Fan profiles, Voltages, fans and temperatures monitoring, System event log, Events management (platform events, trap settings, email settings), Serial Over LAN, and vKVM & vMedia (HTML5). BIOS Upgrade Without Core Components The GIGABYTE BMC Interface allows you to upgrade the BIOS without having installed any core components into the motherboard. GIGABYTE has developed an integrated function for this which doesn't require any additional software. With it, you can update the BIOS through the BMC controller. That means, there is no need for a processor, memory, drives, or operating system. You don't even need to power the system on, but it needs to be connected to a power supply. Then you can update single or multiple boards via IPMI 2.0 or CLI. Feature Highlights AMD EPYC 7000 series processor family 8-Channel RDIMM/LRDIMM DDR4, 16 x DIMMs 2 x SFP+ 10Gb/s LAN ports (Broadcom BCM 57810S) 1 x dedicated management port 4 x SlimSAS (for 16 x SATA 6Gb/s) ports Ultra-Fast M.2 with PCIe Gen3 x4 interface Up to 4 x PCIe Gen3 x16 slots and 3 x PCIe Gen3 x8 slots Aspeed AST2500 remote management controller Package and Accessories GIGABYTE's server motherboards all come in the same style of packaging. The only difference is the size of the box. Well, there's also the label on the side which will identify what's inside. The MZ31-AR0 comes with a quick-reference guide, two SlimSAS cables, and an IO shield. There is no padding or similar features on the IO shield, but everything is clearly labelled as it should be. The included SATA cables are thin, which will make it easy to route them to your backplane - or even to the drives directly should that be your need. A Closer Look & Layout Analysis We had a look at the general feature on the first page and now its time to take a closer look at where everything is located. Power Connections and Onboard Buttons A server motherboard usually doesn't come with any onboard buttons, they simply aren't needed here. Not in the same way they are on an overclocking board, for example. We do have power connectors though, and they're all in the top corner behind the IO area. There are two 8-pin and one 24-pin power connectors to supply enough power to all components. All the caps and power regulators on the board are high-quality as you'd expect them to be. They're located all around the main CPU and memory area, making sure all the components get a stable power delivery. Processor and Memory Sockets Server motherboards have a generally different layout as they need to fit into rack chassis rather than desktop setups. EPYC boards such as this look different tho as there are so many core component options. The CPU and memory area takes up nearly half the PCB space. We can also spot the two 4-pin fan headers for the CPU cooler right next to the CPU socket. Storage Connectors and USB Headers The storage connectors are located at the inner bottom corner. There are four SlimSAS ports which each supports four SATA drives. A great setup choice over adding 16 SATA ports on the board. Not only would it take up a lot of space, the SlimSAS setup allows for a much cleaner cable setup. Also, I really wouldn't know where to put 16 standard SATA ports on this motherboard. There simply isn't the space for them. The PCIe 3.0 x4 M.2 slot is located right next to the SlimSAS ports which keeps all storage options close by each other. PCI-Express Slots With 128 PCIe lanes from the CPU, there's enough to bandwidth to run all expansion slots at full speed. The MZ31-AR0 motherboard has five x16 and two x8 slots available which allow you to install a lot of cards without any worry about performance throttling. It should be noted that you won't be able to install longer add-in cards in a lot of the slots. At least not unless you use riser cards or cables. HHHL cards will fit with no issues, but graphics cards, for example, can be too long to fit in anything but the bottom x16 or x8 slots. This isn't an issue for most server setups, but it's worth mentioning. IO Area There aren't a whole lot of connection options, because you don't need a lot. From left to right, you get a COM port, a D-Sub VGA port, two 10GbE SFP+ ports, a Gigabit Ethernet management port, two USB 3.0 ports, and two USB 2.0 ports. The button all the way to the right is for identification and location purposes. The 10GbE LAN controller is placed directly behind the ports and it is well-cooled thanks to the added heatsink. Other Headers and Onboard Connectors The onboard headers and connectors are located at the bottom side, starting with the serial COM header. Next to that, you get four 4-pin FAN connectors and the USB 2.0 and 3.0 headers. The backplane header follows those before we get to the frontpanel header. And that's it. A realy simple setup without a lot of not-needed extras. The BIOS battery and header, as well as the on-board speaker, are located at the rear side, close to the IO area. The only thing left to show is the TPM connector next to the Aspeed AST2500 controller. We can also spot the SKHynix memory for it right below. Setup and Testing Scenario Our goal with the server and workstation motherboard review is to bring you a wide variety of tests and results which will, hopefully, allow you to make the best choice for your future setup. Additional Hardware Components It is not possible to test motherboards without additional hardware, but we also don't want to test the connected devices. That is a fine line to walk, and we hope we're hitting the spot. We will use the hardware needed, but we won't go above that. By keeping it to the basics, we're eliminating a false of performance originating from another piece of hardware. We'll install up to 8 memory modules, a processor with cooling, and storage drives of each type possible. There's also a GPU installed, but not a flagship. We do need monitor-output, and the GPU is also needed to run some benchmarks. Changes in Testing Methodology I hate changing my testing methodology as it makes comparing results to previous reviews more difficult, but it can't always be avoided. With the new generation of motherboards, we've truly entered a new era. SATA is still supported, but it's a true legacy device now. As such, we've moved to a PCIe NVMe card for our operating system. The same goes for graphics cards. Our old card was BIOS only with no UEFI support. The new generation boards won't boot with legacy BIOS and as such, we also had to upgrade the GPU. Due to those changes, some previous results seem far lower in comparison. However, I will retest what is possible and update previous reviews as well as new ones with new and better comparable results. That will be done in between my normal reviews and not on a set schedule. There is one more change and that is the OS. In the past, I've used Windows Server 2012 R2, but driver support is very tricky in that aspect. So I've moved to Windows 10 instead which makes the whole setup a lot easier. Hardware Utilized CPU: AMD EPYC 7551P RAM: Crucial DDR4 2133 MHz (16x 16GB) PSU: be quiet! Dark Power Pro 11 1200W GPU: ASUS ROG Strix RX570 Cooler: Noctua NH-U14 OS Drive: Intel Optane 900p 280GB PCIe Drive: Intel Optane 900p 280GB M2 Drive: Toshiba OCZ RD400 512GB SATA Drive: Toshiba OCZ VX500 512GB SATA3 USB Drive: SilverStone TS11C with Toshiba OCZ VX500 512GB SATA3 Software Utilized We use a wide variety of applications to gain a broad spectrum of results for comparing diverse aspects of the system performance. The OS will be Windows 10 Pro, and most should be familiar with the other applications and tools too. Windows 2016 Data Center Edition AIDA64 Engineer Edition Anvil's Storage Utilities Cinebench R11.5 Cinebench R15.0 Geekbench 4 Handbrake OCCT PerformanceTest 8.0 SiSoftware Sandra Platinum 2017 SP2 SPECwpc SuperPi Mod 1.9 WP wPrime 2.10 External Hardware There are more components needed such as a switch to connect the network ports which the motherboard has to offer. For this, I'm luckily equipped with a nice D-Link DXS-1210-12TC switch and a full set of CAT7 cables. We'll also be using a power meter to check on the electricity consumption. CPU Performance AIDA64 Engineer AIDA64 Engineer provides several methods to measure system performance. These benchmarks are synthetic, i.e. the results show only the theoretical (maximum) performance of the system. In contrast to application tests, synthetic benchmarks do not tend to reflect the "real world" performance of the computer. These benchmarks provide a quick and easy comparison between computer states, e.g. when certain parameters (CPU clock speed, memory timings, etc.) change in system configuration. Cinebench R11.5 & R15.0 Cinebench is a real-world cross platform test suite that evaluates your computer's performance capabilities. It is a perfect tool to compare CPU and graphics performance across various systems and platforms, and best of all: It's completely free. Geekbench 4 Geekbench 4 measures your system's power and tells you whether your computer is ready to roar. How strong is your mobile device or desktop computer? How will it perform when push comes to crunch? These are the questions that Geekbench can answer. HandBrake HandBrake is a tool for converting video from nearly any format to a selection of modern, widely supported codecs. There are three reasons to love this tool: It converts video from nearly any format, it is free and open-source, and it is available on Windows, Mac, and Linux. PerformanceTest 8.0 PerformanceTest is a fast, easy to use software benchmarking tool that allows everybody to quickly assess the performance of their PC and compare it to a list of standard 'baseline' computer systems. With it, you can compare the performance of your machine, measure the effect of configuration changes and upgrades, and make objective, independent measurements on which to base your purchasing decision. SiSoftware Sandra SiSoftware Sandra is a system analyser that includes benchmarking, testing and listing modules. It tries to go beyond other utilities to show you more of what is going on under the hood, so you draw comparisons at both a high and low-level in a single product. SuperPi Super PI is a computer program that calculates pi to a specified number of digits after the decimal point—up to a maximum of 32 million. It uses Gauss–Legendre algorithm and is a Windows port of the program used by Yasumasa Kanada in 1995 to compute pi to 232 digits. wPrime wPrime is a leading multithreaded benchmark for x86 processors that tests your processor performance by calculating square roots with a recursive call of Newton's method for estimating functions, with f(x)=x2-k, where k is the number we're square rooting, until Sgn(f(x)/f'(x)) does not equal that of the previous iteration, starting with an estimation of k/2. It then uses an iterative calling of the estimation method a set amount of times to increase the accuracy of the results. Then it confirms that n(k)2=k to ensure the calculation was correct. This is repeated for all numbers from 1 to the requested maximum. Memory Performance For our memory tests we use the built-in memory benchmarks in AIDA64 Engineer, Geekbench 4, PerformanceTest, and SiSoftware Sandra. A wider array of tests will give us a wider array of results. AIDA64 Engineer AIDA64 Engineer provides several methods to measure system performance. These benchmarks are synthetic, i.e. the results show only the theoretical (maximum) performance of the system. In contrast to application tests, synthetic benchmarks do not tend to reflect the "real world" performance of the computer. These benchmarks provide a quick and easy comparison between computer states, e.g. when certain parameters (CPU clock speed, memory timings, etc.) change in system configuration. GeekBench 4 Geekbench 4 measures your system's power and tells you whether your computer is ready to roar. How strong is your mobile device or desktop computer? How will it perform when push comes to crunch? These are the questions that Geekbench can answer. PerformanceTest 8.0 PerformanceTest is a fast, easy to use software benchmarking tool that allows everybody to quickly assess the performance of their PC and compare it to a list of standard 'baseline' computer systems. With it, you can compare the performance of your machine, measure the effect of configuration changes and upgrades, and make objective, independent measurements on which to base your purchasing decision. SiSoftware Sandra SiSoftware Sandra is a system analyser that includes benchmarking, testing and listing modules. It tries to go beyond other utilities to show you more of what is going on under the hood, so you draw comparisons at both a high and low-level in a single product. Memory Latency Storage Performance To test the storage performance in our motherboard reviews, I use Anvil's Storage Utilities. It has a wide variety of tests per run and allows me to define the data to be used. We run the benchmark tests on the SATA III, PCIe and USB 3.0 and other available storage options. For SATA III and USB testing, I'll use a Toshiba OCZ VX500 512GB SSD. The PCIe capabilities will be tested with one of Intel's awesome 900p Optane SSD drives with 280GB. It should be noted that the PCIe drive also is the OS drive. SATA Performance SATA is the somewhat traditional interface used by most drives. The Toshiba OCZ VX500 is a nice mid-range drive which will suit the interface well. NVMe PCIe Performance NVMe is used by the next generation of drives, and they eliminate the bottlenecks created by the by now old SATA standard. These drives are the future, and I'll be testing those capabilities with the Intel 900p SSD with 280GB capacity. NVMe M.2 Performance M.2 has become the most common next-generation standard which means we'll test that too, whenever it is available. For these tests, I'll be using one of Toshiba OCZ's RD400 drives with 512GB capacity. Network Performance For the networking tests I will connect the test system up to my other 10Gbps test benches through a managed switch and run our tests. This setup should present a good real-world performance and give a better result than a direct connection would. Passmark Performance Test 8 The PassMark Advanced Network Test, a part of PerformanceTest, is designed to test the data transfer rate between two computers both of which must be running PerformanceTest. One of the computers acts as the server and will wait for a connection while the other computer acts as a client. System Performance We do not use a powerful and dedicated graphics card in this system. Instead, we try to test the motherboard performance as pure as possible. We still chose to include the tests for comparison reasons. GeekBench 4 PerformanceTest 8.0 PerformanceTest is a fast, easy to use software benchmarking tool that allows everybody to quickly assess the performance of their PC and compare it to a list of standard 'baseline' computer systems. With it, you can compare the performance of your machine, measure the effect of configuration changes and upgrades, and make objective, independent measurements on which to base your purchasing decision. SPECwpc 1.2 SPECwpc is the first benchmark to measure all key aspects of workstation performance based on diverse professional applications. The latest version is SPECwpc V1.2 (introduced on June 3, 2015), which extends performance measurement from physical to virtualised workstations. Results from SPECwpc V1.2 are comparable to those from SPEC V1.0.4, but not to any version before that. More than 30 workloads are included in SPECwpc V1.2 to test CPU, graphics, I/O and memory bandwidth. The tests are divided into the application categories listed below. Individual scores are generated for each test, and a composite score for each category is generated based on a reference machine, yielding a "bigger is better" result. The reference machine has the following configuration: x3430 processor, 8GB (4x2GB) memory, V4800 graphics, and SATA 7.2k rpm hard disk drive. Power Consumption The power consumption is measured at the power outlet and is as such the power draw from the entire system. Idle and off figures speak for themselves and for the load figures I stress the CPU to 90% and memory to 90% load with OCCT. Final Thoughts & Pricing Pricing At the time of writing this review, GIGABYTE's MZ31-AR0 can be yours for $665.00 in NA. In the UK, only one slightly used was listed at the time of review which comes in at £550. Conclusion What a motherboard, just think of the options you have. GIGABYTE took in the EPYC platform and created an impressive motherboard for it. The CPU option delivers a great base while the expansion options are what gives you all the options you could dream of. With 7 PCIe expansion slots, there is the option for a lot of graphics cards, storage controllers, network cards, and what else might tickle your fancy. The 128 PCIe lanes of the CPU allow you to run it all at full speed. GIGABYTE's MZ31-AR0 support all EPYC 7000 series CPUs as it allows for a TDP of up to 180W on the processor. With 16 DDR4 DIMM slots, you can also install an incredible amount of memory. I only had 16GB modules running, you can easily get four times that these days; and 128GB modules aren't far in the future. The overall board-design is well made too. GIGABYTE put what fits together in the same area. All the power connectors are in one corner and the storage connectors in another. All of the headers are located on a single side too, which makes it quick and easy to set up without the need to search for anything. Connection Options The connection options are simple in the amount, but advanced in the sense of what they offer. There are no slow Gigabit Ethernet ports for the normal operation. Instead, GIGABYTE added a dual 10GbE interface for fast-paced throughput. Well, there is a single Gigabit Ethernet port too, but that is for the IPMI interface. The ability to remotely control and maintain your system is something that will make any system administrators life easier. You can even upgrade your BIOS through it, without having any core components installed. Another thing to make the admin's life easier, especially since Gigabyte's version allows for the upgrade of multiple boards at once. Heck, upgrade your entire data-centre at once, if you want to. With 16 SATA ports onboard, the MZ31-AR0 has a base to be a great storage system. There's also an M.2 slot which could be well suited for an OS drive. Of course, you can add a lot more storage drives if you install some HBA cards. With 7 PCIe slots and 128 PCIe lanes, there's no reason not to. If you would use current 14TB drives, you'd get close to 2PB raw storage in a single system. That is, of course, assuming that you install seven 16-port HBAs into the system. Other than that, you get a couple USB ports and enough headers for the things you might need extra. Whether it's cooling fans, front-USB, an additional COM port, or TPM, PMBus, or IPMB, it's all there. A little more The dedicated management system also comes with onboard graphics in the form of an Aspeed controller. While not powerful as such, it eliminates the need for a dedicated graphics card that only would serve to take up a PCIe slot. What's left to say. The motherboard is very well built and GIGABYTE only used quality components all around. From the LAN controller to the power delivery: What you'd expect and want from a board of this class. There aren't any things you don't need and lot of what you do want: a perfect combination. It should be noted that longer add-in cards might require a riser cable or system. Due to the design of the motherboard with all the memory sockets and large CPU socket, there'll be a conflict otherwise. The bottom two slots are free of this obstruction, so you can install some long cards. normal HHL cards will fit nicely in any slot without any conflict. Pros Support for all EPYC 7000 series processors 8-channel RAM setup with 16 slots 16 SATA ports via 4x SlimSAS and M.2 PCIe 3.0 x4 Dual 10GbE Ethernet Aspeed AST2500 remote management controller Nearly endless build-options Cons none "GIGABYTE's MZ31-AR0 offers nearly endless build options on a fair budget. Who wouldn't want a system built around this motherboard? Or maybe two? Or three? I know I do!" Thank You, GIGABYTE, for providing us with this review sample.