Microsoft Azure is dedicated to offering its clients with industry-leading real-world AI capabilities. In December 2021, Microsoft Azure debuted its management efficiency with the MLPerf coaching v1.1 outcomes. Azure debuted at primary amongst cloud suppliers and quantity two general at scale amongst all submitters. Azure’s supercomputer’s constructing blocks had been used to generate the leads to our v2.zero submissions for the MLPerf inferencing outcomes printed on April 6, 2022.
These industry-leading outcomes are pushed by Microsoft’s publicly obtainable supercomputing capabilities designed for real-world AI inferencing workloads. Microsoft allows clients of all scales to deploy highly effective AI options, whether or not at a centered native scale or on the scale of the biggest supercomputers on the earth.
Microsoft Azure’s publicly obtainable AI inferencing capabilities are led by the NDm A100 v4, ND A100 v4, and NC A100 v4 digital machines (VMs) which might be powered by NVIDIA A100 SXM and PCIe Tensor Core graphics processing items (GPUs). These outcomes showcase Azure’s dedication to creating AI inferencing obtainable to all in essentially the most accessible method—whereas elevating the bar for AI inferencing in Azure.
In our quest to repeatedly present the very best expertise for our clients, Azure has not too long ago introduced the preview for the NC A100 v4. With this introduction of the NC A100 v4 sequence, we have now supplied our clients with three totally different VM sizes starting from one to 4 GPUs. From our benchmarking, we have now seen greater than two instances efficiency over the earlier technology. Azure’s clients can get entry to those new methods at the moment by signing up for the preview program.
Some highlights for this spherical of MLPerf inferencing submissions will be seen within the following tables.
Highlights from the outcomes
ND96amsr A100 v4 powered by NVIDIA A100 80G SXM Tensor Core GPU
| Benchmark | Samples/second | Queries/second | Eventualities |
| bert-99 | 27,500 plus | ~22,500 plus | Offline and server |
| resnet | 300,000 plus | ~200,000 plus | Offline and server |
| 3d-unet | 24.87 | Offline |
NC96advertisements A100 v4 powered by NVIDIA A100 80G PCIe Tensor Core GPU
| Benchmark | Samples/second | Queries/second | Eventualities |
| bert-99 | ~6,300 | ~5,300 | Offline and server |
| resnet | 144,000 | ~119,600 | Offline and server |
| 3d-unet | 11.7 | Offline |
The above tables showcase three of the six benchmarks the workforce ran utilizing NVIDIA A100 SXM and PCIe Tensor Core GPUs for offline and server eventualities respectively. Check out the complete record of outcomes for the assorted divisions.
Azure works intently with NVIDIA
The outcomes had been generated by deploying the surroundings utilizing the VM choices and Azure’s Ubuntu 18.04-HPC market picture. We labored intently with NVIDIA to rapidly deploy the surroundings and carry out benchmarks with industry-leading leads to efficiency and scalability.
These outcomes are a testomony to Azure’s give attention to providing scalable supercomputing for any workload whereas enabling our clients to make the most of “on-demand” supercomputing capabilities within the cloud to unravel their most advanced issues. Go to the Azure Tech Group weblog to learn the steps to breed the outcomes.
Extra about MLPerf
MLPerf is a consortium of AI leaders from academia, analysis labs, and industry the place the mission is to “construct honest and helpful benchmarks” that present unbiased evaluations of coaching and inference efficiency for hardware, software program, and companies—all carried out beneath prescribed situations. To remain on the chopping fringe of industry developments, MLPerf continues to evolve, holding new exams at common intervals and including new workloads that symbolize state-of-the-art AI. MLPerf’s exams are clear and goal, so customers can depend on the outcomes to make knowledgeable shopping for selections. The industry benchmarking group, fashioned in Might 2018, is backed by dozens of industry leaders. The benchmark exams throughout inferencing are more and more changing into the important thing exams that hardware and software program distributors use to exhibit efficiency. Check out the complete record of outcomes for MLPerf Inference v2.zero.
