GPU Benchmarking

Within the PoA's health check, the drill test incorporates sophisticated benchmarking techniques such as MLPerf to evaluate machine performance comprehensively. By conducting benchmarking assessments, including MLPerf, the algorithm quantifies the machine's efficiency. This quantitative measure serves as a reliable indicator of the machine's condition, ensuring robustness and reliability in its operational capabilities.

Work flow of Drill Test

Here are some sample results of the drill test on Nvidia A100:

MLPerf Results Summary:

SUT name : BERT SERVER

Scenario : Offline

Mode : PerformanceOnly

Samples per second: 1532.17

Result is : VALID

Min duration satisfied : Yes

Min queries satisfied : Yes

Early stopping satisfied: Yes

Additional Stats:

Min latency (ns) : 3559383281

Max latency (ns) : 1292280950807

Mean latency (ns) : 788846755872

50.00 percentile latency (ns) : 840201049914

90.00 percentile latency (ns) : 1234598190171

95.00 percentile latency (ns) : 1268998116410

97.00 percentile latency (ns) : 1280065956777

99.00 percentile latency (ns) : 1289280826440

99.90 percentile latency (ns) : 1292043266934

Test Parameters Used:

samples_per_query : 1980000

target_qps : 3000

target_latency (ns): 0

max_async_queries : 1

min_duration (ms): 600000

max_duration (ms): 0

min_query_count : 1

max_query_count : 0

qsl_rng_seed : 13281865557512327830

sample_index_rng_seed : 198141574272810017

schedule_rng_seed : 7575108116881280410

accuracy_log_rng_seed : 0

accuracy_log_probability : 0

accuracy_log_sampling_target : 0

print_timestamps : 0

performance_issue_unique : 0

performance_issue_same : 0

performance_issue_same_index : 0

performance_sample_count : 10833