by Steve Rumsby, VIAVI
AI workloads, particularly large‑scale training and distributed inference, place unique demands on data center fabrics. While traditional network test metrics such as throughput, average latency, and packet loss are still necessary, they are not sufficient to characterize real AI performance.
A key reason for this is the straggler effect, where the synchronous nature of AI training means that P99 or P99.9 outliers can have an undue effect on the east-west sync and burst traffic.
As a result, a delayed packet or slow communication flow can stall synchronization across thousands of GPUs, extending completion time for the entire workload.
Traditional metrics struggle to quantify this effect, and job completion time (JCT) has emerged as a key KPI for AI systems. Indeed, it is now seen by many as the most meaningful performance metric for AI fabrics, ahead of even raw link‑level metrics.
In this article, we’ll look at why this is, and the implications of switching to methodologies that use it.
Why traditional network KPIs fail for AI
Before AI, enterprise networks tended to be transactional, with request/response workloads. KPIs therefore focused on ways to monitor the network’s effectiveness for these workloads, with metrics such as peak throughput, average latency, jitter, and packet loss becoming the de facto measurements taken.
However, in AI, these KPIs (while still valuable) by themselves are ill-suited to predicting AI job performance.
The traffic of AI workloads is hugely different, generating highly synchronized and bursty flows, with east-west data transfers happening in a many-to-many pattern. This creates significant stress for buffers, congestion control, and collective communication mechanisms that these traditional KPIs simply do not highlight.
Numerous industry studies have shown that these metrics often fail to expose tail-latency, congestion, and synchronization effects that materially impact AI training performance.
Job completion time
A better view can be obtained by tracking the total elapsed time from the AI job being admitted to the system through to its successful completion with the final output or checkpoint.
This is Job Completion Time, or JCT, and while it cannot be described as a pure compute metric, in large-scale AI training it will take into consideration and measure a range of essential factors. These include data staging and ingestion, distributed compute execution, the efficiency of collective communication operations (such as all-reduce vs. all-gather), synchronization barriers, and checkpointing and storage interactions.
In large-scale distributed training environments, JCT is often heavily influenced by network behavior because communication phases must complete before computation can continue. The reason for this is that AI workloads are extremely sensitive to factors such as tail latency, incast congestion, microbursts, buffer pressure, and load imbalance across paths.
When measured only using traditional isolated link metric approaches, all of the above will have merely negligible effects, as these look at the average latency or throughput. But the effect on the network will be huge as they force costly GPUs to idle while waiting for straggling communications.
By measuring the complete start-to-finish time, we instead get to see the full picture, with JCT shown to be a direct correlative indicator of several vital outcomes, including time-to-model, time-to-insight, cost per training run, and cluster usage efficiency.
As such, modern industry test frameworks explicitly identify JCT, alongside throughput and latency, for use in validating AI data center fabrics.
Implications of using JCT as a network KPI
Before we examine how to implement JCT as a KPI within an AI network’s test methodology, we should quickly state that modern testing focuses on emulating real AI workloads, including synchronized GPU communication patterns, rather than using purely synthetic traffic streams.
As such, JCT can be implemented readily and will accurately align network testing with AI workload reality. And in medium- to large AI clusters where behavior is often the dominant variable, JCT becomes a composite KPI that reflects congestion management effectiveness, as well as a validation metric for lossless or near-lossless Ethernet fabrics. Doing so also gives a means to compare (and effectively A-B test) architectural choices, for example Ethernet vs. InfiniBand, or scale-up vs. scale-out fabrics.
But including JCT within the testing methodology does come with several implications. Firstly, there is the implicit shift from link-level to job-level validation. Here we gain the benefit of shifting from asking how fast an individual link is, to instead asking how long a specific task takes to finish.
This is a far more pertinent question, but asking it requires that the testing environment be capable of accurately emulating collective communication patterns, introducing realistic synchronization points, and accurately stressing congestion control mechanisms under AI-like load.
Secondly, it is not enough to simply observe that a job ran slowly. Instead, AI testing needs to track and correlate network behaviors so engineers can identify which behaviors actively inflate JCT.
Together, these diagnostic KPIs help distinguish whether JCT inflation is caused by congestion, load imbalance, transport inefficiencies, or fabric-level design limitations. These should be considered the minimum, rather than an exhaustive list.
Taking this approach will enable a true apples-to-apples comparison across the network, especially when seeking to measure and/or compare changes in congestion control schemes, switch architectures, fabric topologies, and transport protocols.
JCT within the test framework
By implementing this layered KPI model, engineering teams can evaluate end-to-end workload outcomes rather than isolated component performance. This creates a direct line of sight between network design decisions and measurable AI infrastructure efficiency.
But while JCT should be considered the primary KPI, it should not be used as the only KPI. Traditional metrics, such as link latency, throughput, and loss, remain valid, explaining why JCT changes. JCT instead explains whether specific changes matter.
From network metrics to AI outcomes
To align network engineering with business outcomes, modern AI network testing methodologies must be redesigned around job-level validation, rather than the more simplistic packet-level metrics used for classical networks.
For this reason, JCT is increasingly becoming the defining KPI for AI fabric validation. Combined with traditional link-level and diagnostic metrics, it provides both a measure of real workload performance and a framework for identifying the network behaviors that influence it most.