Standard deviation and average are poor statistical measures of latency - Journal of Omnifarious
Nov. 24th, 2008
10:39 am - Standard deviation and average are poor statistical measures of latency
I've noticed that ping and a few other similar utilities that measure network latency have begun to include an interesting statistic. They show the standard deviation of all the latencies gathered from each individual ping packet. I think this is bad statistics.
If I am not mistaken standard deviation is based on the idea that your sample set follows a normal distribution, a bell-curve. Network ping times do not follow this distribution. I would guess that network ping times follow a power-law curve in which the majority of ping times are hover just above the theoretical minimum value for the path with increasingly rare outliers arbitrarily far from that value.
It would be nice to have some sort of statistical measure that more accurately reflected this measure. Perhaps something like a measure of how shallow the curve was. The shallower it is, the more uncertainty there is.
That also means the the mean ping time is also a poor measure. There should be some measure of a power law curve where you can guess that 50% of the values would be below and 50% would be above.
The reason I'm guessing that ping times follow a power law curve is that I remember seeing research showing that measuring network traffic bursts showed that network traffic burstiness displayed scale invariant properties. That basically a measure of traffic spikes looked approximately the same at almost any scale you wanted to examine. Scale invariance, fractal patterns and power law curves are strongly related.
And this brings to mind another issue. Given the widespread applicability of Benford's Law, it's clear the scale invariance is a property of many statistical sample sets. Yet it seems that standard bell-curve distributions are considered the default. IMHO, power law curve based statistics are what should be taught in High School, not the mean/mode/median/standard-deviation 'normal distribution' based statistics that are currently taught.
Incidentally, the widespread applicability of Benford's Law also lends even more support to the already overwhelming evidence that scale invariance is the default property of almost any network, a hypothesis that is thoroughly explored in Linked: The New Science of Networks.