criterion performance measurements

overview

want to understand this report?

bytestring-read/short

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.871990513902483e-7 3.9031477517761356e-7 3.9286803745730374e-7
Standard deviation 8.551939657211476e-9 9.97092154549203e-9 1.18101696922444e-8

Outlying measurements have moderate (0.35393137279368686%) effect on estimated standard deviation.

bytestring-read/long

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0686758573931825e-6 1.0723947372418401e-6 1.0771486028883048e-6
Standard deviation 1.144862828565305e-8 1.4065066700795679e-8 1.977759665797805e-8

Outlying measurements have moderate (0.11717920221630587%) effect on estimated standard deviation.

bytestring-read(Lazy)/short

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.7805978186745114e-7 3.8160236646923135e-7 3.8401049950293446e-7
Standard deviation 7.204917149657172e-9 9.801250527953904e-9 1.5822210588421482e-8

Outlying measurements have moderate (0.3580828655056597%) effect on estimated standard deviation.

bytestring-read(Lazy)/long

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1183901567294006e-6 1.133588220209855e-6 1.1449907244543843e-6
Standard deviation 3.259245418355493e-8 4.5024080502700415e-8 5.73107422181539e-8

Outlying measurements have severe (0.5494867224326775%) effect on estimated standard deviation.

text/short

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.930886613294211e-6 1.9518169343387877e-6 1.968394028243429e-6
Standard deviation 4.5462353576988114e-8 5.8823577206724994e-8 7.724078534664008e-8

Outlying measurements have moderate (0.39640478239024646%) effect on estimated standard deviation.

text/long

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.146947272177313e-5 2.1640820884982994e-5 2.1812504299087905e-5
Standard deviation 4.6656808739144735e-7 5.992752110459349e-7 8.65696945863067e-7

Outlying measurements have moderate (0.2913262130533435%) effect on estimated standard deviation.

bytestring-lexing/short

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.5563088684618697e-7 4.5869478026091517e-7 4.6190834134970195e-7
Standard deviation 8.504071760784233e-9 1.0173572244985155e-8 1.2165381489633621e-8

Outlying measurements have moderate (0.2910364617622874%) effect on estimated standard deviation.

bytestring-lexing/long

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.6668481925482002e-6 3.6865652316334563e-6 3.707939711787143e-6
Standard deviation 5.228725779815216e-8 6.882608089738974e-8 9.285851291740723e-8

Outlying measurements have moderate (0.19030727137747097%) effect on estimated standard deviation.

attoparsec/short

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.6338644678388995e-6 4.697196672784298e-6 4.762070972359596e-6
Standard deviation 1.8997350017632536e-7 2.1464929244854916e-7 2.464505160201899e-7

Outlying measurements have severe (0.5828289246255699%) effect on estimated standard deviation.

attoparsec/long

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.8893666021040232e-5 2.904531608702304e-5 2.9150802954544414e-5
Standard deviation 2.82194140697655e-7 4.1149806826572184e-7 6.101645542854993e-7

Outlying measurements have slight (9.326277069336177e-2%) effect on estimated standard deviation.

read/short

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.607987059017719e-5 1.6246042175777217e-5 1.641152669475765e-5
Standard deviation 5.067074461469622e-7 5.731979739249643e-7 6.835411916106077e-7

Outlying measurements have moderate (0.4136010909819513%) effect on estimated standard deviation.

read/long

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.153022782480029e-5 6.202506718700818e-5 6.244306727791669e-5
Standard deviation 1.0392335589119437e-6 1.5272934899343277e-6 2.2187291810301526e-6

Outlying measurements have moderate (0.2177662453113236%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.