Quantcast

Strange benchmark result for Array.iterator

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Strange benchmark result for Array.iterator

Alexandru Nedelcu-4
Hi folks,

Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:

[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s

Details:

- params: -i 10 -wi 10 -f 2 -t 1

So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.

I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.

This can't be possible. So what is wrong in this benchmark?

--
Alexandru Nedelcu
alexn.org

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Aw: [scala-user] Strange benchmark result for Array.iterator

Dennis Haupt-2
i would guess the VM is optimizing things during the execution of your array benchmark. 
 
try running all the tests multiple times instead of just once or change the order of the tests. the first one should always be extremely slow
 
Gesendet: Dienstag, 14. März 2017 um 10:02 Uhr
Von: "Alexandru Nedelcu" <[hidden email]>
An: scala-user <[hidden email]>
Betreff: [scala-user] Strange benchmark result for Array.iterator
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu
alexn.org

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange benchmark result for Array.iterator

Viktor Klang
In reply to this post by Alexandru Nedelcu-4
Guess: Boxing vs already-boxed.

--
Cheers,

On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,

Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:

[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s

Details:

- params: -i 10 -wi 10 -f 2 -t 1

So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.

I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.

This can't be possible. So what is wrong in this benchmark?

--
Alexandru Nedelcu

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Aw: Re: [scala-user] Strange benchmark result for Array.iterator

Dennis Haupt-2
in that case using the array iterator should also be slow, which it isn't
 
Gesendet: Dienstag, 14. März 2017 um 15:21 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Alexandru Nedelcu" <[hidden email]>
Cc: scala-user <[hidden email]>
Betreff: Re: [scala-user] Strange benchmark result for Array.iterator
Guess: Boxing vs already-boxed.
 
--
Cheers,
 
On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: [scala-user] Strange benchmark result for Array.iterator

Viktor Klang
arrayIter is the slow one.

On Tue, Mar 14, 2017 at 6:13 PM, Dennis Haupt <[hidden email]> wrote:
in that case using the array iterator should also be slow, which it isn't
 
Gesendet: Dienstag, 14. März 2017 um 15:21 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Alexandru Nedelcu" <[hidden email]>
Cc: scala-user <[hidden email]>
Betreff: Re: [scala-user] Strange benchmark result for Array.iterator
Guess: Boxing vs already-boxed.
 
--
Cheers,
 
On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--
Cheers,

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Aw: Re: [scala-user] Strange benchmark result for Array.iterator

Dennis Haupt-2
In reply to this post by Dennis Haupt-2
i removed the @benchmark thingies so everybody can run it directly:
 
https://gist.github.com/HamsterofDeath/2e455d0909ff4ac080c3a1895eeaa034
 
results are as expected (if you expect the VM to take a random pause for optimizing the code on the fly and from then on random pauses for GC in the iterator runs)
 
run 1
array took 5.54 ms
array iterator took 25.19 ms
list took 8.96 ms
list iterator  took 11.20 ms
 
run 2
array took 0.69 ms
array iterator took 15.56 ms
list took 5.17 ms
list iterator  took 5.75 ms
 
run 3
array took 0.69 ms
array iterator took 11.12 ms
list took 6.40 ms
list iterator  took 7.10 ms
 
run 4
array took 0.69 ms
array iterator took 7.45 ms
list took 6.42 ms
list iterator  took 6.79 ms
 

 
Gesendet: Dienstag, 14. März 2017 um 18:13 Uhr
Von: "Dennis Haupt" <[hidden email]>
An: "Viktor Klang" <[hidden email]>
Cc: "Alexandru Nedelcu" <[hidden email]>, scala-user <[hidden email]>
Betreff: Aw: Re: [scala-user] Strange benchmark result for Array.iterator
in that case using the array iterator should also be slow, which it isn't
 
Gesendet: Dienstag, 14. März 2017 um 15:21 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Alexandru Nedelcu" <[hidden email]>
Cc: scala-user <[hidden email]>
Betreff: Re: [scala-user] Strange benchmark result for Array.iterator
Guess: Boxing vs already-boxed.
 
--
Cheers,
 
On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Aw: Re: Re: [scala-user] Strange benchmark result for Array.iterator

Dennis Haupt-2
In reply to this post by Viktor Klang
looks like i misinterpreted the numbers.
the one sticking out was the first test, so i assumed blindly that it was the problematic one and that 50k was the time in ms or something.
but given the new insight, isn't the performance always terrible compared to the raw array access? we're talking about factors 10-20 here
 
 
 
Gesendet: Dienstag, 14. März 2017 um 18:17 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Dennis Haupt" <[hidden email]>
Cc: "Alexandru Nedelcu" <[hidden email]>, scala-user <[hidden email]>
Betreff: Re: Re: [scala-user] Strange benchmark result for Array.iterator
arrayIter is the slow one.
 
On Tue, Mar 14, 2017 at 6:13 PM, Dennis Haupt <[hidden email]> wrote:
in that case using the array iterator should also be slow, which it isn't
 
Gesendet: Dienstag, 14. März 2017 um 15:21 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Alexandru Nedelcu" <[hidden email]>
Cc: scala-user <[hidden email]>
Betreff: Re: [scala-user] Strange benchmark result for Array.iterator
Guess: Boxing vs already-boxed.
 
--
Cheers,
 
On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
 
 
--
Cheers,

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [scala-user] Strange benchmark result for Array.iterator

Viktor Klang

On Tue, Mar 14, 2017 at 6:30 PM, Dennis Haupt <[hidden email]> wrote:
looks like i misinterpreted the numbers.
the one sticking out was the first test, so i assumed blindly that it was the problematic one and that 50k was the time in ms or something.
but given the new insight, isn't the performance always terrible compared to the raw array access? we're talking about factors 10-20 here
 
 
 
Gesendet: Dienstag, 14. März 2017 um 18:17 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Dennis Haupt" <[hidden email]>
Cc: "Alexandru Nedelcu" <[hidden email]>, scala-user <[hidden email]>
Betreff: Re: Re: [scala-user] Strange benchmark result for Array.iterator
arrayIter is the slow one.
 
On Tue, Mar 14, 2017 at 6:13 PM, Dennis Haupt <[hidden email]> wrote:
in that case using the array iterator should also be slow, which it isn't
 
Gesendet: Dienstag, 14. März 2017 um 15:21 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Alexandru Nedelcu" <[hidden email]>
Cc: scala-user <[hidden email]>
Betreff: Re: [scala-user] Strange benchmark result for Array.iterator
Guess: Boxing vs already-boxed.
 
--
Cheers,
 
On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
 
 
--
Cheers,

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--
Cheers,

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: [scala-user] Strange benchmark result for Array.iterator

Viktor Klang
Managed to overcome the want to not spoil the revelation.

array took 1.34 ms
array iterator took 34.56 ms
array better iterator took 23.30 ms
preboxed array took 4.06 ms
preboxed array iterator took 26.87 ms
preboxed array better iterator took 2.89 ms
list took 20.66 ms
list iterator  took 17.10 ms

On Tue, Mar 14, 2017 at 7:02 PM, Viktor Klang <[hidden email]> wrote:

On Tue, Mar 14, 2017 at 6:30 PM, Dennis Haupt <[hidden email]> wrote:
looks like i misinterpreted the numbers.
the one sticking out was the first test, so i assumed blindly that it was the problematic one and that 50k was the time in ms or something.
but given the new insight, isn't the performance always terrible compared to the raw array access? we're talking about factors 10-20 here
 
 
 
Gesendet: Dienstag, 14. März 2017 um 18:17 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Dennis Haupt" <[hidden email]>
Cc: "Alexandru Nedelcu" <[hidden email]>, scala-user <[hidden email]>
Betreff: Re: Re: [scala-user] Strange benchmark result for Array.iterator
arrayIter is the slow one.
 
On Tue, Mar 14, 2017 at 6:13 PM, Dennis Haupt <[hidden email]> wrote:
in that case using the array iterator should also be slow, which it isn't
 
Gesendet: Dienstag, 14. März 2017 um 15:21 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Alexandru Nedelcu" <[hidden email]>
Cc: scala-user <[hidden email]>
Betreff: Re: [scala-user] Strange benchmark result for Array.iterator
Guess: Boxing vs already-boxed.
 
--
Cheers,
 
On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
 
 
--
Cheers,

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--
Cheers,



--
Cheers,

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange benchmark result for Array.iterator

Seth Tisue-3
In reply to this post by Alexandru Nedelcu-4
reminder: there is JMH-based benchmarking infra and examples in https://github.com/scala/scala/tree/2.12.x/test/benchmarks . if we're all using the same benchmarking code it's a lot easier to have confidence that we're getting meaningful results

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange benchmark result for Array.iterator

Alexandru Nedelcu-4
In reply to this post by Viktor Klang
Hey guys, thanks for the input.

Totally forgot to take boxing into account.

--
Alexandru Nedelcu
alexn.org



On Tue, Mar 14, 2017, at 20:45, Viktor Klang wrote:
Managed to overcome the want to not spoil the revelation.

array took 1.34 ms
array iterator took 34.56 ms
array better iterator took 23.30 ms
preboxed array took 4.06 ms
preboxed array iterator took 26.87 ms
preboxed array better iterator took 2.89 ms
list took 20.66 ms
list iterator  took 17.10 ms

On Tue, Mar 14, 2017 at 7:02 PM, Viktor Klang <[hidden email]> wrote:

On Tue, Mar 14, 2017 at 6:30 PM, Dennis Haupt <[hidden email]> wrote:
looks like i misinterpreted the numbers.
the one sticking out was the first test, so i assumed blindly that it was the problematic one and that 50k was the time in ms or something.
but given the new insight, isn't the performance always terrible compared to the raw array access? we're talking about factors 10-20 here
 
 
 
Gesendet: Dienstag, 14. März 2017 um 18:17 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Dennis Haupt" <[hidden email]>
Cc: "Alexandru Nedelcu" <[hidden email]>, scala-user <[hidden email]>
Betreff: Re: Re: [scala-user] Strange benchmark result for Array.iterator
arrayIter is the slow one.
 
On Tue, Mar 14, 2017 at 6:13 PM, Dennis Haupt <[hidden email]> wrote:
in that case using the array iterator should also be slow, which it isn't
 
Gesendet: Dienstag, 14. März 2017 um 15:21 Uhr
Von: "Viktor Klang" <[hidden email]>
An: "Alexandru Nedelcu" <[hidden email]>
Cc: scala-user <[hidden email]>
Betreff: Re: [scala-user] Strange benchmark result for Array.iterator
Guess: Boxing vs already-boxed.
 
--
Cheers,

 
On Mar 14, 2017 10:02 AM, "Alexandru Nedelcu" <[hidden email]> wrote:
Hi folks,
 
Doing some benchmarks I discovered these really strange results. Unless I'm doing something really wrong, the performance of Array iterators is absolutely terrible:
 
[info] IteratorBenchmark.array      thrpt   20  50151.919 ± 382.623  ops/s
[info] IteratorBenchmark.arrayIter  thrpt   20   3693.442 ±  27.582  ops/s
[info] IteratorBenchmark.list       thrpt   20   6085.927 ± 650.919  ops/s
[info] IteratorBenchmark.listIter   thrpt   20   5596.069 ± 376.200  ops/s
 
Details:
 
- params: -i 10 -wi 10 -f 2 -t 1
 
So the first benchmark is for iterating an array with a simple while loop. The second benchmark uses `array.iterator`. The third iterates over a Scala List using a simple while loop, while the fourth iterates using its iterator.
 
I realize that benchmarking array access is hard due to the compiler, runtime and CPU optimizing array access, which is why the first result isn't so surprizing, but what is surprising is that it is worse than iterating over a List. Worse still, it has lower throughput than iterating over a List.iterator.
 
This can't be possible. So what is wrong in this benchmark?
 
--
Alexandru Nedelcu

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
 
 
--
Cheers,

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--

Cheers,






--

Cheers,



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Loading...