Re: [scala-user] Determining if actors really multi-thread

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [scala-user] Determining if actors really multi-thread

James Iry-2
Not so, if you do a bunch of blocking IO sequentially then you never have 2000 threads tying up system resources.   But I suspect that wasn't what you meant.  And I'm willing to give the benefit of the doubt that you meant "any attempt to use blocking IO using multiple threads."

But I'm surprised you think it's irrelevant.  One of the eplicit goals of both the FJ framework and the actors library is to keep thread usage down.  Work stealing is a huge part of why these libraries are as complicated as they are.  If it was irrelevant, the libraries would be much easier.  An easy design for actors is to tie an actor to a thread.  FJ could have been done with a traditional mechanism of putting each task on a thread from a thread pool.

My point all along was that not only are threads doing nothing and thus, as you say, runnable tasks are sitting idle.  But also, importantly and relevantly, because of the thread pool growth heuristic you're causing the libraries to fail in the design goal of minimizing idle threads.

On Mon, Dec 15, 2008 at 5:42 PM, David MacIver <[hidden email]> wrote:

What you describe is a problem with any attempt to use blocking IO.
 
What I am describing is a problem with using blocking IO in forkjoin tasks.

Therefore I consider the problem I describe to be a reason blocking IO is recommended against in forkjoin tasks and the problem you describe to be an irrelevancy.

Reply | Threaded
Open this post in threaded view
|

Re: Re: [scala-user] Determining if actors really multi-thread

Derek Chen-Becker-2
OK, so I think I have a better understanding of how Actors work, but I'm
still not sure if Actors are appropriate for what I'm doing. Essentially
I have 3000 tasks that run once (no event loop) that use blocking I/O
over the network, and so have quite a bit of variation in their latency.
Right now, I can use the ExecutorService in java.util.concurrent
essentially as a task queue to set up the 3000 tasks, let them run and
then call shutdown on the ExecutorService at the end to wait for all of
the tasks to complete. As each task completes, it adds a message object
to a blocking queue, which a single thread pulls from to update a
database. I can easily see how Actors work in the latter half of this,
but I'm still not sure what the most appropriate method would be for the
first half. Here are the options as I see them:

1. Create one Actor for the tasks and submit 3000 messages. This has a
big drawback that the Actor only runs on one thread at a time, so with
the blocking I/O it could take a very long time to complete.

2. Create 3000 Actors, one for each task. Simple, but how much overhead
is there for an Actor instance? How far does this scale?

3. Create N Actors and divvy up 3000/N tasks to each one. Not exactly a
fan of this one since it requires manual partitioning of the tasks.

4. Just leave my code the way it is. It works fine now, so I don't
necessarily need to change it.

I really appreciate everyone's feedback on this so far. I definitely
understand Actors better now.

Thanks,

Derek

dpp
Reply | Threaded
Open this post in threaded view
|

Re: Re: [scala-user] Determining if actors really multi-thread

dpp

Derek,

There's a library that bridges between Mina (an nio library) and actors.  It might be just what you need.

Thanks.

David

On Dec 16, 2008 7:06 AM, "Derek Chen-Becker" <[hidden email]> wrote:

OK, so I think I have a better understanding of how Actors work, but I'm
still not sure if Actors are appropriate for what I'm doing. Essentially
I have 3000 tasks that run once (no event loop) that use blocking I/O
over the network, and so have quite a bit of variation in their latency.
Right now, I can use the ExecutorService in java.util.concurrent
essentially as a task queue to set up the 3000 tasks, let them run and
then call shutdown on the ExecutorService at the end to wait for all of
the tasks to complete. As each task completes, it adds a message object
to a blocking queue, which a single thread pulls from to update a
database. I can easily see how Actors work in the latter half of this,
but I'm still not sure what the most appropriate method would be for the
first half. Here are the options as I see them:

1. Create one Actor for the tasks and submit 3000 messages. This has a
big drawback that the Actor only runs on one thread at a time, so with
the blocking I/O it could take a very long time to complete.

2. Create 3000 Actors, one for each task. Simple, but how much overhead
is there for an Actor instance? How far does this scale?

3. Create N Actors and divvy up 3000/N tasks to each one. Not exactly a
fan of this one since it requires manual partitioning of the tasks.

4. Just leave my code the way it is. It works fine now, so I don't
necessarily need to change it.

I really appreciate everyone's feedback on this so far. I definitely
understand Actors better now.

Thanks,

Derek

Reply | Threaded
Open this post in threaded view
|

Re: Re: [scala-user] Determining if actors really multi-thread

David MacIver
In reply to this post by James Iry-2
On Tue, Dec 16, 2008 at 10:05 AM, Derek Chen-Becker <[hidden email]> wrote:
OK, so I think I have a better understanding of how Actors work, but I'm
still not sure if Actors are appropriate for what I'm doing. Essentially
I have 3000 tasks that run once (no event loop) that use blocking I/O
over the network, and so have quite a bit of variation in their latency.
Right now, I can use the ExecutorService in java.util.concurrent
essentially as a task queue to set up the 3000 tasks, let them run and
then call shutdown on the ExecutorService at the end to wait for all of
the tasks to complete. As each task completes, it adds a message object
to a blocking queue, which a single thread pulls from to update a
database. I can easily see how Actors work in the latter half of this,
but I'm still not sure what the most appropriate method would be for the
first half. Here are the options as I see them:

1. Create one Actor for the tasks and submit 3000 messages. This has a
big drawback that the Actor only runs on one thread at a time, so with
the blocking I/O it could take a very long time to complete.

2. Create 3000 Actors, one for each task. Simple, but how much overhead
is there for an Actor instance? How far does this scale?

3. Create N Actors and divvy up 3000/N tasks to each one. Not exactly a
fan of this one since it requires manual partitioning of the tasks.

4. Just leave my code the way it is. It works fine now, so I don't
necessarily need to change it.

My vote would be 4. It sounds like you could make this work with actors but that you wouldn't receive a massive benefit from doing so.