Quantcast

Old-style array operations

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Old-style array operations

Tim P
Hi
I've got to partition a big array of 2d points (big as in about 40m of them) into n rectangles. That's a lot of data and I want to do it in place.
Now I can do this easily enough with java.util.Arrays which allows me to sort sub-sets of the data. But I was just curious as to whether there is any scala equivalent?

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Old-style array operations

Hugo Ferreira
Hello,

How about ?

* https://github.com/scalanlp/breeze/wiki
* https://github.com/non/spire
* http://nd4j.org/scala.html
  (see also nd4s)

You may also consider other Java matrix libraries (Colt, Jama).

HTHs,



On Monday, 6 February 2017 09:55:55 UTC, Tim P wrote:
Hi
I've got to partition a big array of 2d points (big as in about 40m of them) into n rectangles. That's a lot of data and I want to do it in place.
Now I can do this easily enough with java.util.Arrays which allows me to sort sub-sets of the data. But I was just curious as to whether there is any scala equivalent?

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Old-style array operations

Rex Kerr-2
In reply to this post by Tim P
I generally use java.util.Arrays if it's possible/convenient.  There's no advantage to avoiding it if it does just what you want and you're running on the JVM.

  --Rex

On Mon, Feb 6, 2017 at 1:55 AM, Tim P <[hidden email]> wrote:
Hi
I've got to partition a big array of 2d points (big as in about 40m of them) into n rectangles. That's a lot of data and I want to do it in place.
Now I can do this easily enough with java.util.Arrays which allows me to sort sub-sets of the data. But I was just curious as to whether there is any scala equivalent?

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Old-style array operations

Tim P
Thanks Rex,
Having looked at the others mentioned I came to the conclusion that would be the simplest approach

On 6 February 2017 at 14:37, Rex Kerr <[hidden email]> wrote:
I generally use java.util.Arrays if it's possible/convenient.  There's no advantage to avoiding it if it does just what you want and you're running on the JVM.

  --Rex

On Mon, Feb 6, 2017 at 1:55 AM, Tim P <[hidden email]> wrote:
Hi
I've got to partition a big array of 2d points (big as in about 40m of them) into n rectangles. That's a lot of data and I want to do it in place.
Now I can do this easily enough with java.util.Arrays which allows me to sort sub-sets of the data. But I was just curious as to whether there is any scala equivalent?

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

For more options, visit https://groups.google.com/d/optout.




--
Tim Pigden
Optrak Distribution Software Limited
+44 (0)1992 517100
http://www.linkedin.com/in/timpigden
http://optrak.com
Optrak Distribution Software Ltd is a limited company registered in England and Wales.
Company Registration No. 2327613 Registered Offices: Suite 6,The Maltings, Hoe Lane, Ware, SG12 9LR England 
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Optrak Distribution Software Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Old-style array operations

Oliver Ruebenacker

     Hello,

  You don't need to sort them if all you want to do is partition, right?

  Let's say you want to split into two partitions. Scan from the beginning of the array for elements that do not belong to the first partition while scanning backwards from the end for elements that do not belong into the second partition, swapping them as you go, until you meet somewhere.

  For more than two partitions, bisect successively.

     Best, Oliver

On Mon, Feb 6, 2017 at 9:44 AM, Tim Pigden <[hidden email]> wrote:
Thanks Rex,
Having looked at the others mentioned I came to the conclusion that would be the simplest approach

On 6 February 2017 at 14:37, Rex Kerr <[hidden email]> wrote:
I generally use java.util.Arrays if it's possible/convenient.  There's no advantage to avoiding it if it does just what you want and you're running on the JVM.

  --Rex

On Mon, Feb 6, 2017 at 1:55 AM, Tim P <[hidden email]> wrote:
Hi
I've got to partition a big array of 2d points (big as in about 40m of them) into n rectangles. That's a lot of data and I want to do it in place.
Now I can do this easily enough with java.util.Arrays which allows me to sort sub-sets of the data. But I was just curious as to whether there is any scala equivalent?

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

For more options, visit https://groups.google.com/d/optout.




--
Tim Pigden
Optrak Distribution Software Limited
<a href="tel:+44%201992%20517100" value="+441992517100" target="_blank">+44 (0)1992 517100
http://www.linkedin.com/in/timpigden
http://optrak.com
Optrak Distribution Software Ltd is a limited company registered in England and Wales.
Company Registration No. 2327613 Registered Offices: Suite 6,The Maltings, Hoe Lane, Ware, SG12 9LR England 
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Optrak Distribution Software Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--
Oliver Ruebenacker
Senior Software Engineer, Diabetes Portal, Broad Institute

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Old-style array operations

Tim P
Hmm - not quite. I want to split into approximately equal sized rectangles by number of points in each sub-rectangle. So I was thinking to sort them at each iteration since then I'd guarantee the median to make the split.
But instead of sorting them I can take a random sample, find the median of the random sample and split on that - so then yes - I can partition about the median of the random sample for an approximation. As indeed Rex pointed out way back in 2010 :-) as I just discovered:



On 6 February 2017 at 14:52, Oliver Ruebenacker <[hidden email]> wrote:

     Hello,

  You don't need to sort them if all you want to do is partition, right?

  Let's say you want to split into two partitions. Scan from the beginning of the array for elements that do not belong to the first partition while scanning backwards from the end for elements that do not belong into the second partition, swapping them as you go, until you meet somewhere.

  For more than two partitions, bisect successively.

     Best, Oliver

On Mon, Feb 6, 2017 at 9:44 AM, Tim Pigden <[hidden email]> wrote:
Thanks Rex,
Having looked at the others mentioned I came to the conclusion that would be the simplest approach

On 6 February 2017 at 14:37, Rex Kerr <[hidden email]> wrote:
I generally use java.util.Arrays if it's possible/convenient.  There's no advantage to avoiding it if it does just what you want and you're running on the JVM.

  --Rex

On Mon, Feb 6, 2017 at 1:55 AM, Tim P <[hidden email]> wrote:
Hi
I've got to partition a big array of 2d points (big as in about 40m of them) into n rectangles. That's a lot of data and I want to do it in place.
Now I can do this easily enough with java.util.Arrays which allows me to sort sub-sets of the data. But I was just curious as to whether there is any scala equivalent?

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

For more options, visit https://groups.google.com/d/optout.




--
Tim Pigden
Optrak Distribution Software Limited
<a href="tel:+44%201992%20517100" value="+441992517100" target="_blank">+44 (0)1992 517100
http://www.linkedin.com/in/timpigden
http://optrak.com
Optrak Distribution Software Ltd is a limited company registered in England and Wales.
Company Registration No. 2327613 Registered Offices: Suite 6,The Maltings, Hoe Lane, Ware, SG12 9LR England 
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Optrak Distribution Software Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--
Oliver Ruebenacker
Senior Software Engineer, Diabetes Portal, Broad Institute




--
Tim Pigden
Optrak Distribution Software Limited
+44 (0)1992 517100
http://www.linkedin.com/in/timpigden
http://optrak.com
Optrak Distribution Software Ltd is a limited company registered in England and Wales.
Company Registration No. 2327613 Registered Offices: Suite 6,The Maltings, Hoe Lane, Ware, SG12 9LR England 
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Optrak Distribution Software Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Loading...