Hello,
-- I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up the parameters I do something like: val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch ) val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch ) val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch ) No, I have a function that generates the Cartesian product for the lists so: val g1 = List( t1.toStream, t2.toStream, t3.toStream ) def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x } ) The result g2 produces a sequence of `Parameter`s because that is the base class. In order to keep the specific types, I see only two solutions: 1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream of the appropriate type and combine that with the next one. Keep accumulating the result also as an HList and return that. 2. Just construct each function manually - also returning an HList. For the record here is my Cartesian product function. def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = { def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = { lst match { case Stream.Empty => List( f( acc ) ).toStream case h #:: t => h.toStream.flatMap { x => loop( t, x +: acc ) } } } loop( list, List() ) } Solution 2 is not really a solution. I have been over a week trying to use priority based implicit solutions to generate something like automatically: def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C]) = { l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } } } using the compiler. I have not succeeded because I cannot figure out the typing. Can anyone tell me if what I am attempting is doable and if so can you point me to a solution. Barring that, are their any other suggestions? TIA. You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Hello,
-- I have just realised I have not explained the motivation. The reason why I need types is to call a model: val m2 = new ModelB( pt1, pt2, pt3 ) and make sure that the correct parameters are passed onto the model. In the case below, because pt1, pt2 and pt3 are typed as the base class `Parameter`, I can inadvertently pass on the wrong values. Hope this makes it clearer. TIA On Wednesday, 25 January 2017 17:02:12 UTC, Hugo Ferreira wrote:
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Depending on your use case, maybe there are other solutions? On Wed, Jan 25, 2017 at 12:23 PM, Hugo Ferreira <[hidden email]> wrote:
-- You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
In reply to this post by Hugo Ferreira
What's the reason of calling this construct a Cartesian product? Are you familiar with the definition. It's anything but. Thanks, -Vlad On Wed, Jan 25, 2017 at 9:02 AM, Hugo Ferreira <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
In reply to this post by Oliver Ruebenacker
Hello Oliver,
-- On Wednesday, 25 January 2017 17:42:58 UTC, Oliver Ruebenacker wrote:
Each model defines its own set of parameters. It merely extends a trait that defines a set of functions that need to be implemented. Ideally each of these parameters would be specific to a given model. The idea is to use dependent types if possible later on. Note that these parameters also extend a `Parameter` trait so that they can be used to generate set of those values (ex: grid search of s set machine learning hyper-parameters ). Here is an example: class ModelB[ T, U[ _ ] ]( val v1 : ParamOne[_], val v2 : ParamTwo[_], val3 : ParamThree[_]) extends Model[ T, U, Double ] { def fit( data: U[ T ] ): Unit = {} def predict( datum: T ): Double = { 0.0 } } I can also use HLists for the parameters in order to retain type checking. The reason being I wanted to generalise the boilerplate fort any number/type of parameters.
I am already regretting it. 8-(
Th idea is to have the following type check: val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch ) val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch ) val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch ) val pt1 = t1.toStream(1) // ParamOne[Double].Self val pt2 = t2.toStream(1) // ParamTwo[Double].Self val pt3 = t3.toStream(1) // ParamThree[Int].Self val m2 = new ModelB( pt1, pt2, pt3 ) An alternate would be something like: val grid = t1 cross_product t2 cross_product t3 // List[ParamOne[_], ParamTwo[_], ParamThree[]] Then get the output of grid to deed the model. How could I generalise that for a any model? Thanks for the feedback. Hugo F.
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
In reply to this post by Vlad Patryshev
Hello Vlad,
-- You lost me. I am referring to [1]. Can you explain? TIA, Hugo F 1. https://en.wikipedia.org/wiki/Cartesian_product On Wednesday, 25 January 2017 20:18:58 UTC, Vlad Patryshev wrote:
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
In reply to this post by Hugo Ferreira
Hi,
-- Op donderdag 26 januari 2017 11:45:26 UTC+1 schreef Hugo Ferreira:
I'm not completely following everything that has been said here, but for this you could do something like the following:
Kind regards, Jasper You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
In reply to this post by Hugo Ferreira
Ok, sure. But a List is not a Set. Do you mean a list of all pairs of elements of the two lists? It may make sense, but I have to figure out why this tensor product can qualify as a Cartesian product. The signature of your method, though, does not imply you take two objects and produce the third one. Thanks, -Vlad On Thu, Jan 26, 2017 at 2:50 AM, Hugo Ferreira <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
In reply to this post by Jasper-M
Hello Jasper,
-- On Thursday, 26 January 2017 12:22:08 UTC, Jasper-M wrote:
Thanks for the feedback. The issue is not the generation of the parameter range. I have that done. In fact the code you show is surprisingly very similar to what I have. The problem is how can I generate a Cartesian product of those ranges without loosing the parameter type. The goal was to ensure that I pass the correct parameters to a model. I tried using HLists. More effort than what I bargained for. 8-( Regards, Hugo F.
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
In reply to this post by Vlad Patryshev
Hello Vlad,
-- On Thursday, 26 January 2017 20:42:45 UTC, Vlad Patryshev wrote:
These are not tensors. They are simply lists of values. I make no assumptions regrading duplicates nor the type of the elements.
GridPramaterSearch.cartesian5 take in a list of streams (or sets, lists or any other traversable) and generates a stream (result can be very large) of all combinations of each element of the input. So we have: (1,2) x (3,4) x (5,6) = ((1,3), (1,4), (2,3), (2,4)) x (5,6) = ((1,3,5), (1,3,6), (1,4,5), (1,4,6), ..... ) Note that these are not vectors (not a cross product). They can contain strings, Booleans, etc.... Hope this is clear now. Regards, Hugo F
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Ok, and this is not a cartesian product for two and a half reasons. 1. You mentioned the definition from Wiki, and it's only applicable to sets. Not even multisets. Definitely not lists. 2. If you want to use a categorical definition of a cartesian product, you have to come up with the definition of a category where lists are objects. I am sure there's more than one option for that. Then you will have to prove that your definition does produce a cartesian product. I'm not sure about it at all. A cartesian product is a limit of a certain diagram. 3. The way you do it, all combinations, is one of the ways to define tensor product in a monoidal category of some sort. This is a known construct. Don't mix tensors (the definition of which is also probably more sophisticated than most people think) and tensor product in general. So there. Linear algebra has nothing to do with any of this. Thanks, -Vlad On Fri, Jan 27, 2017 at 2:40 AM, Hugo Ferreira <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Hi Vlad,
I must confess this is way out of my league. I have no iintention of defining the Cartesian formally. Thanks for the feedback though. Regards, Hugo F -- You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Okay, never mind; I know it is a long way. Just don't call it "cartesian product", it has a pretty legit name "tensor product". Tensor product is not uniquely defined; another one is defined by zipping two lists. They have similar properties, but different behaviors. Thanks, -Vlad On Fri, Jan 27, 2017 at 2:30 PM, Hugo Ferreira <[hidden email]> wrote: Hi Vlad, You received this message because you are subscribed to the Google Groups "scala-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Free forum by Nabble | Edit this page |