Cartesian product of lists in kept in an HList - keeping the types

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Cartesian product of lists in kept in an HList - keeping the types

Hugo Ferreira
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Hugo Ferreira
Hello,

I have just realised I have not explained the motivation. The
reason why I need types is to call a model:

    val m2 = new ModelB( pt1, pt2, pt3 )

and make sure that the correct parameters are passed onto the model.
In the case below, because pt1, pt2 and pt3 are typed as the base
class `Parameter`, I can inadvertently pass on the wrong values.

Hope this makes it clearer.

TIA



On Wednesday, 25 January 2017 17:02:12 UTC, Hugo Ferreira wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Oliver Ruebenacker

     Hello,

  Just wondering: how do you make sure the correct types are passed? Do you specify all the exact types in the definition of ModelB? Or is ModelB generic and there is some code that needs to know what types have been assigned to the type parameters of ModelB?

  I'm starting to wonder whether you might regret your extensive use of HLists later, because I have tried out things similar to HList and each time decided they are not worth it. At least for my use cases. They make writing code difficult and make compilation take more time and memory. Bugs usually lead to cryptic error messages. It is difficult to express things like "the first element can be anything, but the second element needs to be of type T".

  Depending on your use case, maybe there are other solutions?

     Best, Oliver

On Wed, Jan 25, 2017 at 12:23 PM, Hugo Ferreira <[hidden email]> wrote:
Hello,

I have just realised I have not explained the motivation. The
reason why I need types is to call a model:

    val m2 = new ModelB( pt1, pt2, pt3 )

and make sure that the correct parameters are passed onto the model.
In the case below, because pt1, pt2 and pt3 are typed as the base
class `Parameter`, I can inadvertently pass on the wrong values.

Hope this makes it clearer.

TIA




On Wednesday, 25 January 2017 17:02:12 UTC, Hugo Ferreira wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--
Oliver Ruebenacker
Senior Software Engineer, Diabetes Portal, Broad Institute

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Vlad Patryshev
In reply to this post by Hugo Ferreira
What's the reason of calling this construct a Cartesian product? Are you familiar with the definition. It's anything but.

Thanks,
-Vlad

On Wed, Jan 25, 2017 at 9:02 AM, Hugo Ferreira <[hidden email]> wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Hugo Ferreira
In reply to this post by Oliver Ruebenacker
Hello Oliver,


On Wednesday, 25 January 2017 17:42:58 UTC, Oliver Ruebenacker wrote:

     Hello,

  Just wondering: how do you make sure the correct types are passed? Do you specify all the exact types in the definition of ModelB? Or is ModelB generic and there is some code that needs to know what types have been assigned to the type parameters of ModelB?

Each model defines its own set of parameters. It merely extends a trait that defines a set of functions that need to be implemented. Ideally each of these parameters would be specific to a given model. The idea is to use dependent types if possible later on. Note that these parameters also extend a `Parameter` trait so that they can be used to generate set of those values (ex: grid search of s set machine learning hyper-parameters ).
 
Here is an example:

class ModelB[ T, U[ _ ] ]( val v1 : ParamOne[_],  val v2 : ParamTwo[_], val3 : ParamThree[_]) extends Model[ T, U, Double ] {
  def fit( data: U[ T ] ): Unit = {}
  def predict( datum: T ): Double = { 0.0 }
}

I can also use HLists for the parameters in order to retain type checking. The reason being I wanted to generalise the
boilerplate fort any number/type of parameters.


  I'm starting to wonder whether you might regret your extensive use of HLists later, because I have tried out things similar to HList and each time decided they are not worth it. At least for my use cases. They make writing code difficult and make compilation take more time and memory. Bugs usually lead to cryptic error messages. It is difficult to express things like "the first element can be anything, but the second element needs to be of type T".


I am already regretting it. 8-(
 
  Depending on your use case, maybe there are other solutions?


Th idea is to have the following type check:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

    val pt1 = t1.toStream(1)  // ParamOne[Double].Self
    val pt2 = t2.toStream(1)  // ParamTwo[Double].Self
    val pt3 = t3.toStream(1)  // ParamThree[Int].Self
    val m2 = new ModelB( pt1, pt2, pt3 )

An alternate would be something like:
 
    val grid = t1 cross_product t2 cross_product t3 // List[ParamOne[_], ParamTwo[_], ParamThree[]]

Then get the output of grid to deed the model. How could I generalise that for a any model?

Thanks for the feedback.

Hugo F.


     Best, Oliver

On Wed, Jan 25, 2017 at 12:23 PM, Hugo Ferreira <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Zt4sBjn9AwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">hugo6f...@...> wrote:
Hello,

I have just realised I have not explained the motivation. The
reason why I need types is to call a model:

    val m2 = new ModelB( pt1, pt2, pt3 )

and make sure that the correct parameters are passed onto the model.
In the case below, because pt1, pt2 and pt3 are typed as the base
class `Parameter`, I can inadvertently pass on the wrong values.

Hope this makes it clearer.

TIA




On Wednesday, 25 January 2017 17:02:12 UTC, Hugo Ferreira wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Zt4sBjn9AwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">scala-user+...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.



--
Oliver Ruebenacker
Senior Software Engineer, <a href="http://www.type2diabetesgenetics.org/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.type2diabetesgenetics.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFVE8xTGqlzANyuan2sVDo75uy-vA&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.type2diabetesgenetics.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFVE8xTGqlzANyuan2sVDo75uy-vA&#39;;return true;">Diabetes Portal, <a href="http://www.broadinstitute.org/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.broadinstitute.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHloBLjHUP5Yhb6LyVOZ-JE5b4Y5A&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.broadinstitute.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHloBLjHUP5Yhb6LyVOZ-JE5b4Y5A&#39;;return true;">Broad Institute

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Hugo Ferreira
In reply to this post by Vlad Patryshev
Hello Vlad,

You lost me. I am referring to [1]. Can you explain?

TIA,
Hugo F

1. https://en.wikipedia.org/wiki/Cartesian_product


On Wednesday, 25 January 2017 20:18:58 UTC, Vlad Patryshev wrote:
What's the reason of calling this construct a Cartesian product? Are you familiar with the definition. It's anything but.

Thanks,
-Vlad

On Wed, Jan 25, 2017 at 9:02 AM, Hugo Ferreira <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="JP5MGrwFBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">hugo6f...@...> wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="JP5MGrwFBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">scala-user+...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Jasper-M
In reply to this post by Hugo Ferreira
Hi,

Op donderdag 26 januari 2017 11:45:26 UTC+1 schreef Hugo Ferreira:

Th idea is to have the following type check:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

    val pt1 = t1.toStream(1)  // ParamOne[Double].Self
    val pt2 = t2.toStream(1)  // ParamTwo[Double].Self
    val pt3 = t3.toStream(1)  // ParamThree[Int].Self
    val m2 = new ModelB( pt1, pt2, pt3 )

I'm not completely following everything that has been said here, but for this you could do something like the following:

scala> :paste
// Entering paste mode (ctrl-D to finish)


trait
Parameter[T] { def value: T }
case class ParamOne[T](value: T) extends Parameter[T]
case class ParamTwo[T](value: T) extends Parameter[T]
case class ParamThree[T](value: T) extends Parameter[T]


class ParameterRange[P[X] <: Parameter[X], T] private(from: P[T], to: P[T], step: T) {
 
def toStream: Stream[P[T]] = Stream(from, to) // dummy implementation
}


object ParameterRange {
 
def apply[P1[X] <: Parameter[X], P2[X] <: Parameter[X], T](from: P1[T], to: P2[T], step: T)(implicit ev: P1[T] =:= P2[T]) =
   
new ParameterRange(ev(from), to, step)
}


// Exiting paste mode, now interpreting.


scala
> val t1 = ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4 )
t1
: ParameterRange[ParamOne,Double] = ParameterRange@3e5506fb


scala
> val pt1 = t1.toStream(1)
pt1
: ParamOne[Double] = ParamOne(1.0)


scala
> val error = ParameterRange( ParamOne( 0.0 ), ParamTwo( 1.0 ), 0.4 )
<console>:16: error: Cannot prove that ParamOne[Double] =:= ParamTwo[Double].
       val error
= ParameterRange( ParamOne( 0.0 ), ParamTwo( 1.0 ), 0.4 )
                                 
^

Kind regards,
Jasper
 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Vlad Patryshev
In reply to this post by Hugo Ferreira
Ok, sure. But a List is not a Set. Do you mean a list of all pairs of elements of the two lists? It may make sense, but I have to figure out why this tensor product can qualify as a Cartesian product.
The signature of your method, though, does not imply you take two objects and produce the third one.

Thanks,
-Vlad

On Thu, Jan 26, 2017 at 2:50 AM, Hugo Ferreira <[hidden email]> wrote:
Hello Vlad,

You lost me. I am referring to [1]. Can you explain?

TIA,
Hugo F

1. https://en.wikipedia.org/wiki/Cartesian_product


On Wednesday, 25 January 2017 20:18:58 UTC, Vlad Patryshev wrote:
What's the reason of calling this construct a Cartesian product? Are you familiar with the definition. It's anything but.

Thanks,
-Vlad

On Wed, Jan 25, 2017 at 9:02 AM, Hugo Ferreira <[hidden email]> wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Hugo Ferreira
In reply to this post by Jasper-M
Hello Jasper,


On Thursday, 26 January 2017 12:22:08 UTC, Jasper-M wrote:
Hi,

Op donderdag 26 januari 2017 11:45:26 UTC+1 schreef Hugo Ferreira:

Th idea is to have the following type check:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

    val pt1 = t1.toStream(1)  // ParamOne[Double].Self
    val pt2 = t2.toStream(1)  // ParamTwo[Double].Self
    val pt3 = t3.toStream(1)  // ParamThree[Int].Self
    val m2 = new ModelB( pt1, pt2, pt3 )

I'm not completely following everything that has been said here, but for this you could do something like the following:



Thanks for the feedback. The issue is not the generation of the parameter range. I have that done. In fact
the code you show is surprisingly very similar to what I have. The problem is how can I generate a
Cartesian product of those ranges without loosing the parameter type. The goal was to ensure that
I pass the correct parameters to a model.

I tried using HLists. More effort than what I bargained for. 8-(
  
Regards,
Hugo F.

 
scala> :paste
// Entering paste mode (ctrl-D to finish)


trait
Parameter[T] { def value: T }
case class ParamOne[T](value: T) extends Parameter[T]
case class ParamTwo[T](value: T) extends Parameter[T]
case class ParamThree[T](value: T) extends Parameter[T]


class ParameterRange[P[X] <: Parameter[X], T] private(from: P[T], to: P[T], step: T) {
 
def toStream: Stream[P[T]] = Stream(from, to) // dummy implementation
}


object ParameterRange {
 
def apply[P1[X] <: Parameter[X], P2[X] <: Parameter[X], T](from: P1[T], to: P2[T], step: T)(implicit ev: P1[T] =:= P2[T]) =
   
new ParameterRange(ev(from), to, step)
}


// Exiting paste mode, now interpreting.


scala
> val t1 = ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4 )
t1
: ParameterRange[ParamOne,Double] = ParameterRange@3e5506fb


scala
> val pt1 = t1.toStream(1)
pt1
: ParamOne[Double] = ParamOne(1.0)


scala
> val error = ParameterRange( ParamOne( 0.0 ), ParamTwo( 1.0 ), 0.4 )
<console>:16: error: Cannot prove that ParamOne[Double] =:= ParamTwo[Double].
       val error
= ParameterRange( ParamOne( 0.0 ), ParamTwo( 1.0 ), 0.4 )
                                 
^

Kind regards,
Jasper
 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Hugo Ferreira
In reply to this post by Vlad Patryshev
Hello Vlad,


On Thursday, 26 January 2017 20:42:45 UTC, Vlad Patryshev wrote:
Ok, sure. But a List is not a Set. Do you mean a list of all pairs of elements of the two lists? It may make sense, but I have to figure out why this tensor product can qualify as a Cartesian product.

These are not tensors. They are simply lists of values. I make no assumptions regrading duplicates nor the type of the elements.
 
The signature of your method, though, does not imply you take two objects and produce the third one.


GridPramaterSearch.cartesian5 take in a list of streams (or sets, lists or any other traversable) and
generates a stream (result can be very large) of all combinations of each element of the input.
So we have:
 
(1,2) x (3,4) x (5,6) =
((1,3), (1,4), (2,3), (2,4)) x (5,6) =
((1,3,5), (1,3,6), (1,4,5), (1,4,6), ..... )

Note that these are not vectors (not a cross product). They can contain strings, Booleans, etc....

Hope this is clear now.

Regards,
Hugo F




Thanks,
-Vlad

On Thu, Jan 26, 2017 at 2:50 AM, Hugo Ferreira <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="1ksGGJ1VBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">hugo6f...@...> wrote:
Hello Vlad,

You lost me. I am referring to [1]. Can you explain?

TIA,
Hugo F

1. <a href="https://en.wikipedia.org/wiki/Cartesian_product" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FCartesian_product\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHs41lnCAWje2ZAQv5xnk1NiiK98Q&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FCartesian_product\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHs41lnCAWje2ZAQv5xnk1NiiK98Q&#39;;return true;">https://en.wikipedia.org/wiki/Cartesian_product


On Wednesday, 25 January 2017 20:18:58 UTC, Vlad Patryshev wrote:
What's the reason of calling this construct a Cartesian product? Are you familiar with the definition. It's anything but.

Thanks,
-Vlad

On Wed, Jan 25, 2017 at 9:02 AM, Hugo Ferreira <[hidden email]> wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="1ksGGJ1VBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">scala-user+...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Vlad Patryshev
Ok, and this is not a cartesian product for two and a half reasons.

1. You mentioned the definition from Wiki, and it's only applicable to sets. Not even multisets. Definitely not lists.

2. If you want to use a categorical definition of a cartesian product, you have to come up with the definition of a category where lists are objects. I am sure there's more than one option for that. Then you will have to prove that your definition does produce a cartesian product. I'm not sure about it at all. A cartesian product is a limit of a certain diagram.

3. The way you do it, all combinations, is one of the ways to define tensor product in a monoidal category of some sort. This is a known construct. Don't mix tensors (the definition of which is also probably more sophisticated than most people think) and tensor product in general. 

So there. Linear algebra has nothing to do with any of this.


Thanks,
-Vlad

On Fri, Jan 27, 2017 at 2:40 AM, Hugo Ferreira <[hidden email]> wrote:
Hello Vlad,


On Thursday, 26 January 2017 20:42:45 UTC, Vlad Patryshev wrote:
Ok, sure. But a List is not a Set. Do you mean a list of all pairs of elements of the two lists? It may make sense, but I have to figure out why this tensor product can qualify as a Cartesian product.

These are not tensors. They are simply lists of values. I make no assumptions regrading duplicates nor the type of the elements.
 
The signature of your method, though, does not imply you take two objects and produce the third one.


GridPramaterSearch.cartesian5 take in a list of streams (or sets, lists or any other traversable) and
generates a stream (result can be very large) of all combinations of each element of the input.
So we have:
 
(1,2) x (3,4) x (5,6) =
((1,3), (1,4), (2,3), (2,4)) x (5,6) =
((1,3,5), (1,3,6), (1,4,5), (1,4,6), ..... )

Note that these are not vectors (not a cross product). They can contain strings, Booleans, etc....

Hope this is clear now.

Regards,
Hugo F




Thanks,
-Vlad

On Thu, Jan 26, 2017 at 2:50 AM, Hugo Ferreira <[hidden email]> wrote:
Hello Vlad,

You lost me. I am referring to [1]. Can you explain?

TIA,
Hugo F

1. https://en.wikipedia.org/wiki/Cartesian_product


On Wednesday, 25 January 2017 20:18:58 UTC, Vlad Patryshev wrote:
What's the reason of calling this construct a Cartesian product? Are you familiar with the definition. It's anything but.

Thanks,
-Vlad

On Wed, Jan 25, 2017 at 9:02 AM, Hugo Ferreira <[hidden email]> wrote:
Hello,

I have the following scenario - I want to set up a grid search for a number of lists/streams, each of which
have their own types. More specifically, each type is a parameter that must be fed to a model. To set-up
the parameters I do something like:

    val t1 = new ParameterRange( ParamOne( 0.0 ), ParamOne( 1.0 ), 0.4, linSearch )
    val t2 = new ParameterRange( ParamTwo( 0.0 ), ParamTwo( 1.0 ), 0.4, linSearch )
    val t3= new ParameterRange( ParamThree( 0 ), ParamThree( 10 ), 4, linISearch )

No, I have a function that generates the Cartesian product for the lists so:

    val g1 = List( t1.toStream, t2.toStream, t3.toStream )
    def g2 = GridPramaterSearch.cartesian5( g1.toStream, { x: Seq[ Parameter[_] ] => x  } )

The result g2 produces a sequence of `Parameter`s because that is the base class. In order
to keep the specific types, I see only two solutions:

1. Store t1, t2 and t3 in an HList, use each element one at a time to retrieve the stream
  of the appropriate type and combine that with the next one. Keep accumulating the
  result also as an HList and return that.

2. Just construct each function manually - also returning an HList.

For the record here is my Cartesian product function.

  def cartesian5[ A, B ]( list: => Stream[ Seq[ A ] ], f: Seq[ A ] => B ): Stream[ B ] = {
    def loop( lst: => Stream[ Seq[ A ] ], acc: Seq[ A ] ): Stream[ B ] = {
      lst match {
        case Stream.Empty => List( f( acc ) ).toStream
        case h #:: t      => h.toStream.flatMap { x => loop( t, x +: acc ) }
      }
    }
    loop( list, List() )
  }

Solution 2 is not really a solution. I have been over a week trying to use priority based
implicit solutions to generate something like automatically:

  def combineHList_0[A, B, C]( l1 : List[A], l2: List[B], l3: List[C])  = {
    l1.flatMap { x1 => l2.flatMap { x2 => l3.map { x3 => HCons(x1, HCons(x2, HCons(x3, HNil))) } } }
  }

using the compiler. I have not succeeded because I cannot figure out the typing.

Can anyone tell me if what I am attempting is doable and if so can you point me to a solution.
Barring that, are their any other suggestions?

TIA.



--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]m.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Hugo Ferreira
Hi Vlad,

I must confess this is way out of my league. I have no iintention of defining the Cartesian formally.

Thanks for the feedback though.

Regards,
Hugo F

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Cartesian product of lists in kept in an HList - keeping the types

Vlad Patryshev
Okay, never mind; I know it is a long way.
Just don't call it "cartesian product", it has a pretty legit name "tensor product".
Tensor product is not uniquely defined; another one is defined by zipping two lists. They have similar properties, but different behaviors.

Thanks,
-Vlad

On Fri, Jan 27, 2017 at 2:30 PM, Hugo Ferreira <[hidden email]> wrote:
Hi Vlad,

I must confess this is way out of my league. I have no iintention of defining the Cartesian formally.

Thanks for the feedback though.

Regards,
Hugo F

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.