Blog post: turning Scala code into Spark

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Blog post: turning Scala code into Spark

Anthony Cros
Hi all,

Hopefully it is ok for me to post this here, I just finished writing a blog post entitled "Turning Scala code into Spark" at http://anthonycros.com/turn-scala-into-spark/ (also on github at https://anthony-cros.github.io/blog/turn-scala-into-spark/). I would love to have some feedback from the community!

I will link comments on here to the dedicated issue I created for discussing it: https://github.com/anthony-cros/anthony-cros.github.io/issues/1 (low tech, but I haven't set everything up for that blog yet).

Thanks a lot in advance!

Anthony

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Blog post: turning Scala code into Spark

Koert Kuipers
interesting!

note that spark's flatMap is a bit of a challenge in the context of keeping lines the same verbatim, since its RDD[X].flatMap[Y](f: X => TraversableOnce[Y]) instead of RDD[X].flatMap[Y](f: X => RDD[Y])

On Mon, Sep 19, 2016 at 11:06 PM, Anthony Cros <[hidden email]> wrote:
Hi all,

Hopefully it is ok for me to post this here, I just finished writing a blog post entitled "Turning Scala code into Spark" at http://anthonycros.com/turn-scala-into-spark/ (also on github at https://anthony-cros.github.io/blog/turn-scala-into-spark/). I would love to have some feedback from the community!

I will link comments on here to the dedicated issue I created for discussing it: https://github.com/anthony-cros/anthony-cros.github.io/issues/1 (low tech, but I haven't set everything up for that blog yet).

Thanks a lot in advance!

Anthony

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Blog post: turning Scala code into Spark

Anthony Cros
In reply to this post by Anthony Cros
Thanks for the feedback, Koert!

I think in the future I'll try and use a more comprehensive example, and this time I'll include a flatMap. I think it's worth pointing out were Scala and Spark do differ as well. This would be valuable info for someone having to do such a translation (such as myself soon actually!). I'd even like to create a cheatsheet for it eventually...

On Monday, September 19, 2016 at 11:06:12 PM UTC-4, Anthony Cros wrote:
Hi all,

Hopefully it is ok for me to post this here, I just finished writing a blog post entitled "Turning Scala code into Spark" at <a href="http://anthonycros.com/turn-scala-into-spark/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fanthonycros.com%2Fturn-scala-into-spark%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEIlnWNf8iwfd5VHuA1VSb_x0kcdQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fanthonycros.com%2Fturn-scala-into-spark%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEIlnWNf8iwfd5VHuA1VSb_x0kcdQ&#39;;return true;">http://anthonycros.com/turn-scala-into-spark/ (also on github at <a href="https://anthony-cros.github.io/blog/turn-scala-into-spark/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fanthony-cros.github.io%2Fblog%2Fturn-scala-into-spark%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNETVGy958WfNrJTZTo78P2OdyUOaA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fanthony-cros.github.io%2Fblog%2Fturn-scala-into-spark%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNETVGy958WfNrJTZTo78P2OdyUOaA&#39;;return true;">https://anthony-cros.github.io/blog/turn-scala-into-spark/). I would love to have some feedback from the community!

I will link comments on here to the dedicated issue I created for discussing it: <a href="https://github.com/anthony-cros/anthony-cros.github.io/issues/1" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fanthony-cros%2Fanthony-cros.github.io%2Fissues%2F1\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFBtAp1XZror7KC7cF5GvQr3_r1Pw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fanthony-cros%2Fanthony-cros.github.io%2Fissues%2F1\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFBtAp1XZror7KC7cF5GvQr3_r1Pw&#39;;return true;">https://github.com/anthony-cros/anthony-cros.github.io/issues/1 (low tech, but I haven't set everything up for that blog yet).

Thanks a lot in advance!

Anthony

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.