Scala Days

Reactive Applications are the next major evolution of the Internet. They allow for applications to be responsive, scalable and resilient by building on a fully event-driven foundation. Lightbend’s Reactive Platform, consisting of the Play Framework, the Akka middleware and the Scala programming language embraces this new programming paradigm which allows developers to write interactive applications that are always available and which adapt to changing load by being distributed by design.While the reactive approach enable us to build highly scalable and resilient applications it also introduces new challenges in how to monitor them. Almost every current monitoring tool relies on a stack frame based approach where using the stack trace can provide good answers to what caused the exceptional state. In message driven, or asynchronous, applications this approach no longer provides any good information. We therefore need to invent new approaches for how to monitor these types of applications. During this session we will cover the traditional monitoring approach, different possible ways of how to monitor asynchronous applications and finally show the way we have chosen to build a monitoring tool for reactive applications at Lightbend.

Microservices architecture are becoming a de-facto industry standard, but are you satisfied with the current state of the art? We are not, as we believe that building microservices today is more challenging than it should be. Lagom is here to take on this challenge. First, Lagom is opinionated and it will take some of the hard decisions for you, guiding you to produce microservices that adheres to the Reactive tenents. Second, Lagom was built from the ground up around you, the developer, to push your productivity to the next level. If you are familiar with the Play Framework's development environment, imagine that but tuned for building microservices; we are sure you are going to love it! Third, Lagom comes with batteries included for deploying in production: going from development to production could not be easier.

In this session you will get an introduction to the Lightbend Lagom framework. There will be code and live demos to show you in practice how it works and what you can do with it, making you fully equipped to build your next microservices with Lightbend Lagom.

I will present the ongoing effort to port and cross-compile (most of) Akka, one of the biggest frameworks in the Scala land, against the alternative compiler of Scala.js.

We dig into various use cases from taming UI complexity to server side concurrent code made easy on Node.Js server side and more over!

Properly handling distributed and concurrent runtimes with the very same abstraction layer will turn your world upside-down breaking the frontiers between client and server development.

Machine Learning to some is still very magical. The truth however is that this magic is actually much easier to use than you'd expect. Come and learn how you can use Apache Spark and Akka together to build a service that recommends items to users. In this session, I'm going to show you some of the bits that go into building a recommender system, how to actually implement one in Spark and finally how to integrate the recommender system into your application using Akka HTTP.

A 'taking to production' experience based talk about Scala.js, specifically tailored for Scala folks with chronic Javascript pain. People with and without Scala.js knowledge are equally invited. What to expect? Quick yet concise overview of moving Scala.js parts following by presentation of non coast-to-coast approach to Scala.js. Still not convinced to come to the talk? How about:

• Spoiler #0: There will be no preaching to the choir. Huh? No slides about Javascript vs Scala. We all know the answer, aren't we?
• Spoiler #1: Overview will be based on perks of Scala.js cross project. Crossbuilding knowledge is a power, you will see.
• Spoiler #2: There is a strong presumption that a number of your frontend teammates willing to move to Scala.js is 0. You tell me if I'm wrong, I show you how ""my way or the highway"" is not your only Scala.js option.
• Spoiler #3: It all will start making sense. See you on scala-js gitter.

Normally, we use Java or Scala to build applications for large organizations running on servers. We wanted to find out if we could use the same languages and tools on IoT hardware. We also wanted to investigate whether or not (remote) actors could replace REST endpoints. The Lego trains are equipped with a Raspberry Pi, camera, wireless dongle, infrared transmitter, speaker, RFID reader, and battery pack. Next to that, we have automated switch tracks a Lego Ferris Wheel and camera's again with the help of Raspberry Pi's. We also built some lightning effects with LEDs controlled by Particle Photon’s. To control the trains and other parts, we built a remote actor based application with Scala, Akka, Akka HTTP and AngularJS. We will show you when and how to use Akka HTTP and remote actors. We will also show the results of the performance tests we did to compare the two options. Next to that we will talk about our experiences and challenges and of course we will give a live demo!

One of the great benefits of the Lightbend ecosystem is its ability to empower great scalability, and it is often considered for this very reason.Indeed, if employed correctly, Scala, Akka and the rest can make it possible to much more easily write systems that scale up (and out) to extremes. However, not every design is able to scale, and the approach and architecture can sharply limit this option, no matter how good the tools. What does ""perfect scalability"" look like? Is there even such a thing? Can systems be designed to scale virtually without limit? Where does the pattern break down, and what can you do about it?

In this session, we will examine the practical architectural constraints on systems intended to scale up in near-linear fashion. It turns out that having the ability to scale to extreme load is more about what not to do than it is about what to do. Why do some system scale up and others don't? How can you take a design that has problems and refactor to a design that has fewer limits? Our teams encounter these problems every day, and we keep track of what works and what doesn't in the real world. What have we seen in the field? What proven real-world design and architecture approaches can we use to help ensure our systems can scale, and what should we avoid? How can we leverage the power of the Lightbend platform to architect such systems? Which parts for which kinds of problems? How does Akka and the actor model fit these patterns? How does Spark, Kafka, Akka streaming?

Highly scalable systems often put together tools such as Docker, Ansible, Salt, Mesos, ConductR with techniques such as microservices, monitoring, and continious delivery. Automation of the deploy pipeline and deep performance monitoring are essential parts of highly scalable systems - we will look at examples of such systems, and which of the tools in the stack they employ, and why. In this talk, we'll examine both the practices and the pitfalls, and what combinations we have used to scale not only within single clusters, but across data centers for continent-spanning applications where required.We will use use-cases all the way from massive IoT scale wearable devices all the way to high finance, and find the commonalities in the solutions that don't limit their own expansion. If your organization needs to take your scalability to significantly higher levels, this talk will be where you need to start.

In this talk/live-coding session, we will have a practical look at how a reactive web application is different from a "normal" one and how resilience, elasticity and responsiveness translate into code. We will start by having a quick theoretical introduction to asynchronous computation and then build, run, deploy and load-test a small reactive web application built with the Play Framework, exploring a few key concepts such as Futures, Actors and Circuit Breakers along the way.

At the keynote at Scala Exchange, Jessica Kerr mentioned that there is very much documentation for starting and expert Scala developers but (almost) nothing in between. In this talk I want to demystify implicits. Implicits are a fairly advanced feature and a very important aspect of the Scala language. They help with writing concise code and enable lots of DSLs. On the other side, they can be very magical for an untrained eye. In this talk, we will delve into idiomatic use cases of implicits and how the Scala compiler resolves them. If time allows, we will jump into type classes as well.

Typeclasses are a hidden gem of the Scala language. They provide an immense power not seen in imperative languages, and so their approach might be unusual or alien to those approaching Scala from an imperative background. I will show how typeclasses allow developers to effectively attach their own interfaces to code written by others. In this talk, I describe what a genetic algorithm is and provide an example implementation in Scala. Using this implementation, I will demonstrate how to define a specific typeclass for our problem. I will then derive several different implementations, showing how to get rock solid confidence in testing our algorithm - with the help of ScalaCheck - and then provide a completely different typeclass to provide a fun, visual and creative solution, illustrating the iterations and improvements as the genetic algorithm’s fitness function runs. The talk will be particularly hands-on, with plenty of examples run directly from the REPL.

The actor model is one of the de-facto standards when it comes to building reliable distributed systems. There are many reasons why actors are so attractive. On one hand, actors ensure that message-processing is serialized within each actor, preserving the familiar sequential programming model. On the other hand, programs written in the actor model are location-transparent, which is one of the prerequisites for scaling systems. Importantly, the actor model is sufficiently low-level to express arbitrary message protocols. However, composing these message protocols is the key to high-level abstractions, and it is difficult to reuse or compose message protocols with actors. Lack of simple composition is an obstacle to building complex systems.

The reactor model is the new answer to the challenges of distributed computing. This model simplifies protocol composition with first-class typed channels and event streams. In this talk, I will present the Reactors.IO framework which is based on the reactor programming model. I will compare the reactor and the actor models on concrete Scala programs. I will show specific obstacles for composition in the classic actor model, and how to overcome them. I will then show how to build reusable, composable distributed computing components in the new model.

Futures and promises in Scala are an integral part of asynchronous and concurrent code. Popular websites rely on futures for responsiveness in the presence of a large number of concurrent visitors. Furthermore, futures work well together with functional programming abstractions, thanks to the nature of future-based computations--single-assignment dataflow. However, futures also have important restrictions. For example, a future is completed with at most one result. Therefore, it is impossible to complete a future with a preliminary result and subsequently refine the result, for example, when more precise information becomes available. Finally, futures do not support resolving cyclic dependencies, instead resulting in deadlocks when such dependencies occur.

In this talk, I report on an extension of Scala's futures with lattices and quiescence, inspired by related features in Haskell's LVish package and the LVars programming model. In this extended model the state of a future is an element of a lattice. Importantly, multiple updates of its corresponding "promise" are possible where updates correspond to join operations of the lattice. In addition, resolution of cyclic dependencies is supported through a mechanism based on detecting quiescence of the underlying execution context. The programming model is currently being exploited in the context of OPAL, a new Scala-based, concurrent static analysis framework developed at Technische Universität Darmstadt. In the last part of my talk I will report on experimental results (performance, code complexity) using lattice-based futures for several large-scale static analysis tasks (e.g., purity analysis, bug finding) with OPAL.

Developers who are new to Scala often shy away from coming into contact with implicits, and by extension, understanding typeclasses. In big organizations that have been adopting Scala at scale, you sometimes even come across hard rules that put a ban on the use of implicits because that language feature is considered to be too advanced and not understood by a lot of developers. On the other hand, implicits and typeclasses are used heavily not only by a lot of the most important Scala frameworks and libraries, but also in the standard library. Given the fact that it is so hard to evade them when writing real world Scala code, I would like to encourage developers adopting Scala to overcome their fear of implicits and instead embrace the typeclass pattern. In this talk, as an intermediate Scala developer, you will learn everything you really need to know about typeclasses: What they are good for and how they compare to what you are familiar with from object-oriented languages, when you should and should not use them, how the pattern can be encoded in Scala and how to write your own typeclasses, how to provide instances of typeclasses for your own or existing types, and how to do all of this with minimal boilerplate. Throughout the talk, you will see numerous examples of typeclasses used in the Scala ecosystem and the standard library, and you'll see that you don't need to know anything about category theory to benefit from embracing typeclasses.

It was true that, until pretty recently, the language of choice to manipulate and to make sense out of the data for Data Scientists was mainly one of Python, R or Matlab. This lead to split in the communities and duplication of efforts in languages offering a similar set functionnaiity. Although, it was foreseen that Julia (for instance) could gather parts of these communities, an unexpected event happened: the amount of available data and the distributed technologies to handle them. Distributed technologies raised out of the blue by data engineer and most of them are using a convenient and easy to deploy platform, the JVM. In this talk, we’ll show how the Data Scientists are now part of an heterogeneous team that has to face many problems and have to work towards a global solution together. This is including a new responsibility to be productive and agile in order to have their work integrated into the platform. This is why technologies like Apache Spark is so important nowadays and is gaining this traction from different communities. And even though some binding are available to legacy languages there, all the creativity in new ways to analyse the data has to be done in scala. So that, the second part of this talk will introduce and summarize all the new methodologies and scientific advances in machine learning done Scala as the main language, rather than others. We’ll demonstrate that all using the right tooling for Data Scientists which is enabling interactivity, live reactivity, charting capabilities and robustness in Scala, something there were still missing from the legacy languages. Hence, the examples will be provided and shown in a fully productive and reproducible environment combining the Spark Notebook and Docker.

Processing streaming data is becoming increasingly important in many areas. Scala and the Lightbend Reactive platform offer multiple solutions for processing streaming data, including Akka, Akka Streams and Apache Spark. This talk introduces the advantages and concepts of streaming data processing. It will mention differences between static data and data in motion and their usage as streaming data sources. The main goal of the presentation is detailed discussion of Akka Persistence Query and implementation of the stream production specification in Cassandra plugin for Akka Persistence (akka-persistence-cassandra) that the author participated in. Focus is on architecture and design considerations, implementation details, performance tuning and distributed system specifics such as correctness, efficiency, consistency, order, causality or failure scenario handling that are inherently part of the solution and apply to wide variety of distributed systems. Finally, other improvements to the Cassandra plugin for Akka Persistence project such as reusing the stream generation for non blocking asynchronous Akka Persistence recovery as well as application of the project and the discussed concepts to build modern reactive enterprise stream processing and asynchronous messaging distributed applications are presented.

Why do developers love IntelliJ IDEA? Because it takes care of all the routine and provides intelligent coding assistance. Once you master it, the productivity gains are quite surprising. In this session I'll show you the 30 hidden gems of IntelliJ IDEA that will help you become a more productive Scala developer. 

There has been a huge amount of activity around the Typelevel family of projects in the last eighteen months. It hasn't always been plain sailing, but the arrival of Cats on the scene last year marked the beginning an exciting period of collaboration among the Typelevel projects and reaching out to the wider Scala community that hadn't been possible before. Now, in mid-2016, we have had two Typelevel conferences and things are going from strength to strength. This talk will give a flavour of what has been going on: the collaborations between Algebra, Spire and Cats; between Cats and shapeless; between shapeless and scodec, doobie, ScalaCheck and Circe; and how all of this is feeding into the rebooted Typelevel Scala fork. It's also an open invitation to people right across the Scala spectrum to get involved in these projects and see what they can do for them in their own work.

At Atlassian, we are in the midst of an architectural shift towards microservices to provide a scalable and flexible platform for our cloud services. In order to achieve this safely with no risk of losing customer data, we are investigating ‘event sourcing’ for representing our domain models i.e. capturing streams of immutable events to represent data instead of the traditional update-in-place paradigm. With event sourcing, old versions of data can be easily restored and audit trails are available by default to help with debugging. In combination with the command-query responsibility separation (CQRS) pattern, we can seamlessly bring online new functionality requiring schema changes or new query patterns, and re-architect later for more scale simply by replaying and reinterpreting events into new microservices and ephemeral data stores.

In this talk, we will describe in detail what event sourcing and CQRS are, why we are using this approach, and a walk through of our implementation in Scala (leveraging scalaz-streams) in an AWS environment using DynamoDB, Kinesis and Lambdas.

Goals:

By the end of the presentation, the audience should:

• have an understanding of the benefits of event sourcing
• have an understanding of how event sourcing works
• be able to identify how event sourcing can be applied to their environment

Draft content outline:

• Why event sourcing and CQRS, and why for everyone
• safety - being able to revert changes, audit trails for free
• flexibility - support any future queries, simple handling of schema changes
• simple DB requirement - simply a key-value store with some kind of CAS (atomic counter, transaction)
• Easy to implement on distributed databases e.g. Dynamo

Walkthrough of Event sourcing principles by example (user account management) using our open source eventsrc library 

Walkthrough of architecture of event sourcing/CQRS system being developed in Atlassian running on AWS.

The purpose of this talk is to introduce one to differences between Scala and Dotty, as well as to support for Dotty in Scala plugin for IntelliJ IDEA. First of all we will look at the differences between the type systems. Besides that one will learn which aspects of language support in IntelliJ IDEA have already been implemented or will be implemented in the near future. Among other things the talk will cover the build process of the project, the incremental compilation.

A collection of horror stories that derive from a lack of best practices. Everything will be covered: How to make companies lose money, shutdown systems and disable payments (or worse) just from overlooking simple procedures that "take too much time" or "cost too much" to implement. We may even dim the lights at certain points in the presentation.

In this talk we'll have a deeper look into Akka Streams (the implementation) and Reactive Streams (the standard). The term streams has been recently pretty overloaded, so we'll disambiguate what streams are and what they aren't. We'll dive into a number of real life scenarios where applying back-pressure helps to keep your systems fast and healthy at the same time. We'll mostly focus on the Akka Streams implementation, but the general principles apply to any kind of asynchronous programming.

• A pragmatic balance of object-oriented and functional programming.
• An interpreter mode, which allows the same sort of exploratory programming that Data Scientists have enjoyed with Python and other languages. Scala-centric "Notebooks" are also now available.
• A rich Collections library that enables composition of operations for concise, powerful code.
• Tuples are naturally expressed in Scala and very convenient for working with data.
• Pattern Matching makes data "deconstruction" fast and intuitive.
• Type inference provides safety, feedback to the developer, yet minimal, manual typing of actual type signatures.
• Scala idioms lend themselves to the construction of small domain specific languages, which are useful for building libraries that are concise and intuitive for domain experts.

• Custom data layouts that use memory very efficiently with cache-awareness.
• Manual memory management, both on-heap and off-heap, to minimize garbage and GC pressure.
• Code generation to create optimal implementations of certain, heavily-used expressions from user code.

Scala has historically been a JVM-centric programming language. The situation has started to change with appearance of the Scala.js that opened a door of front-end development to Scala programmers. This talk will expand the horizons of Scala even more. We’re going to announce a new ahead-of-time compiler and lightweight managed runtime designed specifically for Scala.

What does it mean to think like a functional programmer? Why is it so hard to adopt functional programming if you are steeped in OO? In this talk I'll address these questions, using ideas from the philosophy of science. The talk will give new insight into the process of becoming productive in Scala, and the mindset needed to aid this process. Using the framework of ""The Structure of Scientific Revolutions"" we can see programming as undergoing three revolutions: structured programming, object-oriented programming, and now, functional programming. Each revolution brings with a new paradigm---basic assumptions and values that make moving across paradigms difficult to achieve. In this talk I'll describe the paradigm of the functional programmer: what is valued and, what is considered good code. I will contrast this to the paradigm of the object-oriented programmer, and finally consider if it is possible to reconcile these two approaches to programming.

Quill is a compile-time language integrated query library with a powerful query compilation mechanism based on the paper "A Practical Theory of Language-Integrated Query".

This talk will give an overview of the "Scylla and Charybdis" theorem that supports the query compilation, present the Quoted Domain Specific Language (QDSL) approach used by the API, and give a quick view of Quill's functionality.

Deep Learning is taking data science by storm. Unfortunately, most existing solutions aren’t particularly scalable. In this talk, we will show how we can easily implement a Spark­ ready version one of the most complex deep learning models, the Long Short­ Term Memory (LSTM) neural network, widely used in the hardest natural language processing and understanding problems, such as automatic summarization, machine translation, question answering and discourse. We will also show an LSTM demo with interactive, real­time visualizations using the Spark Notebook and Spark Streaming.

Have you ever dreamed of understanding how Scala.js works under the covers? Knowing your way around its codebase? This is your chance! This advanced talk will take you to the deepest parts of Scala.js: its compiler, its Intermediate Representation, its linker and its optimizer. Through live coding within the Scala.js codebase, we will explain how the Scala.js compilation pipeline works, and how you can modify it. After this talk, you will understand the Scala.js IR and be able to manipulate it. And you'll be ready to write your custom Linker Plugins! Attendees are expected to be proficient in Scala and have some compiler background (e.g., knowing what an AST is is vital).

In this talk we are going to discuss various state of the art open-source distributed streaming frameworks, their similarities and differences, implementation trade-offs, their intended use-cases and how to choose between them. I’m going to focus on the popular frameworks including Spark Streaming, Storm, Samza and Flink.  In addition, I’m going to cover theoretical introduction, common pitfalls, popular architectures and many more. The demand for stream processing is increasing. Immense amounts of data has to be processed fast from a rapidly growing set of disparate data sources. This pushes the limits of traditional data processing infrastructures. These stream-based applications include trading, social networks, Internet of things or system monitoring, are becoming more and more important. A number of powerful, easy-to-use open source platforms have emerged to address this. My goal is to provide comprehensive overview about modern streaming solutions and to help fellow developers with picking the best possible decision for their particular use-case. This talk should be interesting for anyone who is thinking about, implementing or have already deployed streaming solution.

Since its inception more than 2 years ago the effort for establishing a standard protocol for asynchronous stream processing under the name "Reactive Streams" (RS) has received a lot of attention. Many users and organizations are excited by this powerful new abstraction for defining and scaling pipelined processing logic in a fully asynchronous, non-blocking and generally reactive fashion.

Typically applications built on RS thereby rely on an RS infrastructure implementation to provide the general building blocks for defining arbitrarily complex business logic as a series of stream transformations. An RS infrastructure implementation also forms the "glue" that's required for interfacing with RS-compatible domain adapters (e.g. for databases, HTTP APIs and messaging systems).

While the number of RS-compatible domain adapters has been rising steadily over the course of the last year the range of available RS infrastructure implementations is still limited.

For Scala users there is currently only a single one, akka-stream, which is solidly engineered towards a particular set of design goals.

In order to further explore the design space of the still young Reactive Streams domain another fully-featured Reactive Streams infrastructure toolkit for Scala, called "Swave", has been built from scratch with a clear focus on maximum performance, a simple and powerful API as well as minimal dependencies.

This talk will introduce you to the project, its general design approach, feature set and core implementation choices as well as basic benchmark and performance figures.

We'll contrast with other RS implementations and highlight pros and cons from a user's perspective.

So the next time you are faced with the question of which Reactive Streams implementation to use you'll have one more choice to pick from.

And even if you can't decide you're still good! After all, the "Reactive Streams" label means: Interoperability for the win!

Getting to grips with a new programming language can be daunting, especially if it requires learning a new discipline such as functional programming. In this talk, Sofia and Dave will provide a step-by-step guide to adopting and mastering Scala from two different perspectives: the graduate developer and the visiting consultant.

Sofia Cole is a developer at YOOX NET-A-PORTER GROUP with eigtheen months' experience working with Scala. Dave Gurnell is a consultant at Underscore with over a decade's experience writing functional code. Between them, they will walk you through their experiences at YOOX NET-A-PORTER GROUP migrating from a legacy Perl monolith to a Scala microservice architecture.

Topics covered will include:

• How to sell Scala within your organization. 
• What benefits do managers and developers care about?
• A step-by-step guide for navigating the sea of Scala and FP concepts, without losing yourself in a monadic storm
• Shared joys and pains from different developers from disparate backgrounds, and how to reconcile them
• The tactical and targeted use of hack days, and functional programming workshops, including a step-by-step guide for running coding dojos
• A guide to pair programming from two different perspectives: peer-pairing and teacher-student pairing

This will be an entertaining and enlightening alternative take on the Scala adoption story. The speakers absolutely promise not to quarrel on-stage (unless the situation particularly demands it).

In ScalaTest 3.0's new async testing styles, tests have a result type of Future[Assertion]. Instead of blocking until a future completes, then performing assertions on the result, you map assertions onto the future and return the resulting Future[Assertion] to ScalaTest. The test will complete asynchronously when the Future[Assertion] completes. This non-blocking way of testing requires a very different mindset and different API. In this talk Bill Venners will show you how async testing was integrated into ScalaTest, and explain what motivated the design decisions. He'll show you how to use the new features, and suggest best practices for async testing on both the JVM and Scala.js.

Doing I/O in Scala (and Java) involves either invoking some magic "FileUtil" or browsing through StackOverflow. In this talk, we will introduce better-files (https://github.com/pathikrit/better-files) - a thin wrapper around Java NIO to enable simple, safe and sane I/O in Scala. We will also discuss problems with designing an I/O library that would make everyone happy and different schools of thoughts e.g. monadic vs non-blocking vs effect-based APIs

Building a complete system out of individual Microservices is not as simple as we're being told. While Microservices-based Architecture continues to draw more and more attention we're also starting to learn about the trade-offs and drawbacks. Individual Microservices are fairly easy to understand and implement, but they only make sense as systems, and it is in-between the services that the most challenging (and interesting) problems arise—here we are entering the world of distributed systems.

As we all know, distributed systems are inherently complex and we enterprise developers have been spoiled by centralized servers for too long to easily understand what this really means. Just slicing an existing system into various REST services and wiring them back together again with synchronous protocols and traditional enterprise tools—designed for a monolithic architectures—will set you up for failure.

This talk is going to distill the essence Microservices-based systems and then introduce you to a new development approach to Microservices which gets you started quickly with a guided, minimalistic approach on your local machine and supports you every single step on the way to a productive scaled out Microservices-based system composed of hundreds of services. At the end of this talk, you've experienced first hand how creating systems of microservices on the JVM is dead-simple, intuitive, frictionless and last and more importantly—a lot of fun!