Enforcing rules at compile-time: an example

I recently solved a problem I had in Scala. I was able to solve the problem quickly and easily, where as I remember when I was less experienced with Scala, solving a problem like this was difficult. It would take a lot of thought and effort, and even if I came away with a solution that technically worked, it often felt off and wasn't pleasant to use.

I thought it'd be nice to share this example of my current approach to writing good Scala. I hope you find it useful.

Premise

I'm the author and maintainer of scalajs-react which is a Scala.JS library that provides a type-safe interface to React, a JavaScript UI library.

In scalajs-react, the primary way to create a UI component is via "the builder pattern". The builder API is separated into 4 steps so that you first specify the prerequisites in a deliberate order, then at the final step you can specify a bunch of optional lifecycle methods. Usage looks like this:

val MyComponent =
  ScalaComponent.builder[Props]("MyComponent")
    .stateless               // step 1
    .noBackend               // step 2
    .render(...)             // step 3
    .componentWillMount(...) // step 4 (optional)
    .componentDidMount(...)  // step 4 (optional)
    .build                   // step 4

Steps 1 and 2 are optional and are made so in the API via implicits. A minimal example looks like this:

val MyComponent =
  ScalaComponent.builder[Props]("MyComponent")
    .render(...)             // step 3
    .build                   // step 4

Multiple specifications of the same lifecycle method compose. For example, this is valid and will result in all three procedures executing at the .componentDidMount lifecycle event.

val MyComponent =
  ScalaComponent.builder[Props]("MyComponent")
    .render(...)             // step 3
    .componentDidMount(...)  // step 4 (optional)
    .componentDidMount(...)  // step 4 (optional)
    .componentDidMount(...)  // step 4 (optional)
    .build                   // step 4

Problem

A recent version of React introduced some changes to lifecycle methods and so I was updating scalajs-react the other day.

The two changes relevant to this article are:

1. A new lifecycle method getSnapshotBeforeUpdate is added, from which you return any arbitrary value called a snapshot.
2. The lifecycle method componentDidUpdate gets a new parameter which is the value from getSnapshotBeforeUpdate above.

Goals

Let's break down the new React changes into a few rules:

1. The return type of getSnapshotBeforeUpdate needs to match the type of the new componentDidUpdate param.
2. In the case that getSnapshotBeforeUpdate isn't specified, the type of the new componentDidUpdate param will be Unit (which is undefined in JS).
3. In order to support composition of multiple getSnapshotBeforeUpdate functions, a means of return value composition (most naturally a Semigroup typeclass) is required.

Regarding (3),

• Semigroup would only be required for subsequent calls, not the first, which adds a little complication for values for which Semigroup isn't defined.
• scalajs-react doesn't have external dependencies and I don't want to add a Semigroup typeclass to the public API.
• I'd be surprised if anyone ever wanted to supply multiple getSnapshotBeforeUpdate functions anyway; if so, one can do it oneself.

Therefore I've decided to just not support multiple getSnapshotBeforeUpdate functions. We don't lose parity with React JS anyway.

Let's break down (1) and (2) into more concrete rules:

1. getSnapshotBeforeUpdate can only be called 0 or 1 times
2. getSnapshotBeforeUpdate sets the Snapshot type
3. componentDidUpdate receives the Snapshot type
4. When componentDidUpdate is called and the Snapshot type is undefined, it's set to Unit
5. getSnapshotBeforeUpdate cannot occur after componentDidUpdate because it will change the Snapshot type which would invalidate the previous componentDidUpdate where the Snapshot was Unit (and a fn to Unit would be pointless here).

Before we continue it's time to emphasise: type-safety is very important to me. One of the biggest features of scalajs-react is its strong type-safety. (As much is reasonable in Scala) if it compiles, I want confidence that it works and is correct.

I want to encode the above rules into the types so that end-users don't have to read any documentation, have any internal knowledge of these rules, or experience any runtime exceptions; the compiler should just enforce everything we discussed such that violations wont even compile.

Rejected solution

Probably the first solution that earlier-me would've reached for, is to create a new step in the builder API like this:

  [STEP 3]                                          [STEP 5]

  .render   ----(implicit with Snapshot=Unit)--->   last step
     \                                              /
      \                                            /
       --------> .getSnapshotBeforeUpdate ------> /

                         [STEP 4]

There are problems with such a solution:

• It doesn't scale. If React adds more constraints in future it will become harder to keep a fluent API without introducing unnecessary usage constraints.
• External component config fns (LastStep => LastStep) need to be able to configure any part of the lifecycle.
• ScalaComponent.builder.static is an example where it returns a half-built component allowing further configuration. It needs to set the shouldComponentUpdate method which would skip step 4 in this approach, or else require that we add nearly everything to both steps (yuk).

Basic solution

Consider this pseudo-code:

var snapshotType = None

def getSnapshotBeforeUpdate[A](f: X => A) = {
  snapshotType match {
    case None    => snapshotType = Some(A)
    case Some(_) => error("SnapshotType already defined!")
  }
  getSnapshotBeforeUpdate = f
}

def componentDidUpdate(f) = {
  snapshotType = Some(snapshotType.getOrElse(Unit))
  componentDidUpdate.append(f)
}

We could track this at the term-level at runtime using Option (and typetags). It's not very type-safe though. We can still keep the same approach and logic, we just need to lift it up into the type-level so that it runs at compile-time instead of at runtime. To do so we'll use a type-level encoding of Option.

This is how you encode Option at the type-level in Scala; first you'll see a term-level equivalent for contrast:

object TermLevel {

  sealed trait Option[+A] {
    def getOrElse[B >: A](default: => B): B
  }
  final case class Some[+A](value: A) extends Option[A] {
    override def getOrElse[B >: A](default: => B) = value
  }
  case object None extends Option[Nothing] {
    override def getOrElse[B >: Nothing](default: => B) = default
  }

  // Example usage
  def value: Option[Any] => Any = _.getOrElse(())
}

// ===============================================================

object TypeLevel {

  sealed trait TOption {
    type GetOrElse[B]
  }
  sealed trait TSome[A] extends TOption {
    override final type GetOrElse[B] = A
  }
  sealed trait TNone extends TOption {
    override final type GetOrElse[B] = B
  }

  // Example usage
  type Value[T <: TOption] = T#GetOrElse[Unit]
}

Ok, now let's code up a skeleton that will enforce our rules at compile-time:

final class Builder[SnapshotType <: TOption] {
  type SnapshotValue = SnapshotType#GetOrElse[Unit]

  def getSnapshotBeforeUpdate[A](f: ... => A)
                                (implicit ev: SnapshotType =:= TNone)
                                : Builder[TSome[A]]

  def componentDidUpdate(f: SnapshotValue => ...)
                        : Builder[TSome[SnapshotValue]]
}

Let's compare this to our pseudo-code:

1. The snapshotType var is now a type parameter of Builder.
2. getSnapshotBeforeUpdate would check snapshotType is None and set it to Some(A). Now we ask for implicit proof that SnapshotType =:= TNone, set in the return type we can see return a new Builder with SnapshotType set to TSome
3. getSnapshotBeforeUpdate throw an error when snapshotType is Some(_). Now the compiler will throw an implicit not found error at compile-time when SnapshotType =:= TSome[_].
4. Where as before in componentDidUpdate we had snapshotType = Some(snapshotType.getOrElse(Unit)), we now have the equivalent in that the return type is Builder[TSome[SnapshotValue]] where SnapshotValue = SnapshotType#GetOrElse[Unit].

Nice solution

A while back I would've been satisfied with the above solution; it works right? If you know all of the rules, sure, but the error message to users is going to be pretty confusing and probably even lead them to think there's some kind of bug in the library. This is what an error looks like at the moment:

[error] ScalaComponentTest.scala:189: Cannot prove that japgolly.scalajs.react.example.TSome[A] =:= japgolly.scalajs.react.example.TNone.
[error]         .getSnapshotBeforeUpdate(???)
[error]                                 ^
[error] one error found

Nice UX in Scala is a bit of an art; in this case we'll do away with the generic TOption and create a custom construct for this one specific problem.

First, the new shape. Because this isn't generic anymore we no longer need the inner type member to be a type constructor which makes usage nicer too (i.e. T#Value instead of T#GetOrElse[Unit]):

sealed trait UpdateSnapshot {
  type Value
}

object UpdateSnapshot {
  sealed trait None extends UpdateSnapshot {
    override final type Value = Unit
  }

  sealed trait Some[A] extends UpdateSnapshot {
    override final type Value = A
  }
}

Easy enough. Now to improve the UX on failure. First we change the (implicit ev: SnapshotType =:= TNone) to (implicit ev: UpdateSnapshot.SafetyProof[U]) and create:

object UpdateSnapshot {

  @implicitNotFound("You can only specify getSnapshotBeforeUpdate once, and it has to be before " +
    "you specify componentDidUpdate, otherwise the snapshot type could become inconsistent.")
  sealed trait SafetyProof[U <: UpdateSnapshot]

  implicit def safetyProof[U <: UpdateSnapshot](implicit ev: U =:= UpdateSnapshot.None): SafetyProof[U] =
    null.asInstanceOf[SafetyProof[U]]
}

The (implicit ev: U =:= UpdateSnapshot.None) is still part of the solution, but this time it's indirect. It's a dependency on the availability of implicit SafetyProof. Thus the logic is still the same, just users will never see it as an error message.

The @implicitNotFound annotation on SafetyProof is the pudding. It will cause our custom error message to be displayed as a compilation error when someone breaks the rules.

Using null.asInstanceOf[SafetyProof[U]] is a performance optimisation; new SafetyProof[U]{} is fine too but I'd prefer to avoid the allocation and more importantly, by never actually creating or using SafetyProof it can be completely elided from Scala.JS output which means a smaller download for your webapp's end-users.

Finally, our new builder excerpt looks like this:

final class Builder[U <: UpdateSnapshot] {
  type SnapshotValue = U#Value

  def getSnapshotBeforeUpdate[A](f: ... => A)
                                (implicit ev: UpdateSnapshot.SafetyProof[U])
                                : Builder[UpdateSnapshot.Some[A]]

  def componentDidUpdate(f: SnapshotValue => ...)
                        : Builder[UpdateSnapshot.Some[SnapshotValue]]
}

And let's look at what errors look like now:

[error] ScalaComponentTest.scala:189: You can only specify getSnapshotBeforeUpdate once, and it has to be before you specify componentDidUpdate, otherwise the snapshot type could become inconsistent.
[error]         .getSnapshotBeforeUpdate(???)
[error]                                 ^
[error] one error found

Done

That's all. I hope you've enjoyed. If you're interested, the full patch that went into scalajs-react is here:

https://github.com/japgolly/scalajs-react/commit/ee81acf12c1039997460a7cac3d759fda6577533

Testing Scala.JS on Firefox & Chrome from SBT

It's been fantastic being able to write Scala and compile it to JavaScript thanks to Scala.JS. Going further, Scala.JS lets you write accompanying unit tests, run them from SBT, and choose a target environment from {Rhino, Node.JS, Phantom.JS}. If you needed DOM in your testing, your only option was Phantom.JS which seems great at first but—because it only simulates DOM—you soon discover that there are many cases in which its behaviour diverges from normal browsers (eg. DOM types for <td> tags), or isn't supported at all (text selection, anything to do with focus, more). Oh, and it's also riddled with bugs, many significant and long-standing. So while Phantom.JS's effort is appreciated, it's no substitute for a real browser. This was where the story ended until very recently.

Recently, the Scala.JS team released scala-js-env-selenium which allows you to use Selenium in the same way you would the other JS environments. That means that Scala.JS can now interface with real browsers, namely Firefox and Chrome. Awesome!

The next step, and the purpose of this blog post, is to effectively integrate it into your SBT/Scala/Scala.JS project. Let me tell you about my ideal environment and then I'll show you how to achieve it.

Goals

In my ideal environment:

• I write my Scala.JS unit tests the same way I already do, and I can continue to run them against Node.JS or Phantom.JS because it's fast. No changes there.
• To run tests against Firefox or Chrome, I simply prefix my SBT command with firefox: or chrome:.
  For example, firefox:test would run tests in Firefox, chrome:testOnly a.b.c would run my a.b.c test in Chrome.
• Testing in FF/Chrome doesn't require recompilation (All in the Land know Scalac is slow). It should use all of the same bits and bobs (especially the output JS) that normal tests use.
• I specify that certain tests will only run in FF and/or Chrome.
  For example, tests that use focus should skip Phantom.JS where I know focus doesn't work.
• I run testAll to run the same tests in all environments (fast-env & Firefox & Chrome). This will happen concurrently.
• I can use FF/Chrome headlessly (i.e. without the windows popping up when the browsers are launched and running).

1. Selenium support.

Add this to your project/plugins.sbt:

libraryDependencies += "org.scala-js" %% "scalajs-env-selenium" % "0.1.1"

Then install ChromeDriver which is needed so that Selenium can interface with Chrome.

2. General SBT Config

Create a file called project/InBrowserTesting.scala with the following content: This creates SBT configurations for each browser, then delegates most of the settings to the test:* settings.

3. Project-specific SBT config

Next you need to look at your existing SBT build settings.

For each cross-JVM/JS project, add:

  .configure(InBrowserTesting.cross)

For each JS-only project, add:

  .configure(InBrowserTesting.js)

For each JVM-only project, add:

  .configure(InBrowserTesting.jvm)

You might be wondering why JVM projects need any configuration at all. It's so that when testAll is run from a JVM project or the root project, you want it to run the JVM tests. Without this setting, testAll in a JVM project would do nothing at all, and testAll from the root would only run the JS tests.

4. Environment-Dependent Tests

I often forget that I'm writing JS when I write Scala.JS. As we're in JS land, to determine our environment all we have to do is check the user-agent. How you skip tests depends on the test framework you're using but all you have to do is put something like this in your test code:

  if (JsEnvUtils.isRealBrowser) {
    // test here
  } else {
    // skip
  }

5. Headlessness

Super simple (unless you're a Windows user). Install xvfb, "X Virtual FrameBuffer", which starts X without a graphics display.

Here are two different ways you can use it:

1. Either start it in a separate window via
   

   Xvfb :1
   

   or in the background
   

   nohup Xvfb :1 &
   

   then launch SBT like this:
   

   DISPLAY=:1 sbt
   

2. This tip comes from danielkza (thanks!). You can simply prepend your SBT command with xvfb-run -a to have an X server spun up on demand without the need to start it yourself. Beware though, that xvfb-run is a bit naive and susceptible to race conditions so while it'll be fine on your local machine, it may cause you problems on your CI server or similar.

You'll no longer see any Firefox and Chrome windows; all your output just appears in the SBT console as usual. Too easy.

Note: The :1 indicates the X display-number, which is a means to uniquely identify an X server on a host. The 1 is completely arbitrary—you can choose any number so long as there isn't another X server running that's already associated with it.

Done.

All goals described are thus achieved.
I hope you found this helpful.
Happy coding!