Mark Needham

Thoughts on Software Development

Archive for the ‘Scala’ Category

Scala: Creating an Xml element with an optional attribute

with one comment

We have a lot of Xml in our application and one of the things that we need to do reasonably frequently in our test code is create elements which have optional attributes on them.

Our simple first approach looked like this:

def createElement(attribute: Option[String]) = if(attribute.isDefined) <p bar={attribute.get} /> else <p />

That works but it always seemed like we should be able to do it in a simpler way.

Our first attempt was this:

def createElement(attribute: Option[String]) = <p bar={attribute} />

But that ends up in a compilation error:

error: overloaded method constructor UnprefixedAttribute with alternatives:
  (key: String,value: Option[Seq[scala.xml.Node]],next: scala.xml.MetaData)scala.xml.UnprefixedAttribute <and>
  (key: String,value: String,next: scala.xml.MetaData)scala.xml.UnprefixedAttribute <and>
  (key: String,value: Seq[scala.xml.Node],next1: scala.xml.MetaData)scala.xml.UnprefixedAttribute
 cannot be applied to (java.lang.String, Option[String], scala.xml.MetaData)
       def createElement1(attribute: Option[String]) = <p bar={attribute} />

We really need to extract the string value from the option if there is one and not do anything if there isn’t one but with the above approach we try to shove an option in as the attribute value. Unfortunately there isn’t an overload of the constructor which lets us do that.

Eventually one of my colleagues suggested we try passing null in as the attribute value if we had a None option:

def createElement(attribute: Option[String]) = <p bar={attribute.getOrElse(null)} />

Which works pretty well:

scala> createElement(Some("mark"))
res0: scala.xml.Elem = <p bar="mark"></p>
 
scala> createElement(None)
res1: scala.xml.Elem = <p ></p>

Written by Mark Needham

October 25th, 2011 at 8:38 pm

Posted in Scala

Tagged with

Scala: Replacing a trait with a fake one for testing

with one comment

We recently wanted to replace a trait mixed into one of our classes with a fake version to make it easier to test but forgot how exactly to do that!

The class is roughly like this:

trait Foo { def foo : String = "real foo" } 
class Mark extends Foo {}

We originally tried to replace it like this:

trait BrokenFakeFoo { def foo : String = "broken fake foo" }
val m = new Mark with BrokenFakeFoo
error: overriding method foo in trait Foo of type => String;
 method foo in trait BrokenFakeFoo of type => String needs `override' modifier
       val m = new Mark with BrokenFakeFoo

If m compiled it would have two versions of foo but it wouldn’t know which one to use, hence the error message.

Attempt two was this:

trait BrokenFakeFoo { override def foo : String = "broken fake foo" }
error: method foo overrides nothing
       trait BrokenFakeFoo { override def foo : String = "broken fake foo" }

As Uday pointed out, what we actually need to do is make our fake trait extend the original one and then override the method.

trait FakeFoo extends Foo { override def foo : String = "fake foo" }
val m = new Mark with FakeFoo
m.foo
> res5: String = fake foo

Since FakeFoo is the right most of the traits mixed into Mark its foo method will be used over the Foo one mixed into Mark on its class definition.

Written by Mark Needham

September 25th, 2011 at 10:24 am

Posted in Scala

Tagged with

Scala: for comprehensions with Options

with 2 comments

I’ve generally avoided using for expressions in Scala because the keyword reminds me of for loops in Java/C# and I want to learn to program in a less imperative way.

After working with my colleague Mushtaq I realised that in some cases using for comprehensions can lead to much more readable code.

An interesting use case where this is the case is when we want to create an object from a bunch of parameters that may or may not be set. i.e. a bunch of options.

For example we might take some input from the user where they have to enter their name but could choose to leave the field blank:

val maybeFirstName : Option[String] = Some("Mark")
val maybeSurname : Option[String] = None

We only want to create a Person if they have provided both names.

The for comprehension works quite well in allowing us to do this:

case class Person(firstName:String, surname:String)
scala> for { firstName <- maybeFirstName; surname <- maybeSurname } yield Person(firstName, surname)
res27: Option[Person] = None

If we set the surname to have a value:

val maybeSurname : Option[String] = Some("Needham")

Running the same for comprehension will yield a Person

scala> for { firstName <- maybeFirstName; surname <- maybeSurname } yield Person(firstName, surname)
res29: Option[Person] = Some(Person(Mark,Needham))

From what I understand when we have multiple values assigned using ‘<-' inside a for comprehension, each value will have flatMap called on it except for the last one which will have map called instead.

The equivalent code if we didn’t use a for comprehension would therefore look like this:

scala> maybeFirstName.flatMap { firstName => maybeSurname.map { surname => Person(firstName, surname) } } 
res43: Option[Person] = Some(Person(Mark,Needham))

For me the for comprehension expresses intent much better and it seems to excel even more as we add more values to the comprehension.

Written by Mark Needham

September 15th, 2011 at 10:21 pm

Posted in Scala

Tagged with

Scala: Do modifiers on functions really matter?

with 8 comments

A couple of colleagues and I were having an interesting discussion this afternoon about the visibility of functions which are mixed into an object from a trait.

The trait in question looks like this:

trait Formatting {
  def formatBytes(bytes: Long): Long = {
    math.round(bytes.toDouble / 1024)
  }
}

And is mixed into various objects which need to display the size of a file in kB like this:

class SomeObject extends Formatting {
 
}

By mixing that function into SomeObject any of the clients of SomeObject would now to be able to call that function and transform a bytes value of their own!

The public API of SomeObject is now cluttered with this extra method although it can’t actually do any damage to the state of SomeObject because it’s a pure function whose output depends only on the input given to it.

There are a couple of ways I can think of to solve the modifier ‘problem’:

  • Make formatBytes a private method on SomeObject
  • Put formatBytes on a singleton object and call it from SomeObject

The problem with the first approach is that it means we have to test the formatBytes function within the context of SomeObject which makes our test much more difficult than if we can test it on its own.

It also makes the discoverability of that function more difficult for someone else who has the same problem to solve elsewhere.

With the second approach we’ll have a dependency on that singleton object in our object which we wouldn’t be able to replace in a test context even if we wanted to.

While thinking about this afterwards I realised that it was quite similar to something that I used to notice when i was learning F# – the modifiers on functions don’t seem to matter if the data they operate on is immutable.

I often used to go back over bits of code I’d written and make all the helper functions private before realising that it made more sense to keep them public but group them with similar functions in a module.

I’m moving towards the opinion that if the data is immutable then it doesn’t actually matter that much who it’s accessible to because they can’t change the original version of that data.

private only seems to make sense if it’s a function mutating a specific bit of data in an object but I’d be interesting in hearing where else my opinion doesn’t make sense.

Written by Mark Needham

August 13th, 2011 at 2:10 am

Posted in Scala

Tagged with

Scala, WebDriver and the Page Object Pattern

with 6 comments

We’re using WebDriver on my project to automate our functional tests and as a result are using the Page Object pattern to encapsulate each page of the application in our tests.

We’ve been trying to work out how to effectively reuse code since some of the pages have parts of them which work exactly the same as another page.

For example we had a test similar to this…

class FooPageTests extends Spec with ShouldMatchers with FooPageSteps {
  it("is my dummy test") {
    ...
    then(iShouldNotSeeAnyCommonLinks())
  }
}

…where FooPageSteps extends CommonSteps which contains the common assertions:

trait FooPageSteps extends CommonSteps {
  override val page = new FooPage(driver)
}
trait CommonSteps {
  val page : FooPage
  val driver: HtmlUnitDriver
 
  def iShouldNotSeeAnyCommonLinks() {
    page.allCommonLinks.isEmpty should equal(true)
  }
}

FooPage looks like this:

class FooPage(override val driver:WebDriver) extends Page(driver) with CommonSection {
 
}
 
abstract class Page(val driver: WebDriver) {
  def title(): String = driver.getTitle;
}
 
trait CommonSection {
  val driver:WebDriver
  def allCommonLinks:Seq[String] = driver.findElements(By.cssSelector(".common-links li")).map(_.getText)
}

We wanted to reuse CommonSteps for another page like so:

trait BarPageSteps extends CommonSteps {
  override val page = new BarPage(driver)
}
 
class BarPage(override val driver:WebDriver) extends Page(driver) with CommonSection {
 
}

But that means that we need to change the type of page in CommonSteps to make it a bit more generic so it will work for BarPageSteps too.

Making it of type Page is not enough since we still need to be able to call the allCommonLinks which is mixed into FooPage by CommonSection.

We therefore end up with the following:

trait CommonSteps {
  val page : Page with CommonSection
  val driver: HtmlUnitDriver
 
  def iShouldNotSeeAnyCommonLinks() {
    page.allCommonLinks.isEmpty should equal(true)
  }
}

We’re able to mix in CommonSection just for this instance of Page which works pretty well for allowing us to achieve code reuse in this case!

Written by Mark Needham

August 9th, 2011 at 12:54 am

Posted in Scala

Tagged with ,

Scala: Prettifying test builders with package object

without comments

We have several different test builders in our code base which look roughly like this:

case class FooBuilder(bar : String, baz : String) {
	def build = new Foo(bar, baz)
}

In our tests we originally used them like this:

class FooPageTest extends Specs with ShouldMatchers {
	it("should let us load a foo") {
		when(databaseHas(FooBuilder(bar = "Bar", baz = "Bazz")))
		// and so on...
	}
}

This works well but we wanted our tests to only contain domain language and no implementation details.

We therefore started pulling out methods like so:

class FooPageTest extends Specs with ShouldMatchers {
	it("should let us load a foo") {
		when(databaseHas(aFooWithBarAndBaz("Bar", "Bazza")))
		// and so on...
	}
 
	def aFooWithBarAndBaz(bar:String, baz;String) = FooBuilder(bar = bar, baz = baz)
}

This was fine to start with but we eventually ended up with 10-12 different variations oh how Foo could be constructed, negating the value that the builder pattern provides.

Instead what we can do is use of an alias of FooBuilder to achieve something equally readable:

package object TestSugar {
	val aFooWith = FooBuilder
}

We can then use aFooWith like so:

import TestSugar._
 
class FooPageTest extends Specs with ShouldMatchers {
	it("should let us load a foo") {
		when(databaseHas(aFooWith(bar = "Bar", baz = "Bazza")))
		// and so on...
	}
}

We could also achieve that by renaming FooBuilder to aFooWith but that makes it much less discoverable whereas this solution lets us achieve both goals.

The package object approach isn’t really needed – we could easily put those vals onto an Object or Class but they don’t really seem to belong to any which is why we’ve gone for this approach.

Written by Mark Needham

July 26th, 2011 at 10:31 pm

Posted in Scala

Tagged with

Scala: Making it easier to abstract code

without comments

A couple of months ago I attended Michael Feathers’ ‘Brutal Refactoring’ workshop at XP 2011 where he opined that developers generally do the easiest thing when it comes to code bases.

More often than not this means adding to an existing method or existing class rather than finding the correct place to put the behaviour that they want to add.

Something interesting that I’ve noticed so far on the project I’m working on is that so far we haven’t been seeing the same trend.

Our code at the moment is comprised of lots of little classes with small amounts of logic in them and I’m inclined to believe that Scala as a language has had a reasonable influence on that.

The following quote from ‘Why programming languages?‘ sums it up quite well:

Sometimes the growing complexity of existing programming languages prompts language designers to design new languages that lie within the same programming paradigm, but with the explicit goal of minimising complexity and maximising consistency, regularity and uniformity (in short, conceptual integrity).

It’s incredibly easy to pull out a new class in Scala and the amount of code required to do so is minimal which seems to be contributing to the willingness to do so.

At the moment nearly all the methods in our code base are one line long and the ones which aren’t do actually stand out which I think psychologically makes you want to find a way to keep to the one line method pattern.

Traits

As I’ve mentioned previously we’ve been pulling out a lot of traits as well and the only problem we’ve had there is ensuring that we don’t end up testing their behaviour multiple times in the objects which mix-in the trait.

I tend to pull traits out when it seems like there might be an opportunity to use that bit of code rather than waiting for the need to arise.

That’s generally not a good idea but it seems to be a bit of a trade off between making potentially reusable code discoverable and abstracting out the wrong bit of code because we did it too early.

Companion Objects

The fact that we have companion objects in the language also seems to help us push logic into the right place rather than putting it into an existing class.

We often have companion objects which take in an XML node, extract the appropriate parts of the document and then instantiate a case class object.

In Summary

There’s no reason you couldn’t achieve the same things in C#¬†or Java but I haven’t seen code bases in those languages evolve in the same way.

It will be interesting to see if my observations remain the same as the code base increases in size.

Written by Mark Needham

July 23rd, 2011 at 12:05 pm

Posted in Scala

Tagged with

Scala: Companion Objects

without comments

One of the language features available to us in Scala which I think is having a big impact in helping us to make our code base easier to follow is the companion object.

We’ve been using companion objects quite liberally in our code base to define factory methods for our classes.

As I mentioned in a previous post a lot of our objects are acting as wrappers around XML documents and we’ve been pushing some of the data extraction from the XML into companion objects so that our classes can take in non XML values.

This means we can test the data extraction against the companion object and then create simpler tests against any other logic in the object because we don’t have to create XML documents in each of our tests.

The following is an example of a Foo object being constructed with data from an XML document:

object Foo {
   def apply(element: Node) = {
    val bar = element.attribute("bar").get.head.text
    val baz = (element \\ "baz").text
    new Foo(bar, baz)
  }
}

There is also some other logic around how a collection of Foos should be ordered and by using the companion object to parse the XML we can create a test with appropriate bar and baz values to test that.

case class Foo(bar: String, baz:String) extends Ordered[Foo] {
   def compare(that: Foo) = {
     // logic to compare Foos
   }
}

Before we had the companion object we were putting the logic to create Foo inside the object where it is created from which increased the complexity of that object and made it more difficult for people to read.

We’ve also been using this approach to build up page objects representing sub sections of a page in our Web Driver tests and it seems to work quite nicely there as well.

Written by Mark Needham

July 23rd, 2011 at 11:57 am

Posted in Scala

Tagged with

Scala: Rolling with implicit

with one comment

We’ve been coding in Scala on my project for around 6 weeks now and are getting to the stage where we’re probably becoming a big dangerous with our desire to try out some of the language features.

One that we’re trying out at the moment is the implicit key word which allows you to pass arguments to objects and methods without explicitly defining them in the parameter list.

The website we’re working on needs to be accessible in multiple languages and therefore we need to be able to translate some words before they get displayed on the page.

Most of the time it’s just static labels which need to be internationalised but there are a few words which are retrieved from the database and aren’t as easy to deal with.

We introduced the idea of the LanguageAwareString which acts as a wrapper around a String and has its own toString method which delegates to a Language class which contains a dictionary

It’s defined like this:

case class LanguageAwareString(ignorantValue:String)(implicit val language : Language) {
  def toString = language.translate(ignorantValue)
}

We didn’t want to have to pass Language to the LanguageAwareString factory method every time we’re going to be calling it in quite a few places.

We therefore create an implicit val at the beginning of our application in the Scalatra entry code

class Controllers extends ScalatraFilter with ScalateSupport {
	...
	implicit def currentLanguage : Language = // work out the current language
}

As I understand it, whenever the Scala compiler encounters an implicit it looks in its execution scope for any value defined as implicit with the expected type.

As long as there’s only one such value in the scope it make use of that value but if there’s more than one we’d see a compilation error since it wouldn’t know which one to use.

We therefore needed to define Language as an implicit on all the classes/methods which the code follows on its way down to LanguageAwareStrong.

The problem we’ve had is that it’s not immediately obvious what’s going on to someone who hasn’t come across implicit before and we therefore end up having to go the above each time!

We’ve decided that to ease that transition we’d explicitly pass Language down through the first few classes so that it’s more obvious what’s going on.

We therefore have code like this in a few places:

new ObjectThatTakesLanguageImplicitly(someArg)(currentLanguage)

Maybe we can phase that out as people get used to implicit or maybe we’ll just get rid of implicit and decide it’s not worth the hassle!

Written by Mark Needham

July 19th, 2011 at 6:39 am

Posted in Scala

Tagged with

Scala: An attempt to eradicate the if

with 3 comments

In a previous post I included a code sample where we were formatting a page range differently depending on whether the start page and end pages were the same.

The code looked like this:

trait PageAware {
  def startPage:String
  def endPage:String
  def pageRange = if(firstPage == lastPage) "page %s".format(firstPage) else "pages %s-%s".format(firstPage, lastPage)
}

Looking at the if statement on the last line we were curious whether it would be possible to get rid of it and replace it with something else.

In Java we could use the ternary operator:

public class PageAware {
  public String pageRange() {
    return (firstPage == lastPage) ? String.format("page %s", firstPage) : String.format("pages %s-%s", firstPage, lastPage)
  }
}

The if/else statement in Scala is supposed to replace that as far as I understand but I think the ternary operator looks neater.

Beyond defining that we played around with some potential alternatives.

We could use a Map to store the true and false values as keys:

trait PageAware {
  def pageRange = Map(true -> "page %s", false -> "pages %s-%s")(firstPage == lastPage).format(firstPage, lastPage)
}

Uday came up with a pattern matching solution which looks like this:

trait PageAware {
  def pageRange = ((firstPage, lastPage) match { case (`firstPage`, `firstPage`) => "page %s" case _ => "pages %s-%s"}).format(firstPage, lastPage)
}

Unfortunately both of these solutions are significantly less readable than the if/else one so it seems like this is one of the situations where it doesn’t actually make sense to get rid of it.

Written by Mark Needham

July 12th, 2011 at 10:50 pm

Posted in Scala

Tagged with