Mark Needham

Thoughts on Software Development

Scala: Converting an input stream to a string

with 8 comments

I was playing around with Scala over the weekend and one thing that I wanted to do was get the data from a HTTP response as a string so that I could parse the xml returned.

The data source is fairly small so loading the stream into memory wasn’t a problem.

Carlos pointed me to a bit of Java code that did this and I converted it as literally as possible into Scala.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
 def convertStreamToString(is: InputStream): String = {
    val reader = new BufferedReader(new InputStreamReader(is));
    val sb = new StringBuilder();
 
    var line : String = null;
    try {
      while ((line = reader.readLine()) != null) {
        sb.append(line + "\n");
      }
    } catch {
      case e: IOException => e.printStackTrace();
    } finally {
      try {
        is.close();
      } catch {
        case e: IOException => e.printStackTrace();
      }
    }
 
    sb.toString();
  }

The problem with this bit of code becomes clear when we try to run it through the REPL:

7:warning: comparing values of types Unit and Null using `!=' will always yield true
             while ((line = reader.readLine()) != null) {

In Java the expression ‘(line = reader.readLine())’ would return the value of ‘reader.readLine()’ as far as I understand. We can then compare that to null.

In Scala the evaluation of that expression is ‘Unit’ so if we ever run this function we end up in an infinite loop and eventually run out of memory.

I rewrote this function making use of recursion to get around the problem:

  def convertStreamToString(is : InputStream) : String = {
    def inner(reader : BufferedReader, sb : StringBuilder) : String = {
      val line = reader.readLine()
      if(line != null) {
        try {
          inner(reader, sb.append(line + "\n"))
        } catch {
          case e : IOException => e.printStackTrace()
        } finally {
          try {
            is.close()
          } catch {
            case e : IOException => e.printStackTrace()
          }
        }
 
      }
      sb.toString()
    }
 
    inner(new BufferedReader(new InputStreamReader(is)), new StringBuilder())
  }

The code is still peppered with error handling statements but it now allows us to convert the stream into a string.

Is there a better/cleaner/more Scala-esque way to do this?

Be Sociable, Share!

Written by Mark Needham

October 26th, 2009 at 6:32 am

Posted in Scala

Tagged with

  • http://www.davidron.com David Ron

    Couldn’t you just delegate off to a Java library to handle this for you? How about Apache IOUtils?

    http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toString(java.io.InputStream)

  • http://www.markhneedham.com Mark Needham

    Yeh I guess I could. I was just playing around with code in the REPL so I was seeing what I could do just with the libraries available there.

    Haven’t worked out how to import 3rd party libraries into that environment yet. I guess that would be a good next thing to work out!

    Thanks for the link though, will check that out.

  • http://markthomas.info/blog Mark Thomas

    Hi Mark, try this:

    import scala.io.Source
    import java.io.InputStream

    def convertStreamToString(is: InputStream) : String =
    Source.fromInputStream(is).getLines.reduceLeft(_ + _)

    Source has a load of useful methods like fromFile or fromURL.

  • Pingback: DotNetBurner - XML

  • Pingback: Tweets that mention Scala: Converting an input stream to a string at Mark Needham -- Topsy.com

  • wic

    I realize java is showing age, but seriously… you should not ever need 20 lines of code to do this. Not even if you wrote it in C. And recursive as well? This should be a one-liner nowadays. Maybe two.

  • ray

    I also wanted to do this and found that it is in fact very simple using Scala’s Source class:
    io.Source.fromInputStream(someStream).getLines.mkString

    It also reads files, urls etc…

  • http://www.socialrpm.com Bikash

    while ((line = reader.readLine()) != null) {
    when u say line=reader.readLine() its a assignment which return Unit in scala, and its always not null .

    so just have to compare against variable line

    something like this shud work

    while({line=reader.readline();line!=null) {
    doSomething()
    }