Scala高性能编程

使用@transient

根据Programming in Scala书中对@transient的介绍:

Scala provides a @transient annotation for fields that should not be serialized at all. If you mark a field as @transient, then the frame- work should not save the field even when the surrounding object is serialized. When the object is loaded, the field will be restored to the default value for the type of the field annotated as @transient.

所以在case class定义中,如果类中定义的field不需要序列化,就应该声明@transient标记。例如在spark中有如下代码:

case class WeekOfYear(child: Expression) extends UnaryExpression with ImplicitCastInputTypes {

  @transient private lazy val c = {
    val c = Calendar.getInstance(TimeZone.getTimeZone("UTC"))
    c.setFirstDayOfWeek(Calendar.MONDAY)
    c.setMinimalDaysInFirstWeek(4)
    c
  }
}

当然,如果一个trait要被case class继承,则trait中声明的field如果不需要序列化,也应该申明@transient:

trait CaseWhenLike extends Expression {
  @transient lazy val whenList =
    branches.sliding(2, 2).collect { case Seq(whenExpr, _) => whenExpr }.toSeq
}
case class CaseWhen(branches: Seq[Expression]) extends CaseWhenLike 

StringBuilder, StringConcat和StringInterpolation

从Java获得的经验,String是一个不变的对象,为避免内存消耗,提高效率,若要连接大量字符串,应避免使用+连接,推荐使用StringBuidler。

在Scala中,除了这两种方式外,还提供了另外一种称之为String Interpolation的方式,我们常常会使用s String Interpolation,使得字符串拼接的可读性更好。

那么这三种方式在性能、内存消耗方面的表现究竟如何呢?我们来编写如下代码来对比(案例参考StackOverflow的String interpolation vs concatenation):

class Employee(name: String, age: Int) {
  private var s = 0.0

  def salary = s

  def salary_=(s: Double) = this.s = s

  def toStringConcat(): String = {
    "Name: " + name + ", age: " + age + ", salary: " + salary
  }

  def toStringInterpolation(): String = {
    s"Name: $name, age: $age, salary: $salary"
  }

  def toStringBuilder(): String = {
    val sb = new StringBuilder
    sb.append("Name: ")
    sb.append(name)
    sb.append(", age: ")
    sb.append(age)
    sb.append(", salary: ")
    sb.append(salary)
    sb.toString()
  }
}

object Program {
  val empl = new Employee("John", 30)
  empl.salary = 10.50
  val times = 10000000

  def main(args: Array[String]): Unit = {
    // warming-up
    val resultConcat = empl.toStringConcat
    val resultInterpol = empl.toStringInterpolation
    val resultBuilder = empl.toStringBuilder
    println("Concat -> " + resultConcat)
    println("Interpol -> " + resultInterpol)
    println("Builder -> " + resultBuilder)

    val secondsConcat0 = run(empl.toStringConcat)
    val secondsConcat1 = run(empl.toStringConcat)
    val secondsConcat2 = run(empl.toStringConcat)

    println("Concat-0: " + secondsConcat0 + "s")
    println("Concat-1: " + secondsConcat1 + "s")
    println("Concat-2: " + secondsConcat2 + "s")
  }

  def run(call: => String): Double = {
    val startTime = System.nanoTime()
    var result = ""
    for (i <- 0 until times) {
      result = call
    }
    val endTime = System.nanoTime()

    val elapsedTime = endTime - startTime
    elapsedTime / 1000000000.0
  }
}

定义了一个Employee,分别以三种方式来组装Employee的显示字符串。然后调用不同的实现方式分别运行,每次执行10000000遍,连续运行三次。观察运行时间,结果为:

Concat-0: 3.174632445s
Concat-1: 2.282592063s
Concat-2: 2.326627284s

Interpol-0: 4.451407519s
Interpol-1: 3.786795428s
Interpol-2: 3.322004628s

Builder-0: 2.276392409s
Builder-1: 1.966719917s
Builder-2: 2.028558008s

采用StringBuilder的方式自然最优,这点不用质疑;不过代码的简洁性与可读性却是最差的。String Interpolation的方式在可读性方面是最优雅的,也是我最喜欢使用的方式,但在性能表现上却最差。这却有些出乎我意料之外。通过反编译.class文件,可以得到toStringConcat()与toStringInterpolation()方法的真实实现。看起来,在实现toStringConcat()方法时,虽然调用了StringBuilder的append()方法,但由于在传参过程中进行了装箱工作,在一定程度上影响了性能;toStringInterpolation()则利用了数组,性能上存在一定程度的损耗。

public String toStringConcat()
  {
    return new StringBuilder().append("Name: ").append(this.name).append(", age: ").append(BoxesRunTime.boxToInteger(this.age)).append(", salary: ").append(BoxesRunTime.boxToDouble(salary())).toString();
  }

  public String toStringInterpolation()
  {
    return new StringContext(Predef..MODULE$.wrapRefArray((Object[])new String[] { "Name: ", ", age: ", ", salary: ", "" })).s(Predef..MODULE$.genericWrapArray(new Object[] { this.name, BoxesRunTime.boxToInteger(this.age), BoxesRunTime.boxToDouble(salary()) }));
  }

再来看看内存的消耗情况。执行toStringConcat()方法,通过jconsole可以得到如下数据,其中内存的最高峰值为520.9Mb:

△ toStringConcat的性能Benchmark

下图是执行toStringInterpolation的数据,其中内存的最高峰值为248.5Mb:

△ Interpolation的性能Benchmark

执行toStringBuilder获得Benchmark为:

△ toStringBuilder的性能Benchmak

看起来String Interpolation是以时间换取了空间,虽然执行最慢,但在内存消耗上却是最优的。这便是编程的神秘之处,因为它没有标准答案,需要睿智地根据当前环境选择适合它的实现方案。

2015-09-01 16:20151Scala