Tuesday, June 3, 2014

Server Load Testing using Gatling: Some Lessons Learned

I had to load test a server using Gatling to see how many requests per second it could handle. The server built up a cache over time as it stored the input parameters, so that it would not have to look them up on subsequent identical queries. This led to a development I had not anticipated.

Gatling's http behavior is to issue a new request immediately upon receiving a response, unless there is a deliberate pause inserted. My test did not originally include such a pause, as I thought I would find the max rate through stress. In the case of cached queries, the server responded in single-digit milliseconds, which led to Gatling spawning a huge number of requests per second, overwhelming the server. Once I understood this, I inserted a pause into the Gatling scenario. I used a 1 second pause, so that each Gatling "user" could only send at most one request per second. Thus, the overall throughput rate is measured by the number of users: 1000 users at 1 request per second each would mean a total requests-per-second rate of 1000. In this way, I had to keep upping the number of users to find the limit, which was somewhat backwards than what I had originally intended.

Along the way I learned some other useful techniques, such as how to create and use dynamic values for query parameters. I wanted a different random value each time. To do this I used "session attributes". You preface one "exec" with another which calls a function.

exec(func)
      .exec( // http stuff
)

You could also use a "feeder" function to pre-generate such values or a feeder with .csv files for static data.

Here is the sample source code (using Gatling 2.0 jar files)  (SampleScenario.scala)

package basic

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
import io.gatling.http.Headers.Names._  // to become HeaderNames in later versions of 2.x
import scala.concurrent.duration._
import bootstrap._  // to be deprecated in later versions of 2.x, to be commented out then
import scala.util.Random
import io.gatling.core.feeder.Feeder


trait WWWLoadCommandLineArgsSample {
  val users = Integer.getInteger("users", 1)
  val rampup = intToFiniteDuration(Integer.getInteger("rampup", 0))
  val random = new scala.util.Random
  var test_duration = intToFiniteDuration(Integer.getInteger("test_duration", 60))
}

class BasicSimulationGatling2 extends Simulation with WWWLoadCommandLineArgsSample{

  val chars = ('A' to 'Z')
  val quarterUsers = (users.toFloat / 4.0).toInt
  val halfUsers = (users.toFloat / 2.0).toInt

  val chooseRandomACodes = exec((session) => {
    val code1 = (random.nextInt(898) * 100) + 10000
    val code2 = code1 + 100
    val a_code: String = "[" + code1.toString() + "-" + code2.toString() + "]"
    session.set("a_code", a_code)
  })

  val chooseRandomBCodes = exec((session) => {
    val b_code : String = chars(util.Random.nextInt(chars.length)).toString + random.nextInt(10).toString() + chars(util.Random.nextInt(chars.length)).toString
    session.set("b_code", b_code)
  })

  val chooseRandomCCodes = exec((session) => {
    val c_code : String = random.nextInt(9).toString()  + random.nextInt(9).toString()  + chars(util.Random.nextInt(chars.length)).toString
    session.set("c_code", c_code)
  })

  val httpConf = http
    .baseURL("http://www.the_server.com")
    .acceptCharsetHeader("ISO-8859-1,utf-8;q=0.7,*;q=0.7")
    .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
    .acceptEncodingHeader("gzip, deflate")
    .acceptLanguageHeader("fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3")
    .disableCaching

  val scn_a = scenario("A Codes")
    .during(test_duration) {
    exec(chooseRandomACodes)
      .exec(
        http("a_code")
          .get("/")
          .queryParam("aCode", "${a_code}")
          .check(status.is(200))
      ).pause(1000 milliseconds)
  }

  val scn_b = scenario("B Codes")
    .during(test_duration) {
    exec(chooseRandomACodes)
      .exec(
        http("b_code")
          .get("/")
          .queryParam("bCode", "${b_code}")
          .check(status.is(200))
      ).pause(1000 milliseconds)
  }

  val scn_c = scenario("C Codes")
    .during(test_duration) {
    exec(chooseRandomCCodes)
      .exec(
        http("c_code")
          .get("/")
          .queryParam("cCode", "${c_code}")
          .check(status.is(200))
      ).pause(1000 milliseconds)
  }

  // divide the scenarios among the total users, share the rampup, duration and protocolConfig
  setUp(scn_a.inject(ramp(halfUsers) over (rampup)), // ramp to become 'rampUsers' in later versions of 2.x
    scn_b.inject(ramp(quarterUsers) over (rampup)),
    scn_c.inject(ramp(quarterUsers) over (rampup)))
    .protocols(httpConf)
}

To Run:

export JAVA_OPTS="-Dusers=1500 -Dramp=120 -Dduration=3600"
./gatling.sh -s basic. BasicSimulationGatling2 -sf user-files/simulations/basic

Where gatling.sh =

#!/bin/sh
OLDDIR=`pwd`
BIN_DIR=`dirname $0`
cd ${BIN_DIR}/.. && DEFAULT_GATLING_HOME=`pwd` && cd ${OLDDIR}

GATLING_HOME=${GATLING_HOME:=${DEFAULT_GATLING_HOME}}
GATLING_CONF=${GATLING_CONF:=$GATLING_HOME/conf}

export GATLING_HOME GATLING_CONF

echo "GATLING_HOME is set to ${GATLING_HOME}"

JAVA_OPTS="-server -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms512M -Xmx512M -Xmn100M -Xss2M -XX:+HeapDumpOnOutOfMemoryError -XX:+AggressiveOpts -XX:+OptimizeStringConcat -XX:+UseFastAccessorMethods -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly ${JAVA_OPTS}"

CLASSPATH="$GATLING_HOME/lib/*:$GATLING_CONF:${JAVA_CLASSPATH}"

java $JAVA_OPTS -cp $CLASSPATH com.excilys.ebi.gatling.app.Gatling $@