Benchmarking Express vs Vapor

Migrating gitignore.io from Express To Vapor #

I have maintained gitignore.io since February of 2013. Recently, I decided to update the website from the ground up to version 2.0 with lofty goals such as snapshot tests, public metrics, localization, 100% test cases, and 100% code coverage. My goal was to increase site performance under the same server constraints. I’ve been running the site using one free Heroku Dyno since 2013 and as the site slowly grows, I want to ensure that the site is fast.

Why Vapor? #

Vapor fit my 2.0 requirements needs. I tried Perfect and looked at IBM Kitura, but at the time neither had a good localization solution. The server-side Swift community is growing rapidly and each framework solves slightly different problems when it comes to building web applications and web services. That being said, all three communities are awesome and extremely helpful. If you are choosing a framework, you need to research at least these 3 and figure out which one fits your needs the best.

Benchmarks #

I am running these benchmarks on my late 2015 iMac with both web applications running with the production flag enabled.

The JavaScript version of gitignore.io website uses Kraken JS which sits on top of Express JS. Swift uses the Vapor framework.

• wrk - testing threaded requests for 60 seconds

  Express

  $ wrk -t30 -c1000 -d60s http://localhost:8000 
  Running 1m test @ http://localhost:8000
    30 threads and 1000 connections
    Thread Stats   Avg      Stdev     Max   +/- Stdev
      Latency   424.38ms   99.22ms 731.72ms   79.24%
      Req/Sec    70.07     49.81   292.00     65.66%
    120637 requests in 1.00m, 612.64MB read
    Socket errors: connect 42, read 1125, write 0, timeout 0
  Requests/sec:   2007.11
  Transfer/sec:     10.19MB
  

  Vapor

  $ wrk -t30 -c1000 -d60s http://localhost:8080
      Running 1m test @ http://localhost:8080
        30 threads and 1000 connections
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency     9.93ms   28.77ms 282.93ms   95.88%
          Req/Sec   750.62    449.89     1.78k    59.26%
        126771 requests in 1.00m, 690.33MB read
        Socket errors: connect 0, read 1024, write 0, timeout 2
      Requests/sec:   2110.11
      Transfer/sec:     11.49MB
  

  Vapor slightly edges out Express in Requests/sec and Transfer/sec. Where you really see Vapor take the lead is in the threading latency and request per second. Expresses average latency per thread is 42 times slower than Vapor’s. On each thread, Vapor also performs 10 times the requests per second than Express.

• vmmap - virtual memory footprint

  Express

                   VIRTUAL ALLOCATION      BYTES          REGION
  MALLOC ZONE         SIZE      COUNT  ALLOCATED  % FULL   COUNT
  ===========      =======  =========  =========  ======  ======
  DefaultMallocZone  72.0M       7668      4149K      5%       9
  

  Vapor

                   VIRTUAL ALLOCATION      BYTES          REGION
  MALLOC ZONE         SIZE      COUNT  ALLOCATED  % FULL   COUNT
  ===========      =======  =========  =========  ======  ======
  DefaultMallocZone  87.0M      20702      2746K      3%      18
  

  Express edges Vapor out by using about 20% less memory. For my use case, I’m willing to sacrifice as much memory as possible to have a quicker web application.

Conclusion #

A lot of people ask me if server-side Swift is ready for production. The answer to that question depends on your needs, but in my case — it is definitely ready. I’ve been running Vapor for about 2 months now without any problems. The community on the Slack channel is also excellent. As the server-side Swift community does more work to establish Server Foundation, the server-side Swift community will see even greater performance gains.

Check out gitignore.io to see a production server-side Swift web application.