Benchmarking Express vs Vapor
Migrating gitignore.io from Express To Vapor
I have maintained gitignore.io since February of 2013. Recently, I decided to update the website from the ground up to version 2.0 with lofty goals such as snapshot tests, public metrics, localization, 100% test cases, and 100% code coverage. My goal was to increase site performance under the same server constraints. I’ve been running the site using one free Heroku Dyno since 2013 and as the site slowly grows, I want to ensure that the site is fast.
Vapor fit my 2.0 requirements needs. I tried Perfect and looked at IBM Kitura, but at the time neither had a good localization solution. The server-side Swift community is growing rapidly and each framework solves slightly different problems when it comes to building web applications and web services. That being said, all three communities are awesome and extremely helpful. If you are choosing a framework, you need to research at least these 3 and figure out which one fits your needs the best.
I am running these benchmarks on my late 2015 iMac with both web applications running with the production flag enabled.
wrk- testing threaded requests for 60 seconds
$ wrk -t30 -c1000 -d60s http://localhost:8000 Running 1m test @ http://localhost:8000 30 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 424.38ms 99.22ms 731.72ms 79.24% Req/Sec 70.07 49.81 292.00 65.66% 120637 requests in 1.00m, 612.64MB read Socket errors: connect 42, read 1125, write 0, timeout 0 Requests/sec: 2007.11 Transfer/sec: 10.19MB
$ wrk -t30 -c1000 -d60s http://localhost:8080 Running 1m test @ http://localhost:8080 30 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 9.93ms 28.77ms 282.93ms 95.88% Req/Sec 750.62 449.89 1.78k 59.26% 126771 requests in 1.00m, 690.33MB read Socket errors: connect 0, read 1024, write 0, timeout 2 Requests/sec: 2110.11 Transfer/sec: 11.49MB
Vapor slightly edges out Express in Requests/sec and Transfer/sec. Where you really see Vapor take the lead is in the threading latency and request per second. Expresses average latency per thread is 42 times slower than Vapor’s. On each thread, Vapor also performs 10 times the requests per second than Express.
vmmap- virtual memory footprint
VIRTUAL ALLOCATION BYTES REGION MALLOC ZONE SIZE COUNT ALLOCATED % FULL COUNT =========== ======= ========= ========= ====== ====== DefaultMallocZone 72.0M 7668 4149K 5% 9
VIRTUAL ALLOCATION BYTES REGION MALLOC ZONE SIZE COUNT ALLOCATED % FULL COUNT =========== ======= ========= ========= ====== ====== DefaultMallocZone 87.0M 20702 2746K 3% 18
Express edges Vapor out by using about 20% less memory. For my use case, I’m willing to sacrifice as much memory as possible to have a quicker web application.
A lot of people ask me if server-side Swift is ready for production. The answer to that question depends on your needs, but in my case — it is definitely ready. I’ve been running Vapor for about 2 months now without any problems. The community on the Slack channel is also excellent. As the server-side Swift community does more work to establish Server Foundation, the server-side Swift community will see even greater performance gains.
Check out gitignore.io to see a production server-side Swift web application.