until nil {

Some AppEngine Templates Benchmark Icing.

13 Sep 2010

I could’t hang up the call after the last 4 benchmarks (1, 2, 3, 4), but no, I am not back to add Django and Kay to the benchmarks… I was asked to include them and I should have done better than ditching them, seriously, I am sorry to all of those who would have liked them compared with the bunch included over here. Both are impeccable, well oiled, all-in-one monolithic frameworks, but far from what I am looking for: fast, elegant and modular web applications components. Besides, I would have liked to include them anyway, but their well planned nature makes them a bit uneasy to mount beside others apps inside a single AppEngine instance.

So, what I did instead, is welcome smaller request handlers, keeping the same templating engines in the front end. Now included is a very minimal bare metal WSGI handler implementation, as well as webob and a re-implementation of webapp in webapp2. I also took time to implement a variation on Tenjin, with a memcached cache combined.

The best part is that it didn’t turned out as expected, WSGI should have logically performed the best, but it didn’t! Maybe I did a bad implementation? Or could there be an unseen flaw in the benchmark? If you’d like, you can review the code in the benchmark’s repository on Github. Anyway, Here are the new compiled results, as always, take them with a grain of salt.

Low Overhead:

         WSGI w/Templator: | 250x, 0 failed |   8.263s overall, 0.033s each | score:   1.000 |
       web.py w/Templator: | 250x, 0 failed |   8.284s overall, 0.033s each | score:   1.003 |
            WSGI w/Tenjin: | 250x, 0 failed |   8.289s overall, 0.033s each | score:   1.003 |
            tipfy w/Jinja: | 250x, 0 failed |   8.297s overall, 0.033s each | score:   1.004 |
           webob w/Tenjin: | 250x, 0 failed |   8.297s overall, 0.033s each | score:   1.004 |
        tipfy w/Templator: | 250x, 0 failed |   8.309s overall, 0.033s each | score:   1.006 |
  webapp w/SimpleTemplate: | 250x, 0 failed |   8.311s overall, 0.033s each | score:   1.006 |
       webapp w/Templator: | 250x, 0 failed |   8.313s overall, 0.033s each | score:   1.006 |
    WSGI w/SimpleTemplate: | 250x, 0 failed |   8.334s overall, 0.033s each | score:   1.009 |
          web.py w/Tenjin: | 250x, 0 failed |   8.336s overall, 0.033s each | score:   1.009 |
           tipfy w/Tenjin: | 250x, 0 failed |   8.358s overall, 0.033s each | score:   1.011 |
        flask w/Templator: | 250x, 0 failed |   8.361s overall, 0.033s each | score:   1.012 |
   webob w/SimpleTemplate: | 250x, 0 failed |   8.365s overall, 0.033s each | score:   1.012 |
          bottle w/Tenjin: | 250x, 0 failed |   8.366s overall, 0.033s each | score:   1.012 |
        webob w/Templator: | 250x, 0 failed |   8.384s overall, 0.034s each | score:   1.015 |
 webapp2 w/SimpleTemplate: | 250x, 0 failed |   8.394s overall, 0.034s each | score:   1.016 |
   tipfy w/SimpleTemplate: | 250x, 0 failed |   8.404s overall, 0.034s each | score:   1.017 |
  bottle w/SimpleTemplate: | 250x, 0 failed |   8.408s overall, 0.034s each | score:   1.017 |
           flask w/Tenjin: | 250x, 0 failed |   8.417s overall, 0.034s each | score:   1.019 |
       bottle w/Templator: | 250x, 0 failed |   8.419s overall, 0.034s each | score:   1.019 |
   flask w/SimpleTemplate: | 250x, 0 failed |   8.435s overall, 0.034s each | score:   1.021 |
          webapp w/Django: | 250x, 0 failed |   8.437s overall, 0.034s each | score:   1.021 |
     webapp w/TenjinCache: | 250x, 0 failed |   8.451s overall, 0.034s each | score:   1.023 |
    webapp2 w/TenjinCache: | 250x, 0 failed |   8.458s overall, 0.034s each | score:   1.024 |
     web.py w/TenjinCache: | 250x, 0 failed |   8.461s overall, 0.034s each | score:   1.024 |
            flask w/Jinja: | 250x, 0 failed |   8.471s overall, 0.034s each | score:   1.025 |
         webapp2 w/Django: | 250x, 0 failed |   8.495s overall, 0.034s each | score:   1.028 |
      webob w/TenjinCache: | 250x, 0 failed |   8.503s overall, 0.034s each | score:   1.029 |
  web.py w/SimpleTemplate: | 250x, 0 failed |   8.512s overall, 0.034s each | score:   1.030 |
            WSGI w/Django: | 250x, 0 failed |   8.535s overall, 0.034s each | score:   1.033 |
           webob w/Django: | 250x, 0 failed |   8.537s overall, 0.034s each | score:   1.033 |
           webapp w/Breve: | 250x, 0 failed |   8.550s overall, 0.034s each | score:   1.035 |
           flask w/Django: | 250x, 0 failed |   8.571s overall, 0.034s each | score:   1.037 |
          webapp w/Tenjin: | 250x, 0 failed |   8.581s overall, 0.034s each | score:   1.038 |
          webapp2 w/Breve: | 250x, 0 failed |   8.596s overall, 0.034s each | score:   1.040 |
      tipfy w/TenjinCache: | 250x, 0 failed |   8.600s overall, 0.034s each | score:   1.041 |
            webob w/Breve: | 250x, 0 failed |   8.618s overall, 0.034s each | score:   1.043 |
            tipfy w/Breve: | 250x, 0 failed |   8.630s overall, 0.035s each | score:   1.044 |
          bottle w/Django: | 250x, 0 failed |   8.640s overall, 0.035s each | score:   1.046 |
         webapp2 w/Tenjin: | 250x, 0 failed |   8.673s overall, 0.035s each | score:   1.050 |
           bottle w/Breve: | 250x, 0 failed |   8.679s overall, 0.035s each | score:   1.050 |
           web.py w/Breve: | 250x, 0 failed |   8.696s overall, 0.035s each | score:   1.052 |
      webapp2 w/Templator: | 250x, 0 failed |   8.707s overall, 0.035s each | score:   1.054 |
           tipfy w/Django: | 250x, 0 failed |   8.789s overall, 0.035s each | score:   1.064 |
      flask w/TenjinCache: | 250x, 0 failed |   8.816s overall, 0.035s each | score:   1.067 |
     bottle w/TenjinCache: | 250x, 0 failed |   8.863s overall, 0.035s each | score:   1.073 |
          web.py w/Django: | 250x, 0 failed |   8.905s overall, 0.036s each | score:   1.078 |
       WSGI w/TenjinCache: | 250x, 0 failed |   8.977s overall, 0.036s each | score:   1.086 |
             WSGI w/Breve: | 250x, 0 failed |   9.002s overall, 0.036s each | score:   1.089 |
            webob w/Jinja: | 250x, 0 failed |   9.156s overall, 0.037s each | score:   1.108 |
          webapp2 w/Jinja: | 250x, 0 failed |   9.211s overall, 0.037s each | score:   1.115 |
            flask w/Breve: | 250x, 0 failed |   9.217s overall, 0.037s each | score:   1.115 |
           webapp2 w/Mako: | 250x, 0 failed |   9.228s overall, 0.037s each | score:   1.117 |
            bottle w/Mako: | 250x, 0 failed |   9.244s overall, 0.037s each | score:   1.119 |
             tipfy w/Mako: | 250x, 0 failed |   9.268s overall, 0.037s each | score:   1.122 |
           webapp w/Jinja: | 250x, 0 failed |   9.269s overall, 0.037s each | score:   1.122 |
            web.py w/Mako: | 250x, 0 failed |   9.307s overall, 0.037s each | score:   1.126 |
           bottle w/Jinja: | 250x, 0 failed |   9.328s overall, 0.037s each | score:   1.129 |
             flask w/Mako: | 250x, 0 failed |   9.412s overall, 0.038s each | score:   1.139 |
             WSGI w/Jinja: | 250x, 0 failed |   9.471s overall, 0.038s each | score:   1.146 |
             webob w/Mako: | 250x, 0 failed |   9.517s overall, 0.038s each | score:   1.152 |
            webapp w/Mako: | 250x, 0 failed |   9.589s overall, 0.038s each | score:   1.160 |
           web.py w/Jinja: | 250x, 0 failed |   9.763s overall, 0.039s each | score:   1.181 |
              WSGI w/Mako: | 250x, 0 failed |   9.970s overall, 0.040s each | score:   1.207 |

Medium Overhead:

       webapp w/Templator: | 250x, 0 failed |   8.464s overall, 0.034s each | score:   1.000 |
            tipfy w/Jinja: | 250x, 0 failed |   8.470s overall, 0.034s each | score:   1.001 |
        webob w/Templator: | 250x, 0 failed |   8.511s overall, 0.034s each | score:   1.006 |
      webapp2 w/Templator: | 250x, 0 failed |   8.582s overall, 0.034s each | score:   1.014 |
        flask w/Templator: | 250x, 0 failed |   8.608s overall, 0.034s each | score:   1.017 |
          web.py w/Tenjin: | 250x, 0 failed |   8.620s overall, 0.034s each | score:   1.018 |
       bottle w/Templator: | 250x, 0 failed |   8.657s overall, 0.035s each | score:   1.023 |
          webapp w/Tenjin: | 250x, 0 failed |   8.672s overall, 0.035s each | score:   1.025 |
       web.py w/Templator: | 250x, 0 failed |   8.673s overall, 0.035s each | score:   1.025 |
         webapp2 w/Tenjin: | 250x, 0 failed |   8.689s overall, 0.035s each | score:   1.027 |
           flask w/Tenjin: | 250x, 0 failed |   8.713s overall, 0.035s each | score:   1.029 |
      webob w/TenjinCache: | 250x, 0 failed |   8.722s overall, 0.035s each | score:   1.030 |
     webapp w/TenjinCache: | 250x, 0 failed |   8.735s overall, 0.035s each | score:   1.032 |
    webapp2 w/TenjinCache: | 250x, 0 failed |   8.761s overall, 0.035s each | score:   1.035 |
     web.py w/TenjinCache: | 250x, 0 failed |   8.771s overall, 0.035s each | score:   1.036 |
      tipfy w/TenjinCache: | 250x, 0 failed |   8.773s overall, 0.035s each | score:   1.036 |
      flask w/TenjinCache: | 250x, 0 failed |   8.829s overall, 0.035s each | score:   1.043 |
        tipfy w/Templator: | 250x, 0 failed |   8.852s overall, 0.035s each | score:   1.046 |
  webapp w/SimpleTemplate: | 250x, 0 failed |   8.895s overall, 0.036s each | score:   1.051 |
           webob w/Tenjin: | 250x, 0 failed |   8.912s overall, 0.036s each | score:   1.053 |
          bottle w/Tenjin: | 250x, 0 failed |   8.919s overall, 0.036s each | score:   1.054 |
           tipfy w/Tenjin: | 250x, 0 failed |   8.930s overall, 0.036s each | score:   1.055 |
         WSGI w/Templator: | 250x, 0 failed |   8.953s overall, 0.036s each | score:   1.058 |
 webapp2 w/SimpleTemplate: | 250x, 0 failed |   8.995s overall, 0.036s each | score:   1.063 |
   webob w/SimpleTemplate: | 250x, 0 failed |   8.998s overall, 0.036s each | score:   1.063 |
   tipfy w/SimpleTemplate: | 250x, 0 failed |   9.035s overall, 0.036s each | score:   1.067 |
       WSGI w/TenjinCache: | 250x, 0 failed |   9.081s overall, 0.036s each | score:   1.073 |
   flask w/SimpleTemplate: | 250x, 0 failed |   9.157s overall, 0.037s each | score:   1.082 |
     bottle w/TenjinCache: | 250x, 0 failed |   9.199s overall, 0.037s each | score:   1.087 |
             webob w/Mako: | 250x, 0 failed |   9.210s overall, 0.037s each | score:   1.088 |
            WSGI w/Tenjin: | 250x, 0 failed |   9.236s overall, 0.037s each | score:   1.091 |
            flask w/Jinja: | 250x, 0 failed |   9.308s overall, 0.037s each | score:   1.100 |
            web.py w/Mako: | 250x, 0 failed |   9.350s overall, 0.037s each | score:   1.105 |
          webapp w/Django: | 250x, 0 failed |   9.351s overall, 0.037s each | score:   1.105 |
           webapp w/Jinja: | 250x, 0 failed |   9.353s overall, 0.037s each | score:   1.105 |
           webob w/Django: | 250x, 0 failed |   9.363s overall, 0.037s each | score:   1.106 |
           bottle w/Jinja: | 250x, 0 failed |   9.383s overall, 0.038s each | score:   1.109 |
            bottle w/Mako: | 250x, 0 failed |   9.400s overall, 0.038s each | score:   1.111 |
  bottle w/SimpleTemplate: | 250x, 0 failed |   9.411s overall, 0.038s each | score:   1.112 |
           webapp2 w/Mako: | 250x, 0 failed |   9.427s overall, 0.038s each | score:   1.114 |
           tipfy w/Django: | 250x, 0 failed |   9.442s overall, 0.038s each | score:   1.116 |
          webapp2 w/Jinja: | 250x, 0 failed |   9.520s overall, 0.038s each | score:   1.125 |
           web.py w/Jinja: | 250x, 0 failed |   9.539s overall, 0.038s each | score:   1.127 |
          web.py w/Django: | 250x, 0 failed |   9.601s overall, 0.038s each | score:   1.134 |
         webapp2 w/Django: | 250x, 0 failed |   9.615s overall, 0.038s each | score:   1.136 |
            webob w/Jinja: | 250x, 0 failed |   9.652s overall, 0.039s each | score:   1.140 |
           flask w/Django: | 250x, 0 failed |   9.667s overall, 0.039s each | score:   1.142 |
             tipfy w/Mako: | 250x, 0 failed |   9.670s overall, 0.039s each | score:   1.142 |
            webapp w/Mako: | 250x, 0 failed |   9.673s overall, 0.039s each | score:   1.143 |
    WSGI w/SimpleTemplate: | 250x, 0 failed |   9.700s overall, 0.039s each | score:   1.146 |
             WSGI w/Jinja: | 250x, 0 failed |   9.742s overall, 0.039s each | score:   1.151 |
            WSGI w/Django: | 250x, 0 failed |   9.871s overall, 0.039s each | score:   1.166 |
          bottle w/Django: | 250x, 0 failed |   9.974s overall, 0.040s each | score:   1.178 |
              WSGI w/Mako: | 250x, 0 failed |   9.990s overall, 0.040s each | score:   1.180 |
             flask w/Mako: | 250x, 0 failed |  10.011s overall, 0.040s each | score:   1.183 |
          webapp2 w/Breve: | 250x, 0 failed |  10.519s overall, 0.042s each | score:   1.243 |
  web.py w/SimpleTemplate: | 250x, 0 failed |  10.618s overall, 0.042s each | score:   1.255 |
           web.py w/Breve: | 250x, 0 failed |  10.767s overall, 0.043s each | score:   1.272 |
           webapp w/Breve: | 250x, 0 failed |  10.786s overall, 0.043s each | score:   1.274 |
            tipfy w/Breve: | 250x, 0 failed |  10.832s overall, 0.043s each | score:   1.280 |
            flask w/Breve: | 250x, 0 failed |  10.840s overall, 0.043s each | score:   1.281 |
            webob w/Breve: | 250x, 0 failed |  10.999s overall, 0.044s each | score:   1.299 |
           bottle w/Breve: | 250x, 0 failed |  11.004s overall, 0.044s each | score:   1.300 |
             WSGI w/Breve: | 250x, 0 failed |  11.078s overall, 0.044s each | score:   1.309 |

High Overhead:

            tipfy w/Jinja: | 250x, 0 failed |   9.273s overall, 0.037s each | score:   1.000 |
           webapp w/Jinja: | 250x, 0 failed |   9.974s overall, 0.040s each | score:   1.076 |
           web.py w/Jinja: | 250x, 0 failed |  10.060s overall, 0.040s each | score:   1.085 |
          webapp2 w/Jinja: | 250x, 0 failed |  10.164s overall, 0.041s each | score:   1.096 |
           bottle w/Jinja: | 250x, 0 failed |  10.168s overall, 0.041s each | score:   1.097 |
             webob w/Mako: | 250x, 0 failed |  10.189s overall, 0.041s each | score:   1.099 |
            bottle w/Mako: | 250x, 0 failed |  10.230s overall, 0.041s each | score:   1.103 |
            webapp w/Mako: | 250x, 0 failed |  10.255s overall, 0.041s each | score:   1.106 |
           webapp2 w/Mako: | 250x, 0 failed |  10.257s overall, 0.041s each | score:   1.106 |
             tipfy w/Mako: | 250x, 0 failed |  10.260s overall, 0.041s each | score:   1.106 |
            web.py w/Mako: | 250x, 0 failed |  10.269s overall, 0.041s each | score:   1.107 |
             flask w/Mako: | 250x, 0 failed |  10.466s overall, 0.042s each | score:   1.129 |
        webob w/Templator: | 250x, 0 failed |  10.570s overall, 0.042s each | score:   1.140 |
       bottle w/Templator: | 250x, 0 failed |  10.608s overall, 0.042s each | score:   1.144 |
       web.py w/Templator: | 250x, 0 failed |  10.609s overall, 0.042s each | score:   1.144 |
      webapp2 w/Templator: | 250x, 0 failed |  10.650s overall, 0.043s each | score:   1.149 |
        tipfy w/Templator: | 250x, 0 failed |  10.783s overall, 0.043s each | score:   1.163 |
       webapp w/Templator: | 250x, 0 failed |  10.787s overall, 0.043s each | score:   1.163 |
        flask w/Templator: | 250x, 0 failed |  10.849s overall, 0.043s each | score:   1.170 |
           webob w/Tenjin: | 250x, 0 failed |  10.883s overall, 0.044s each | score:   1.174 |
           tipfy w/Tenjin: | 250x, 0 failed |  10.892s overall, 0.044s each | score:   1.175 |
           flask w/Tenjin: | 250x, 0 failed |  10.981s overall, 0.044s each | score:   1.184 |
          web.py w/Tenjin: | 250x, 0 failed |  10.995s overall, 0.044s each | score:   1.186 |
          bottle w/Tenjin: | 250x, 0 failed |  11.013s overall, 0.044s each | score:   1.188 |
     bottle w/TenjinCache: | 250x, 0 failed |  11.030s overall, 0.044s each | score:   1.190 |
      flask w/TenjinCache: | 250x, 0 failed |  11.055s overall, 0.044s each | score:   1.192 |
    webapp2 w/TenjinCache: | 250x, 0 failed |  11.082s overall, 0.044s each | score:   1.195 |
     web.py w/TenjinCache: | 250x, 0 failed |  11.116s overall, 0.044s each | score:   1.199 |
      tipfy w/TenjinCache: | 250x, 0 failed |  11.139s overall, 0.045s each | score:   1.201 |
            webob w/Jinja: | 250x, 0 failed |  11.146s overall, 0.045s each | score:   1.202 |
         webapp2 w/Tenjin: | 250x, 0 failed |  11.154s overall, 0.045s each | score:   1.203 |
          webapp w/Tenjin: | 250x, 0 failed |  11.179s overall, 0.045s each | score:   1.206 |
      webob w/TenjinCache: | 250x, 0 failed |  11.235s overall, 0.045s each | score:   1.212 |
     webapp w/TenjinCache: | 250x, 0 failed |  11.347s overall, 0.045s each | score:   1.224 |
            flask w/Jinja: | 250x, 0 failed |  14.164s overall, 0.057s each | score:   1.528 |
  web.py w/SimpleTemplate: | 250x, 0 failed |  14.352s overall, 0.057s each | score:   1.548 |
  webapp w/SimpleTemplate: | 250x, 0 failed |  14.380s overall, 0.058s each | score:   1.551 |
 webapp2 w/SimpleTemplate: | 250x, 0 failed |  14.458s overall, 0.058s each | score:   1.559 |
  bottle w/SimpleTemplate: | 250x, 0 failed |  14.485s overall, 0.058s each | score:   1.562 |
   tipfy w/SimpleTemplate: | 250x, 0 failed |  14.489s overall, 0.058s each | score:   1.563 |
   flask w/SimpleTemplate: | 250x, 0 failed |  14.969s overall, 0.060s each | score:   1.614 |
              WSGI w/Mako: | 250x, 0 failed |  15.353s overall, 0.061s each | score:   1.656 |
   webob w/SimpleTemplate: | 250x, 0 failed |  15.422s overall, 0.062s each | score:   1.663 |
            WSGI w/Tenjin: | 250x, 0 failed |  15.503s overall, 0.062s each | score:   1.672 |
         WSGI w/Templator: | 250x, 0 failed |  15.520s overall, 0.062s each | score:   1.674 |
       WSGI w/TenjinCache: | 250x, 0 failed |  15.595s overall, 0.062s each | score:   1.682 |
             WSGI w/Jinja: | 250x, 0 failed |  16.165s overall, 0.065s each | score:   1.743 |
          webapp w/Django: | 250x, 0 failed |  18.026s overall, 0.072s each | score:   1.944 |
          bottle w/Django: | 250x, 0 failed |  18.323s overall, 0.073s each | score:   1.976 |
         webapp2 w/Django: | 250x, 0 failed |  18.414s overall, 0.074s each | score:   1.986 |
           webob w/Django: | 250x, 0 failed |  18.492s overall, 0.074s each | score:   1.994 |
           tipfy w/Django: | 250x, 0 failed |  18.506s overall, 0.074s each | score:   1.996 |
          web.py w/Django: | 250x, 0 failed |  19.595s overall, 0.078s each | score:   2.113 |
           flask w/Django: | 250x, 0 failed |  19.596s overall, 0.078s each | score:   2.113 |
    WSGI w/SimpleTemplate: | 250x, 0 failed |  19.851s overall, 0.079s each | score:   2.141 |
            WSGI w/Django: | 250x, 0 failed |  23.853s overall, 0.095s each | score:   2.572 |
           webapp w/Breve: | 250x, 0 failed |  32.844s overall, 0.131s each | score:   3.542 |
            flask w/Breve: | 250x, 0 failed |  33.422s overall, 0.134s each | score:   3.604 |
            tipfy w/Breve: | 250x, 0 failed |  33.513s overall, 0.134s each | score:   3.614 |
            webob w/Breve: | 250x, 0 failed |  34.124s overall, 0.136s each | score:   3.680 |
           web.py w/Breve: | 250x, 0 failed |  34.308s overall, 0.137s each | score:   3.700 |
           bottle w/Breve: | 250x, 0 failed |  34.383s overall, 0.138s each | score:   3.708 |
          webapp2 w/Breve: | 250x, 0 failed |  34.689s overall, 0.139s each | score:   3.741 |
             WSGI w/Breve: | 250x, 0 failed |  37.234s overall, 0.149s each | score:   4.015 |

Very High Overhead:

            tipfy w/Jinja: | 250x, 0 failed |  17.782s overall, 0.071s each | score:   1.000 |
           web.py w/Jinja: | 250x, 0 failed |  18.418s overall, 0.074s each | score:   1.036 |
           webapp w/Jinja: | 250x, 0 failed |  18.489s overall, 0.074s each | score:   1.040 |
          webapp2 w/Jinja: | 250x, 0 failed |  18.902s overall, 0.076s each | score:   1.063 |
           bottle w/Jinja: | 250x, 0 failed |  18.999s overall, 0.076s each | score:   1.068 |
            webob w/Jinja: | 250x, 0 failed |  19.019s overall, 0.076s each | score:   1.070 |
             flask w/Mako: | 250x, 0 failed |  19.782s overall, 0.079s each | score:   1.112 |
             webob w/Mako: | 250x, 0 failed |  19.986s overall, 0.080s each | score:   1.124 |
           webapp2 w/Mako: | 250x, 0 failed |  20.043s overall, 0.080s each | score:   1.127 |
            webapp w/Mako: | 250x, 0 failed |  20.383s overall, 0.082s each | score:   1.146 |
             tipfy w/Mako: | 250x, 0 failed |  20.414s overall, 0.082s each | score:   1.148 |
            bottle w/Mako: | 250x, 0 failed |  20.428s overall, 0.082s each | score:   1.149 |
            web.py w/Mako: | 250x, 0 failed |  20.994s overall, 0.084s each | score:   1.181 |
        webob w/Templator: | 250x, 0 failed |  30.853s overall, 0.123s each | score:   1.735 |
       webapp w/Templator: | 250x, 0 failed |  31.001s overall, 0.124s each | score:   1.743 |
       web.py w/Templator: | 250x, 0 failed |  31.178s overall, 0.125s each | score:   1.753 |
       bottle w/Templator: | 250x, 0 failed |  31.207s overall, 0.125s each | score:   1.755 |
        flask w/Templator: | 250x, 0 failed |  31.234s overall, 0.125s each | score:   1.756 |
        tipfy w/Templator: | 250x, 0 failed |  31.607s overall, 0.126s each | score:   1.777 |
      webapp2 w/Templator: | 250x, 0 failed |  31.678s overall, 0.127s each | score:   1.781 |
           webob w/Tenjin: | 250x, 0 failed |  34.365s overall, 0.137s each | score:   1.933 |
           tipfy w/Tenjin: | 250x, 0 failed |  34.410s overall, 0.138s each | score:   1.935 |
           flask w/Tenjin: | 250x, 0 failed |  34.449s overall, 0.138s each | score:   1.937 |
     bottle w/TenjinCache: | 250x, 0 failed |  34.532s overall, 0.138s each | score:   1.942 |
      tipfy w/TenjinCache: | 250x, 0 failed |  34.551s overall, 0.138s each | score:   1.943 |
     webapp w/TenjinCache: | 250x, 0 failed |  34.552s overall, 0.138s each | score:   1.943 |
         webapp2 w/Tenjin: | 250x, 0 failed |  34.622s overall, 0.138s each | score:   1.947 |
      flask w/TenjinCache: | 250x, 0 failed |  34.669s overall, 0.139s each | score:   1.950 |
          webapp w/Tenjin: | 250x, 0 failed |  34.735s overall, 0.139s each | score:   1.953 |
    webapp2 w/TenjinCache: | 250x, 0 failed |  34.884s overall, 0.140s each | score:   1.962 |
     web.py w/TenjinCache: | 250x, 0 failed |  34.939s overall, 0.140s each | score:   1.965 |
          bottle w/Tenjin: | 250x, 0 failed |  34.964s overall, 0.140s each | score:   1.966 |
          web.py w/Tenjin: | 250x, 0 failed |  35.041s overall, 0.140s each | score:   1.971 |
      webob w/TenjinCache: | 250x, 0 failed |  35.501s overall, 0.142s each | score:   1.996 |
            flask w/Jinja: | 250x, 0 failed |  63.612s overall, 0.254s each | score:   3.577 |
  web.py w/SimpleTemplate: | 250x, 0 failed |  68.376s overall, 0.274s each | score:   3.845 |
  webapp w/SimpleTemplate: | 250x, 0 failed |  68.784s overall, 0.275s each | score:   3.868 |
   tipfy w/SimpleTemplate: | 250x, 0 failed |  69.553s overall, 0.278s each | score:   3.911 |
   flask w/SimpleTemplate: | 250x, 0 failed |  69.728s overall, 0.279s each | score:   3.921 |
 webapp2 w/SimpleTemplate: | 250x, 0 failed |  69.781s overall, 0.279s each | score:   3.924 |
   webob w/SimpleTemplate: | 250x, 0 failed |  70.051s overall, 0.280s each | score:   3.940 |
  bottle w/SimpleTemplate: | 250x, 0 failed |  70.541s overall, 0.282s each | score:   3.967 |
              WSGI w/Mako: | 250x, 0 failed |  73.786s overall, 0.295s each | score:   4.150 |
             WSGI w/Jinja: | 250x, 0 failed |  82.504s overall, 0.330s each | score:   4.640 |
            WSGI w/Tenjin: | 250x, 0 failed |  82.949s overall, 0.332s each | score:   4.665 |
       WSGI w/TenjinCache: | 250x, 0 failed |  83.182s overall, 0.333s each | score:   4.678 |
         WSGI w/Templator: | 250x, 0 failed |  84.371s overall, 0.337s each | score:   4.745 |
          webapp w/Django: | 250x, 0 failed | 103.542s overall, 0.414s each | score:   5.823 |
           webob w/Django: | 250x, 0 failed | 103.625s overall, 0.414s each | score:   5.828 |
           tipfy w/Django: | 250x, 0 failed | 103.984s overall, 0.416s each | score:   5.848 |
          bottle w/Django: | 250x, 0 failed | 104.424s overall, 0.418s each | score:   5.873 |
         webapp2 w/Django: | 250x, 0 failed | 105.751s overall, 0.423s each | score:   5.947 |
           flask w/Django: | 250x, 0 failed | 120.109s overall, 0.480s each | score:   6.755 |
          web.py w/Django: | 250x, 0 failed | 121.021s overall, 0.484s each | score:   6.806 |
    WSGI w/SimpleTemplate: | 250x, 0 failed | 123.487s overall, 0.494s each | score:   6.945 |
            WSGI w/Django: | 250x, 0 failed | 165.765s overall, 0.663s each | score:   9.322 |
           bottle w/Breve: | 250x, 0 failed | 282.629s overall, 1.131s each | score:  15.894 |
            webob w/Breve: | 250x, 0 failed | 283.554s overall, 1.134s each | score:  15.946 |
           web.py w/Breve: | 250x, 0 failed | 284.162s overall, 1.137s each | score:  15.981 |
            flask w/Breve: | 250x, 0 failed | 285.764s overall, 1.143s each | score:  16.071 |
            tipfy w/Breve: | 250x, 0 failed | 286.220s overall, 1.145s each | score:  16.096 |
           webapp w/Breve: | 250x, 0 failed | 286.600s overall, 1.146s each | score:  16.118 |
          webapp2 w/Breve: | 250x, 0 failed | 286.636s overall, 1.147s each | score:  16.120 |
             WSGI w/Breve: | 250x, 0 failed | 330.554s overall, 1.322s each | score:  18.589 |

Overall classification based on cumulative scores:

            tipfy w/Jinja: |  43.821 seconds overall | score:   4.005 |
           webapp w/Jinja: |  47.085 seconds overall | score:   4.342 |
          webapp2 w/Jinja: |  47.797 seconds overall | score:   4.399 |
           bottle w/Jinja: |  47.878 seconds overall | score:   4.402 |
           web.py w/Jinja: |  47.779 seconds overall | score:   4.429 |
             webob w/Mako: |  48.902 seconds overall | score:   4.463 |
           webapp2 w/Mako: |  48.956 seconds overall | score:   4.464 |
            bottle w/Mako: |  49.302 seconds overall | score:   4.481 |
             tipfy w/Mako: |  49.612 seconds overall | score:   4.519 |
            web.py w/Mako: |  49.920 seconds overall | score:   4.519 |
            webob w/Jinja: |  48.973 seconds overall | score:   4.520 |
            webapp w/Mako: |  49.900 seconds overall | score:   4.556 |
             flask w/Mako: |  49.671 seconds overall | score:   4.563 |
        webob w/Templator: |  58.318 seconds overall | score:   4.895 |
       webapp w/Templator: |  58.565 seconds overall | score:   4.913 |
       web.py w/Templator: |  58.744 seconds overall | score:   4.925 |
       bottle w/Templator: |  58.891 seconds overall | score:   4.941 |
        flask w/Templator: |  59.052 seconds overall | score:   4.955 |
        tipfy w/Templator: |  59.551 seconds overall | score:   4.992 |
      webapp2 w/Templator: |  59.617 seconds overall | score:   4.998 |
           webob w/Tenjin: |  62.458 seconds overall | score:   5.163 |
           flask w/Tenjin: |  62.560 seconds overall | score:   5.170 |
           tipfy w/Tenjin: |  62.590 seconds overall | score:   5.176 |
          web.py w/Tenjin: |  62.992 seconds overall | score:   5.184 |
    webapp2 w/TenjinCache: |  63.184 seconds overall | score:   5.216 |
          bottle w/Tenjin: |  63.262 seconds overall | score:   5.220 |
     webapp w/TenjinCache: |  63.085 seconds overall | score:   5.222 |
      tipfy w/TenjinCache: |  63.063 seconds overall | score:   5.222 |
          webapp w/Tenjin: |  63.167 seconds overall | score:   5.222 |
     web.py w/TenjinCache: |  63.287 seconds overall | score:   5.224 |
         webapp2 w/Tenjin: |  63.139 seconds overall | score:   5.226 |
      flask w/TenjinCache: |  63.369 seconds overall | score:   5.252 |
      webob w/TenjinCache: |  63.961 seconds overall | score:   5.268 |
     bottle w/TenjinCache: |  63.624 seconds overall | score:   5.291 |
            flask w/Jinja: |  95.555 seconds overall | score:   7.230 |
  webapp w/SimpleTemplate: | 100.370 seconds overall | score:   7.476 |
   tipfy w/SimpleTemplate: | 101.480 seconds overall | score:   7.558 |
 webapp2 w/SimpleTemplate: | 101.629 seconds overall | score:   7.562 |
   flask w/SimpleTemplate: | 102.289 seconds overall | score:   7.638 |
  bottle w/SimpleTemplate: | 102.844 seconds overall | score:   7.658 |
  web.py w/SimpleTemplate: | 101.858 seconds overall | score:   7.678 |
   webob w/SimpleTemplate: | 102.836 seconds overall | score:   7.678 |
              WSGI w/Mako: | 109.099 seconds overall | score:   8.192 |
            WSGI w/Tenjin: | 115.977 seconds overall | score:   8.431 |
         WSGI w/Templator: | 117.107 seconds overall | score:   8.476 |
       WSGI w/TenjinCache: | 116.834 seconds overall | score:   8.519 |
             WSGI w/Jinja: | 117.882 seconds overall | score:   8.680 |
          webapp w/Django: | 139.356 seconds overall | score:   9.893 |
           webob w/Django: | 140.017 seconds overall | score:   9.961 |
           tipfy w/Django: | 140.722 seconds overall | score:  10.023 |
          bottle w/Django: | 141.361 seconds overall | score:  10.073 |
         webapp2 w/Django: | 142.275 seconds overall | score:  10.097 |
           flask w/Django: | 157.943 seconds overall | score:  11.047 |
          web.py w/Django: | 159.121 seconds overall | score:  11.131 |
    WSGI w/SimpleTemplate: | 161.372 seconds overall | score:  11.240 |
            WSGI w/Django: | 208.025 seconds overall | score:  14.094 |
           bottle w/Breve: | 336.695 seconds overall | score:  21.953 |
           webapp w/Breve: | 338.780 seconds overall | score:  21.969 |
            webob w/Breve: | 337.294 seconds overall | score:  21.969 |
           web.py w/Breve: | 337.934 seconds overall | score:  22.005 |
            tipfy w/Breve: | 339.195 seconds overall | score:  22.035 |
            flask w/Breve: | 339.243 seconds overall | score:  22.071 |
          webapp2 w/Breve: | 340.441 seconds overall | score:  22.144 |
             WSGI w/Breve: | 387.868 seconds overall | score:  25.003 |

Templating Engines classification:

           Mako: |  455.363s overall | score:   1.000 |
          Jinja: |  496.770s overall | score:   1.091 |
      Templator: |  529.844s overall | score:   1.164 |
         Tenjin: |  556.145s overall | score:   1.221 |
    TenjinCache: |  560.408s overall | score:   1.231 |
 SimpleTemplate: |  874.678s overall | score:   1.921 |
         Django: | 1228.821s overall | score:   2.699 |
          Breve: | 2757.450s overall | score:   6.055 |

Web Application Frameworks classification:

          tipfy: |  860.035s overall | score:   1.000 |
         webapp: |  860.308s overall | score:   1.000 |
          webob: |  862.759s overall | score:   1.003 |
         Bottle: |  863.857s overall | score:   1.004 |
        webapp2: |  867.038s overall | score:   1.008 |
         wep.py: |  881.636s overall | score:   1.025 |
          Flask: |  929.682s overall | score:   1.081 |
           WSGI: | 1334.164s overall | score:   1.551 |

Batteries Included Frameworks comparison:

            tipfy w/Jinja: |  43.821s overall | score:   1.000 |
       web.py w/Templator: |  58.744s overall | score:   1.341 |
            flask w/Jinja: |  95.555s overall | score:   2.181 |
  bottle w/SimpleTemplate: | 102.844s overall | score:   2.347 |
          webapp w/Django: | 139.356s overall | score:   3.180 |

Straight away, you may notice that the overall classification order is not entirely coherent with the previous benchmarks run, but the Framework and Templating Engine comparisons stays in line with the previous ordering. I would really like to know why WSGI scores so badly… the problem must be in my implementation, because its technology is the foundation for all the others. What did I do to make it that bad? What about Tenjin with cache? This one is explicable… the benchmark is built to engender the more possible unique queries, so that is why the caching suffer and becomes more of a wasted cycle than a clever speed trick.

blog comments powered by Disqus

}

Older Posts... Blog powered by Jekyll.
Built using Liquid, RedCloth, Pygments and Blueprint.

Copyright © 2008-2010 Louis-Philippe Perron