Simple JSON and Mongo speed test between Perl, Ruby and Python

Hi,

I’m about to try a Web Framework in another programming language than Perl. I have a speed test for Perl with JSON and MongoDB. I give a try on Python and Ruby.

My speed test do this :

  • Read from pipeline (pbzip2)
  • JSON decoding
  • Batch insert into MongoDB

My original file contain 45 million of rows, each of them is a JSON string.

The languages :

  • Perl 5.18 with JSON::XS
  • Ruby 2.1.1 with OJ (Optimized JSON)
  • Python 3.4 with UJSON (Ultra Fast JSON)

The MongoDB engines :

  • TokuMX (MongoDB 2.4)
  • MongoDB 2.6

The speed test resume :

Language Read from pipeline Decode JSON Insert in TokuMX Batch Insert in MongoDB 2.6
Perl 0m6.861s 0m8.104s 0m5.602s 0m4.916s
Ruby 0m6.669s 0m12.588s 0m9.552s 0m8.291s
Python 0m6.858s 0m8.160s 0m8.382s 0m8.097s

Speed test 1 : Read 1M of rows from pipeline reading

Perl Code :

Ruby Code :

Python Code :

Speed test 2 : Decoding JSON 500 000 rows

Perl Code :

Ruby Code :

Python Code :

Speed test 3 : Insert 100 000 rows in TokuMX

I run the code twice (for the drop), and report the time.

Perl Code :

Ruby Code :

Python Code :

Perl win on TokuMX here, now I try the batch insert with MongoDB 2.6 (real batch insert mode).

Speed test 4 : Batch Insert 100 000 rows in MongoDB 2.6

Perl Code :

Ruby Code :

Python Code :

Again, Perl is even faster with real bulk insert ! The driver automatically choose the best mode. Ruby has a unordered bulk insert mode in 2.6 (not the same code), and the result is a speed as good as Python.

Celogeek

Short URL: http://sck.pm/p0

2 thoughts on “Simple JSON and Mongo speed test between Perl, Ruby and Python”

    1. Yeah I see that. I will test also on MongoDB 2.6 but we use TokuMX with 2.4 so multi insert is also fast enough. I will rework on the article, with more version test, and better python code also :) one for 2.4, another for 2.6.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">