I obviously have, but I have a lot of respect for the MRI folks and usually don't publish those results. You're free to try it out yourself. In general, JRuby should be significantly faster than MRI 1.9.2 or 1.9.3 for running Ruby code, and usually faster inside the core classes (String, Array, etc). If it's not, I consider it a bug, and encourage bug reports to that effect.
Until recently I dismissed JRuby, but then I needed to run some stress testing code against an Oracle DB. With MRI Ruby and the OCI8 library, the Ruby overhead stopped me being able to push the database hard enough. I switched to JRuby and JDBC and the results were amazing, so I am converted. The only niggle I have is the startup overhead, which I know is a JVM thing. Not a problem for my benchmarking, but a little bit of an annoyance for a command line ruby tool I use.
As an aside, I did run across a bug, not performance related, but a case around popen where JRuby doesn't behave the same as MRI - I raised a ticket (JRUBY-6225) but it doesn't seem to be getting any traction. I understand there are probably plenty of tickets to be prioritized, but this one is a bit of a show stopper for me. If anyone from the JRuby team is see this and has time to have a look at it I would be grateful.
There seem to be a few bug reports all around the same phenomenon. See also JRUBY-6162. I started digging into it a bit last week, but I was in the middle of a release and didn't have time to get very far. My working hypothesis is that the external process is getting killed off by JRuby before it has time to finish executing. The (hacky) workaround is to make your code wait a bit so the process has time to complete. Faster JRuby execution might only make the problem worse :-), but I'm looking forward to playing with it!
6162 does sound very like my issue. I must try sticking a sleep inside the block passed to popen and see if that gets past my problem. It's not a solution as such, as the process I am calling could take anywhere from a second to many minutes to execute, but it will be interesting to try. If you manage to solve these bugs it will be a great help to me, so good luck!
Recently we've been working more on JRuby master, on new features and performance. With JRuby 1.6.6 coming soon and 1.7 starting to stabilize, we'll be circling back to these bugs.
I'd love to help, but my Java skills are almost none existent! If I thought I could make any headway into that bug I'd jump in - maybe over the holidays I will try and have a look.
Why would respect for the MRI devs stop you from publishing benchmarks? It's not an insult to say that something else is faster. I'd imagine the competition might drive them to improve if at all possible.
I feel like the MRI devs get beat on a bit too much, and they're doing the best job possible with very few paid developers. I'm more interested in showing how JRuby is improving over time than rubbing salt in the wounds.
Speaking of very few paid MRI developers, does anyone know if the Ruby project takes donations to hire full-time paid developers? If yes, could you please point us to the donations page?
After matz announced he was hired by Heroku I spoke with him on how he plans to hire developers. He doesn't want one company to have all the MRI developers, so the best way to create a new paid MRI committer is to hire one specifically to do so.
Really glad that @headius is getting some recognition for his dogged work in this space. I know he's been up against a sometimes quite hostile Ruby crowd but has remained a gentleman throughout.
I'm not really familiar with the ruby development process and community and haven't used the language much, but I could imagine the MRI developers spend considerable time advancing the actual design of the language (syntax, standard APIs, etc.) whereas the other Ruby implementations "only" need to implement that design and thus can maybe focus more on performance. If MRI were to die, that design work would still need to be done in order to push the language forward. As such it could be seen as a symbiotic relationship, not competition.
Yes, I'd say that's at least partially true. The innovation on JRuby's side is in new ways to integrate with the JVM, rather than in API or language design. MRI folks need to innovate both in how they design the language and APIs and in how they implement them.
On my computer
Linux 3.0.0-15-generic #24-Ubuntu SMP Mon Dec 12 15:23:55 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux RAM=6G
"Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz"
A simple comparison using binary-tree (from computer language benchmark) between ruby1.9.1 and jruby show that on my machine jruby time is half of 1.9.1
~/lang/ruby/jruby-1.6.5/bin$ time ./jruby --server -J-Xmx2000m algo.rb 18 gives
real 0m32.447s
user 0m38.478s
sys 0m0.904s
time ruby1.9.1 algo.rb 18
real 1m11.388s
user 1m11.192s
sys 0m0.148s
./jruby --server -J-Xmx2000m -v
jruby 1.6.5 (ruby-1.8.7-p330) (2011-10-25 9dcd388) (OpenJDK 64-Bit Server VM 1.7.0_147-icedtea) [linux-amd64-java]
the algo.rb file is the version contributed by
# contributed by Jesse Millikan
# Modified by Wesley Moxam and Michael Klaus
this is n=18, for n=16,17 ... results are similar. Half the computing time.
a) There will be problems in the Computer Language Benchmark. Like with the excellent LuaJit, they only allow one version of a language, so a war of implementations to appear would not be surprising.
b) Now that Python is in a difficult transition to Python 3, a strong ruby is a game changing point. Antonio Cangiano
`New Relic Holy Shmoly, Ruby 1.9 smokes Python away!` could be Jruby smokes Python away :) (if there is a two-fold increase in speed I expect A. Cangiano article rewrited)
Duby has become Mirah, and though I have not personally had a lot of time to work on it, it has continued slowly forward. It is basically just Ruby syntax for writing Java, though, so it performs identically to Java.
I don't know the status of Groovy performance in general. I do know that on small numeric benchmarks, JRuby + invokedynamic beats fully-dynamic Groovy, but you can "cheat" and static-type some numeric logic in Groovy, which puts it out in front again.
b) Also Jython is virtually dormant, at some older version of Python, while both JRuby and IronRuby are actively worked on.
c) Groovy hasn't added any support for invokedynamic or even started on it. A month ago they announced they're removing their meta-object protocol rework from the upcoming version 2.0.
SpringSource seem to be repositioning Groovy as a staticly-typed language, this year adding primitive types like in Java. Before everything was an object, and primitives were regarded as leaky abstractions. 2 months ago they employed someone to write a static type checker called "Grumpy" for Groovy, and eventually type inference and optimization, but the work's just started so it might take quite some time depending on how good their new hire is. This year they also tried to rewrite the Antlr 2.7 based parser in Antlr 3.2, using a Google Summer of Code worker, but they didn't get very far. Perhaps the static type checking for Groovy will be more successful, perhaps not.
I imagine this repositioning was instigated by the Grails team wanting a faster language, with dynamicity an optional extra to be used only when required.
I find it a bit of a dodge when a dynamic language has to go to static types for performance. That's not to say I haven't wanted to have that dodge available to JRuby users, but being unable (or unwilling) to unilaterally add optional static types to Ruby, we've been forced to find more creative options. We'll probably never approach Java's raw primitive math performance, but I think we can easily match Java's speed (and in some cases, already do) at manipulating objects.
It also appears that the push toward more static typing in Groovy may come at the cost of dynamicity...or at least incompatible changes in Groovy's dynamic features. Perhaps it's the best move for them, though; they've always pushed Groovy's ability to run (and enhance) plain old Java code, but performance was considerably worse than Java.
It's unfortunate that other JVM dynamic languages did not start exploring invokedynamic sooner. It has been a complete game-changer for JRuby.
Oh, and regarding JRuby versus Python 3...yes, if Ruby 1.9 smokes Python, and JRuby is faster than Ruby 1.9, then JRuby should smoke Python even more. I have not done the comparisons myself, though.
I know some of the implementations there use native libraries that are built into python but only available via RubyGems for Ruby, like numpy. That makes it less of a language shootout (at times) and more of a "who ships the best C libs" shootout.
I believe MRI is generally faster than Python now when they're doing roughly equivalent work in Ruby and Python.
Except that Python has scipy,numpy and cython in its camp. I have spent the last six months of my life making Python code run faster. You can write slow code very quickly in Python. If your users complain about it being too slow you can implement some critical sections in Cython. (again with very minor changes). That being said there is a overhead to calling functions in Python which I would like addressed. (Dictionary lookups were three times as fast as function calls when I measured them).
