There's a patchset for this feature being discussed on the mailing list:

http://ffmpeg.org/pipermail/ffmpeg-devel/2016-February/18953...

Yes, much of GitHub's core infrastructure is written in C.

That's awesome. You don't find many people that enjoy writing in C. I assumed it was Ruby.

Ruby MRI and YARV are both written in C, so to be a true Ruby hacker you need some C chops. :-)

Interesting. I've heard that thrown around before, but I've never heard someone say the same thing about Python, even though the main Python implementation is C.

Really? Lots of the most famous Python modules are partially implemented in C(++) for performance reasons too, it's not exclusive to Ruby.

For example, Numpy is 56% C. https://github.com/numpy/numpy

From my own experience with Ruby and MRI, there's quite a strong culture of "transparency of code". It's not unusual if you're really down into some "interesting" problem to delve first into your gems/frameworks, and eventually even into MRI itself. Being a PL polyglot pays off in spades in these cases, since you get a solid understanding of the underpinnings of everything.

Note that there's very rarely any need to go there, but the fact that it's both possible and culturally encouraged makes a huge difference for those of us who've needed it.

From the Python docs:

https://docs.python.org/3/

Extending and Embedding tutorial for C/C++ programmers

Python/C API reference for C/C++ programmers

... right between "Distributing Python Modules" and "FAQs"

I have a talk at a Ruby conference last week and even put a small amount of assembly on the screen... While many Rubyists aren't expert C programmers, they know enough to look at Ruby's source every once in a while to figure out what's going on.

test-queue supports minitest too, and follows the same basic design outlined in this article: a central queue sorted by slowest tests first, with test runners forked off either locally or on other machines to consume off the queue.

We use test-queue in a dozen different projects at GitHub and most CI runs finish in ~30s. The largest suite is for the github/github rails app which runs 30 minutes of tests in 90s.

Thanks for the awesome work you have done on test-queue. We have really appreciated it at SchoolKeep.

The https://github.com/simeonwillbanks/busted gem provides some nice helpers around RubyVM.stat for finding cache invalidations. It can also take advantage of the dtrace/systemtap probes that shipped with ruby 2.1 (http://tmm1.net/ruby21-method-cache) which can be used as an alternative to the ftrace technique described in the blog post.

Before the release of 2.1.0, funny_falcon, samsaffron and I developed a patch that removes the global method cache in favor of a local method cache inside each class. (This was originally proposed for upstream inclusion in https://bugs.ruby-lang.org/issues/9262).

The patch has been running in production on GitHub's servers for over a year, and provides the benefits of a high cache-hit rate without the need for manual tuning. I recently ported this patch to ruby 2.2 as well and it is available in this squashed commit: https://github.com/github/ruby/commit/bd002fc9fc3c7236395df2...

I originally read about this patch here, http://samsaffron.com/archive/2014/04/08/ruby-2-1-garbage-co....

There hasn't been any activity on the upstream proposal for almost a year. Has the core team rejected the idea or is there still hope it will make it into a future version?

Copying `/usr/bin/gdb` and `/usr/libexec/gdb/gdb-i386-apple-darwin` from an OSX Lion installation is one way to get a functional gdb on Mountain Lion or Yosemite.

We implement preload_all, which loops over app/{models,controllers}/ and requires everything. This method is generally called from config.ru, and happens in the unicorn master before it forks off workers.

I recently upstreamed a warmup method for Rack::Builder you can also use for this purpose: https://github.com/rack/rack/pull/617

Thanks for that link (and for submitting it!). I've done something similar in deploy scripts, to warm up a fresh instance before bringing it into rotation - even better having it able to be baked into rack like that.

That's interesting. Thanks for sharing. I'll have to check this out.

Unicorn pre-forks workers upfront. New processes are not created per request.

posix-spawn is also only useful for fork+exec, where the new process starts executing a new binary. This is not the case in unicorn or resque, where the child processes actually need to be copies of the parent.

We're currently using posix-spawn in albino and grit, which execute external commands like `pygmentize` and `git`

Yes, you're right, of course. Sorry.

This is a great article. I came across it while hacking on posix-spawn and it was quite useful.

The reason fork is so slow on Linux is because the default page size is 4k. This means for a 300MB process, 76800 page table entries have to be copied during the fork.

There are ways to enable HugeTBL on linux to increase the page size, which can improve fork performance as well. See http://stackoverflow.com/questions/2731531/faster-forking-of..., http://linuxgazette.net/155/krishnakumar.html and http://sourceforge.net/projects/libhugetlbfs/

Does anyone know how fork is implemented on BSD/OSX and why it doesn't exhibit the same characteristics?

BSD / OS X has native support for superpages.

From version 2.6.38, Linux will get native huge page / superpage support too. http://lwn.net/Articles/423584/

In FreeBSD, it is extent-based, so a 300MB process might actually only have a couple dozen maps - heap, stack, program binary, and whatever dynamically loaded libraries.

I'd be shocked if it wasn't the same in linux.