Smoked salmon is delicious! - Journal of Omnifarious
Aug. 31st, 2005
11:02 pm - Smoked salmon is delicious!
And I'm very pleased with myself. I've been trying to get Mercurial to work at work for a couple of different reasons. But it had one major problem when I tried to clone a repository remotely. The server process was chewing huge quantities of CPU.
This mystified me a bit. But after thinking about the results of doing an strace on it (lots and lots of calls to mmap2(2) and munmap(2) to mmap
MAP_ANONYMOUS pages), I realized that Python must be doing lots of operations with fairly large strings that the memory allocator thought were large enough to be using mmap and munmap to get the memory for instead of the standard heap.
I profiled it to figure out exactly where the time was being spent and discovered that my theory was correct. I replaced the code with some more intelligently written code that used cStringIO and the not-at-all-well-documented buffer type. The speed improved immensely, and went from being CPU bound to using less than 10% of the CPU.
I submitted the patch, and now I bet I'll end up with my name in Mercurial. :-)