Refactoring WordPress to improve memory performance [closed]

by

I had a close look at WordPress memory consumption. On my site, it seems that for each page hit 20MB of RAM gets allocated, just to prepare comfy environment for all the plugins to run in. I plotted it thusly:

There is no single spot to optimize, no single bad guy who eats most of the memory. Consumption is all spread over many many many php modules.

How can we make WordPress initialize its environment in memory only once, and then reuse it many times for each hit? I do not want to have slow PHP eat 20 MB at each user click – even on a server with lots of memory, it takes seconds to get all that work done. You would basically need read-only chunks of memory that can be reused.

Also… why 20MB? Can anyone provide insights into this?

Edit: Here is WinCacheGrind output on the WordPress running on my development machine (vastly faster than the shared hosting). As you can see, it takes over a second of crunching just to produce the main page’s HTML. Slow that down by shared hosting and you have a recipe for trouble. I picked the method that takes most of the time. How would you go about optimizing this?

Edit: Here are query statistics from this fantastic functions.php profiling tool.

Load: 12 queries - 532ms - 19.1MB - 43 cache hits / 53
Query: 15 queries - 563ms - 19.0MB - 72 cache hits / 86
Display: 21 queries - 705ms - 19.2MB - 234 cache hits / 257

Edit: Do you want to see something guaranteed to freak you out? Insert these lines at the end of index.php:


echo "<pre>\n";
print_r(get_defined_vars());
echo "</pre>\n";

I tried to count how many times is the current post’s body stored in the memory. I counted 20 instances. Then I realized that PHP has reference counting, so the amount of copies reduced to just three: two seem to be in WP_Query, one in the object cache. I am investigating further.

This is why I think WordPress is in need of refactoring targeting the memory issues. You can no longer blame its memory consumption on sheer complexity of what it does. It simply does a bunch of things wrong.

Edit: After a day of trying to figure this out, here are my findings:

1) 88% of all memory comes from require or include or include_once type of calls:

2) The php file includes happen mostly during the first part of serving a request (not surprisingly), which is also where all the memory is eaten up:

3) It is quite interesting to plot all the functions that are being executed during making a request. There are over 12000 calls total. I jittered them to make it more visible (the Level axis is basically depth of stack):

4) The only way forward that I can think of is minimizing the amount of .php files included. If I split the functions per file they came from, you can see that many files are hit once or twice at most. We need a way how to skip those when they are not needed. For instance my remote database backup plugin gets loaded and registered, just to be never used at all. Here is the above plot split by file name:

I am offering a bounty worth all my reputation 🙂 for refactorings that would lead to cutting my blogs memory footprint by 30% or more.

Edit: I installed WP 3.1, here is comparison with the old version.

Blue is WP 3.1, red is 3.0.4. The new WP is faster, but also eats more memory.

Here is a list by include file.

This let me realize just how much memory is eaten by “All In One SEO pack” – one avenue would be to use only a fraction of the plugin’s functionality to get what I want. Also, my own plugins seem to be pretty bad.

I would like to try conditional loading on e.g. comment.php (I disallow comments on my blog) and several others. I deleted all the deprecated code. I trimmed down kses.php to only load its global tables on demand. I simplified l10n (I do no localization), making its functions return the strings right away, without lookups. I am still far from the 30% mark I arbitrarily set up.

Edit: I downloaded and enabled APC with default settings (32MB of opcode cache). Here is the comparison:

You can see that the code loading accelerated massively, and the code also takes less space in memory (probably because we deal just with opcodes, not the original source). The memory consumption is however still quite high.

8

Not worth the trouble. WordPress doesn’t eat a lot of memory just-because. It eats a lot of memory because it runs a lot of functionality under the hood.

It is far more easier and efficient to cache results (page generated) with static cache plugin and serve that. That way most visitor will not even hit WP itself.

Leave a Comment