The WordPress Database vs. The Internet

There’s a story on Digg about WordPress. Completely unrelated to the contents of the article, one commenter had this to say:

too bad no one ever sees wordpress themes because the sites go down after five diggs

Dong. That’s the sound of the “I think so too-bell ringing. It’s always the database server that chokes with an error message that’s as welcome as Bush trying to be funny. The author of WordPress claims things are dandy if you install WP Cache caching plugin.

I’ve met the chap, he’s bright. He might be right and maybe if WP-Cache worked for me I wouldn’t be writing this. Maybe if everyone used WP Cache, funny and sad comments like the afore-mentioned digg related one would be as plentiful as good Uwe Boll movies. That is, not.

Maybe if a caching mechanism was built-in to WordPress, I wouldn’t be citing bad metaphors.

Your experience? Butterflies and milk and honey?

12 thoughts on “The WordPress Database vs. The Internet”

  1. Brendan says:

    WP-Cache doesn’t really achieve as much as people would like to think – given that db hits still occur, despite content caching..

    Really it’s more a case of streamlining the db and improving the engine. I’m sorry, but if other blog systems can somehow manage to survive a digg or slashdot moment, wordpress should be able to as well.

    It frequently doesn’t because it doesn’t scale load well, at least without an insanely powerful sql server farm behind it.

    I’ve actually started culling plugins and looking to see what I can streamline, because caching isn’t the answer – they tried it (early 2.0 builds had it – indeed it’s still there afaik, simply disabled) and most things broke as a result.

  2. Brendan says:

    Having said that, I have actually been playing with the built in cache a wee bit to see if it’s a little better than it once was..

  3. Joen says:

    I?m sorry, but if other blog systems can somehow manage to survive a digg or slashdot moment, wordpress should be able to as well.

    I totally agree. In my experience it doesn’t even require a slashdot effect to bring my server to its knees… It seems to me that optimization should be a priority for the next release.

    Having said that, I have actually been playing with the built in cache a wee bit to see if it?s a little better than it once was..

    Please let me know how it goes…

  4. Kevin Cannon says:

    The thing I’ve never really understood with blogs is that they’re very static.

    Wouldn’t it be better to have WordPress generate HTML pages based on Templates, rather than pull the content from it each time there’s a hit.

  5. Joen says:

    Wouldn?t it be better to have WordPress generate HTML pages based on Templates, rather than pull the content from it each time there?s a hit.

    Well, I suppose it would solve some of the problems, but it would introduce others.

    Essentially that’s what Movable Type does, and that’s also one of the main reasons for my switching to WP from it.

    I’m thinking it should be possible to optimize the DB usage so that the DB isn’t hit every other second. It’s gotta be possible to have dynamic pages and low db usage.

    Also, to me it seems totally insane that there’s no upper limit to how much the DB can be hit… essentially traffic just hammers the DB server until it’s crashed and has to be rebooted. What about as simple “max connections” or “the DB is being hammered, please try in a second”…

  6. Nathan says:

    It would seem that a combined approach (creating html files and using database storage) might be needed.

    The most agreesive caching could create html files and server them until they are out of date, which would create a new html file at that time (similar to how many rails application use caching).

    So a comment being submitted would regenerate the html. This would prevent dynamic content that is frequently updated (withit 5 mintues or so) from being feasible, but it might help sites with large load.

    And, finally, I would like to see it run off the db until high load occurs, then generate html files and buckle down until it stops. That would be ideal.

  7. Henrik Lied says:

    The only thing really working is memcached. The problem is that most shared hosting environments don’t allow you to install and run the servers. If more hosts would allow this, Digg wouldn’t be a problem anymore.

  8. Joen says:

    While I am desperate for better WordPress caching, I don’t think static files in the traditional form is the way to go. Of course WP cache actually does write static files. The problem seems to be that even when WP cache is doing it’s thing, it’s either not doing it well enough, or it does work but somehow the DB crashes anyway.

    Memcached looks interesting.

    Even so, I’m thinking why can’t some proper caching mechanism be written into wordpress … Wikipedia does it, and there aren’t any static files.

  9. Henrik Lied says:

    @Joen: Wikipedia is one of the largest memcached installs around. 🙂

  10. Joen says:

    @Joen: Wikipedia is one of the largest memcached installs around. 🙂

    Hah hah, cool, then memcached has my vote!

  11. Henrik Lied says:

    @Joen: It should! 🙂

    Facebook is probably the largest memcached install, with around 200 servers with 16GB of memory each (source: http://lists.danga.com/pipermail/memcached/2007-May/004098.html)

  12. adam says:

    It would seem that a combined approach (creating html files and using database storage) might be needed.

    The most agressive caching could create html files and server them until they are out of date, which would create a new html file at that time (similar to how many rails application use caching).

    So a comment being submitted would regenerate the html. This would prevent dynamic content that is frequently updated (within 5 minutes or so) from being feasible, but it might help sites with large load.

    And, finally, I would like to see it run off the db until high load occurs, then generate html files and buckle down until it stops. That would be ideal.

    supposedly that’s what the MT-burn plugin for movable type does, which also eliminates the need to republish your site every hip stitch (since the pages are generated on the fly, by .htaccess + a 404 handler).

Comments are closed.