BitsBlog has been having problems with the XML-Sitemap plugin, for the last couple of months. This is a plugin that creates a comlete site listing for Google and other search engines, so they know what’s going on here. saves them the trouble of crawling the place and saves me lots of database time and net bandwidth.
Now, when we post, the system updates the search engine XML’s, and pings all the engines to let them know we’ve got new stuff, which is why they’ve been listing our stuff so quickly.
Problem was, that it’s been taking upwads of 90 seconds to run the update and create the proper files to feed the search engines, despite being given a huge 16m of space in memory on the server, and being limited to 8000 posts. It was taking upwards of 5 minutes before I tuned the thing.
Version 3.0.1 of XML sitemap is available for WordPress, as of this morning… and it’s doing the same job in about 18 seconds. I’ve bumped the posts value back to 10,000 posts, which is the limit for some search engines… and it went to 20 seconds.
I’d say offhand, that they fixed something.