"Wikitravel has a speed and convenience the books' publishers can only envy." Time Europe

Tech:Site slowdown

From Wikitravel Shared
Jump to: navigation, search

What happens[edit]

I can't quickly find the existing conversations about it, I think there are a couple... but Wikitravel is often running very slow, STILL, after much talk about it from several users. It's become less of a complaint for me and more of a showstopper. Often when I find that I have 20 or 30 minutes to contribute I only accomplish half or a third or what I could if the site was running comparably to Wikipedia or any other site. It's often the reason that I move on to doing other things on any given day, when I get tired of waiting for pages to load. It keeps me from wanting to patrol edits, where I need to open many window at once and not wait 10 minutes for them to all load. In all honesty, I'm considering halting my contributing even further until the site is seriously sped up. – cacahuate talk

It appears to me that static pages and those with short edit histories are not too bad, but it's the dynamically generated pages (esp. diffs and edits) of big/long articles (VFD, Main Page, Japan, Singapore etc) that are excruciatingly slow to load. DB problems? Jpatokal 00:38, 17 February 2008 (EST)
Special:Watchlist is the main page I start Wikitravel with, and it takes minutes to load every time. --DenisYurkin 03:13, 17 February 2008 (EST)

When it happens[edit]

It goes up and down, but for me at least, it happens fairly regularly, almost every time I'm on the site. – cacahuate talk

What should happen[edit]

How to fix it[edit]

I recall Evan or someone mentioning at some point that something was added to the site by IB to track page views or something like to that effect, and that that may have to do with the problem... if it is can we remove it? – cacahuate talk 19:57, 17 February 2008 (EST)

Additional comments[edit]

This problem is so severe on :en for me that I've basically given up working on that site, at least until the problem is resolved. --Peter Talk 22:09, 16 February 2008 (EST)

Sign below, please[edit]

  • cacahuate talk 13:18, 16 February 2008 (EST)
  • Me too. This has gone beyond annoying into outright painful, even when editing in the middle of the night US time. Jpatokal 00:36, 17 February 2008 (EST)
  • Watchlist and diffs, most of the time. --DenisYurkin 03:13, 17 February 2008 (EST)
  • Do something. And tell us when you have! Riggwelter 16:52, 18 February 2008 (EST)
  • It's ridiculous. Gorilla Jones 08:20, 21 February 2008 (EST)
  • Definitely will limit my enthusiasm for editing.--Padraic 11:10, 21 February 2008 (EST)
  • Agreed on all counts. -- Bill-on-the-Hill 11:50, 21 February 2008 (EST)
  • As usual, I do not get it. I have emailed the general Manager at IB and left him a voice message. If you want to call the number is 800-692-2200 2old 11:54, 21 February 2008 (EST)
  • It's bad. Not good for our contributors. Not good for anybody. --OldPine 11:59, 21 February 2008 (EST)
  • Heres a new one, I added to Tech:Blocking all anonymous log ins on en:? . When I clicked save, it said there is no text on this page and all text has dissapeared.It was oldpines report, now gone, gone, gone. 2old 12:04, 21 February 2008 (EST)
It's there. I guess the page title I used was a bad one--probably due to the ?. It's at Tech:Blocking all anonymous log ins on en now. Just noticed that when I placed that bug report that it logged the blocked IP address rather than mine. -- OldPine 12:27, 21 February 2008 (EST)
  • Absolutely horrible, deserving of immediate action. There is no excuse for any site to run so consistently and terribly slow. I think we are losing potential readers and contributors right and left and we're about to start losing long-time, dedicated contributors as well. Please for god's sake fix this problem. Texugo 19:06, 21 February 2008 (EST)


Second server[edit]

Sorry it has taken awhile to post here. We have been working on the problem and hoped to come on once we fixed it. Our tech team did an analysis of the performance of the application and database servers, and determined that while the database server is performing fine, the application server has been maxing out. Our tech team will come on to post more detail on that.

We bought and configured a second application server and deployed it earlier this week. However, there have been some problems with that implementation which have prevented it from going live. They are working on the problem this morning. I'm optimistic this will go live shortly, but will hold off setting an exact time to avoid any false promises. The expectation is that this should fully solve the problem. If it proves to be only a partial solution, we will order a third application server to further handle the load.

Thank you for your patience on this. I know the site has been frustrating recently, and we hope to get this fixed completely this week.

Also, we have hired a person who is going to be responsible for daily involvement on Wikitravel. We have been remiss in not being more present here on Shared, and she will be remedying that. She just started and we'll introduce her later today.

I'm also meeting today with our tech team to come up with a better process for responding to bug notices and other tech requests. We need to be doing a better job on this, and we will. Redondo 12:53, 21 February 2008 (EST)

This is all great to hear. Please, in the future, don't wait to respond until the problem has been fixed. This has been a huge problem, and we needed to hear from you to know that someone actually is working on it; wikis really do require an exceptional amount of openness to keep everyone working together on the same page. --Peter Talk 16:18, 21 February 2008 (EST)
That's fair, Peter, and I agreed. User JuCo (Justine) will be taking the lead for IB as a daily presence on the site, not that Kevin, me and others will be on less frequently. Redondo 16:59, 21 February 2008 (EST)

New database server[edit]

Separate from our launching a second application server, the existing database server somehow got corrupted. We brought together a team of folks who spent the day building a new database server which is now up and running. This is the reason the site was so problematic today. Our head of Network Operations wrote:

"Wikitravel appears to now be fully up again. Due to the numerous issues we were seeing on the site's old Database server, we moved the database to a different server. This involved pointing the front-ends to the new DB server, which we've now completed and tested. We've now tested the sites both externally and internally and all appears to be working. Please let me know if you see any other issues. A BIG THANKS to Jim Fuentes, Larry Wang, Danny Diaz, Rob Vicchiullo and Drew Beach for all their work on this site over the last few days! Great work, guys."

Now that this is taken care of, we will re-focus our attention to deploying the second application server, which should be finished early next week. Redondo 23:34, 22 February 2008 (EST)

Most of the site does indeed seem to be running much better than before, but there are still some pages which will not load no matter how many times you refresh. See Dallas, Denton, Brisbane, etc. Particularly annoying since Dallas and Denton have been my pet projects of the last few days. The error message reads "1016: Can't open file: 'text.MYI' (errno: 145) (wikidb.internetbrands.com)". Texugo 23:53, 22 February 2008 (EST)
Hate to tell you, but I'm still having frustrating problems editing en:wt pages of many descriptions, because of the same intermittent outages that we've had for the last couple of days. This isn't solved as of midnight server time. -- Bill-on-the-Hill 00:05, 23 February 2008 (EST)
This looks like a familiar, and really obnoxious problem. Bug fix: Clear the cache of the page in question, and the error message will go away. To do so, type
http://wikitravel.org/wiki/en/index.php?title=Dallas&action=purge
into your site address bar and hit enter (for other pages, replace Dallas with the article title). My hunch is that we'll have this problem for a bit for every single page anyone tried to access while the DB server was down. --Peter Talk 00:15, 23 February 2008 (EST)

I'm not sure if it's due to the database upgrade or some other factor, but at least at the moment performance seems much better when doing diffs of large articles. Previously I had quit looking at diffs for articles such as en:United States of America because it would take upwards of a minute to generate the page, but today it's taking a matter of seconds. Thanks for the upgrade! -- Ryan 21:26, 23 February 2008 (EST)

Just wanted to give you a heads up here. I spoke with the tech staff here at Internet Brands and they're doing some ongoing tuning of the more robust server they migrated the db to last week. They will be adding another web server to the environment today which should bring the performance back up to speed. If we don't get where we want speed-wise, they will install another set of servers to act as caches in front of the app servers. JuCo 20:57, 26 February 2008 (EST)

Still a problem?[edit]

Still a problem? I suppose it is not, or we would have heard about it constatntly. Closing ticket. Riggwelter 13:25, 22 June 2012 (EDT)

Variants

Actions

In other languages