UESPWiki:Administrator Noticeboard/Archives/Server Connection Limitation

This is an archive of past UESPWiki:Administrator Noticeboard discussions. Do not edit the contents of this page, except for maintenance such as updating links.

Server Connection Limitation

I've added a new module to the web server to prevent a single, or a few sites, from monopolizing most or all of the site's connections. Basically the current web server can accept up to 50 simultaneous connections (limited by memory). If a misbehaving client (whether on purpose or not) requests many pages in a short period they can easily take up half of these connections. This begins to create a backlog and eventually you end up waiting for a while to access a page or receive a time out (as some may have noticed happening recently).

I've set it up with settings that seem to work (IP 165.228.129.12 is currently spidering or trying to download all content as I type this) but I want to make sure it does not interfere with regular users. If, over the next few days/weeks, see any missing/broken content let me know. If you do trip the limit you will be denied access for that resource, whether be it a whole page, an image, or part of a page. Just doing some random browsing now and I don't notice anything but keep an eye out for things just in case.

Thanks... -- Daveh 22:08, 12 April 2007 (EDT)

I did notice one thing related to this since I set it up. On pages with multiple images (like Oblivion:Merchants) if you reload the page, or its a page you haven't visited before, then some images are not loaded. After some tweaking I've changed the settings such that images are not currently included in the concurrent connection limitation and it seems to be working fine now. Later I can add a bandwidth throttle to just images to prevent someone from monopolizing connections or bandwidth for the images.

Other than that though everything else seems fine. -- Daveh 10:26, 16 April 2007 (EDT)

I have noticed that sometimes pages are shown without the menus on the left and no stylesheet. Could this be related? --DrPhoton 11:46, 16 April 2007 (EDT)

It certainly could be. I recently changed it so it should ignore image/skin data so if you are still seeing this let me know. You can also try forcing a page reload or clearing your cache to check for this problem. -- Daveh 12:25, 16 April 2007 (EDT)

I had the settings wrong originally but they should be correct now (as of last evening) to ignore all images, stylesheets, and JavaScript files when counting the concurrent connections. Current limit is 5 and I may play with this up/down as needed. There was one IP last night that was attempting to download every page again and the site was managing just fine. -- Daveh 14:12, 17 April 2007 (EDT)

(Hi everyone, just hopping into your seemingly rather exclusive circle, hope you don't mind) I wonder if that behaviour might also occur from clients using firefox with fasterfox-extension as that is pre-fetching aggressively. If i guess right, then maybe it's a good idea to leave a remark about it on some prominent location. Even better would be some detection-mechanism like dnsstuff.com uses, that displays a warning to the client and blocks access automatically. Oh well, just an idea... (Yeah right, I got that one, because I got that very warning some days ago myself...) BTW, talking about ideas, how about modifying the "Random Page" link on the left menu so that it first leads to a intermediate page, that allows the user to narrow down a section? "Random Oblivion Page", "Random Morrowind Page" and so on. Would make that one much more useful in my opinion, especially for inspiration purposes. Just my 0,02€...Micromegas (How does that time-stamp thing work? Too stupid to figure that one obviously)

I'll check out the faster-firefox mod and see how it manages. I can, and should, create a custom 503 response page which is served when the concurrent limit is reached to notify people of the reason.

To create a timestamp+username just use four tildes (~~~~) at the end of the edit. The Wiki will automatically convert this to the current date plus your username when you save the page...like this: -- Daveh 14:09, 17 April 2007 (EDT)

Ok, that's easy. Thx. Here's a link to the suspected culprit: https://addons.mozilla.org/en-US/firefox/addon/1269 Micromegas 15:11, 17 April 2007 (EDT)

This morning (GMT+1) I access the Recent Changes page withput problem, but when openning more pages in other tabs of IE7, I got and Error 503. Refreshing didn't help and I've been unable to access the site. Now, midday (GMT+1), I've been able to open the pages, but got one 503 which when refreshed loaded the page without stylesheet again. Hope this helps debug the problem ;) --DrPhoton 07:01, 18 April 2007 (EDT)

This is probably the same issue that Nephele had due to hanging connections. If you check out [1] and look for your IP address in rows that are gracefully finishing ("G", I see one of yours at the moment). What happens is sometimes connections don't terminate properly and hang there. Get enough of these and you'll hit the concurrent connection limit. It is a bug in the Wiki or Apache and I'm not sure the best way to get around it but I'll consider the possibilities. -- Daveh 09:15, 18 April 2007 (EDT)

And as in the cases that I was having trouble with, the hanging connection is being caused by a diff done on a large page (in this case, my fault for the large page... another one of my big data dumps to a sandbox page). I'm guessing that perhaps all of our problems with hanging rss feeds, even going back six months or more, might be caused by this basic problem of trying and failing to do large (>20 kB) diffs. --Nephele^Talk 09:56, 18 April 2007 (EDT)