Announcement

Collapse
No announcement yet.

Resource limits

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Resource limits

    I am aware that we're under heavy load right now. Basically, at the climax of the Spain/Sweden game we had almost 750 people online, including guests. Looking into getting us a little bit more headroom on the server.

    #2
    I wasn't aware we were under heavy load, Snake, but thank you so much for keeping an eye on it.

    Comment


      #3
      Christ. I bet we regulars were outnumbered five or six to one.

      Comment


        #4
        9 to 1. I’d love to see if I can find out if they are “real” people.

        Comment


          #5
          Excellent time to migrate to a new host. Only second to before a World Cup...

          Comment


            #6
            So Snake is basically saying that we need more threads that people aren't interested in reading? Reporting for duty, sah!

            Comment


              #7
              I got messages about this when I tried to visit the site tonight after the France v Germany game.

              Comment


                #8
                I think it's an effect of the tournament having been good so far and the 3 year wait since the last one, but it could also be the bots getting more sophisticated and being targeted at this tournament.

                Comment


                  #9
                  Do we know if these visitors are genuine or just bots and spammers?

                  Comment


                    #10
                    I got a resource limit notice at about 5.30 this morning.

                    Comment


                      #11
                      I got one early this morning too (probably around the same time as Sporting). Not a time when many regulars are on, it's the time when the baseball thread tends to be the only active one

                      Comment


                        #12
                        Originally posted by Antepli Ejderha View Post
                        Do we know if these visitors are genuine or just bots and spammers?
                        The traditional answer to this has been bots, especially web crawlers

                        Comment


                          #13
                          Originally posted by ursus arctos View Post

                          The traditional answer to this has been bots, especially web crawlers
                          Would that explain why it was so slow to go between sub forums?

                          Comment


                            #14
                            Originally posted by Antepli Ejderha View Post
                            Do we know if these visitors are genuine or just bots and spammers?
                            Generally bots and spammers. There is also a fair bit of load thanks to people trying to hack the site - nothing to worry about, they are just trying various known front doors to see if they are unlocked. Generally assuming we are a Wordpress site - I'm going to see if I can put some kind of block on those URLs because the server is spending a lot of time sending out 404s.

                            I'm getting an upgrade added today and will have a look at the logs over the weekend to figure out why we are under load in the middle of the night.

                            Comment


                              #15
                              Not sure about everyone else, but I'm getting the resource limit reached message increasingly frequently.

                              Comment


                                #16
                                We're under load again for some reason.

                                Comment


                                  #17
                                  Looks like we were getting smashed by some crawler bots again. I've updated the robots.txt file to see if the play nicely and stop hammering the site.

                                  Comment


                                    #18
                                    One thing that also probably isn't helping is the missing avatars. That will be causing unnecessary errors on the database. I'll probably make a backup of the data and then replace any missing ones with a standard image.

                                    Overnight, the bot traffic seems to have dropped by about 20%. I don't want to exclude them all (happy for OTF to get indexed by Google, Bing, DuckDuckGo and Yahoo) but I'll do a bit of tinkering after going through the logs.

                                    I'm also ordering the CPU upgrade on Monday.

                                    Comment


                                      #19
                                      AltaVista and Lycos presumably have to send a man round still to index us using pen and paper.

                                      Comment


                                        #20
                                        Heh.

                                        There’s actually one called Majestic-12. I mean, really. You want to build a reputable business doing this stuff and call yourself that?

                                        Comment


                                          #21
                                          It seems to have got a lot quicker this morning, for me at least, when I'm navigating between sub forums. Before it was dial up slow.

                                          Comment


                                            #22
                                            Originally posted by Snake Plissken View Post
                                            Heh.

                                            There’s actually one called Majestic-12. I mean, really. You want to build a reputable business doing this stuff and call yourself that?
                                            Um, have you looked that name up Snake? Just googled it (AskJeeves wasn't forthcoming, alas) and got this from Wikipedia:

                                            Majestic 12, also known as MJ-12 for short, is a purported organization that appears in UFO conspiracy theories. The organization is claimed to be the code name of an alleged secret committee of scientists, military leaders, and government officials, formed in 1947 by an executive order by U.S. President Harry S. Truman to facilitate recovery and investigation of alien spacecraft. The concept originated in a series of supposedly leaked secret government documents first circulated by ufologists in 1984. Upon examination, the Federal Bureau of Investigation (FBI) declared the documents to be "completely bogus", and many ufologists consider them to be an elaborate hoax.[1][2] Majestic 12 remains popular among some UFO conspiracy theorists and the concept has appeared in popular culture including television, film and literature.
                                            (If you were aware of all that already then your comment still makes sense, if not more so, mind.)

                                            Comment


                                              #23
                                              Yes, I think that I first heard about it in the early 90s from a wonderful little movie fanzine called Trash City. I'm sure that it popped up in an early X-Files episode. I'm as happy to name stuff after TV/movie/music references as the next guy (my mobile phone is "Pork Chop Express" for God's sake) but I'm not going to name my business after it.

                                              Unless I go into distribution, in which case Serenity Haulage here I come.

                                              Comment


                                                #24
                                                Barone Waste Management

                                                Comment


                                                  #25
                                                  Quick update - power pack upgrade scheduled to go live within 24 hours. Basically gives us double the CPU resources (these are ones getting slammed) but also has the handy benefit of giving us some more memory and speed so that should help. Some work on getting rid of the bots has chopped off ~300 "guests" at a time and that makes it easier to look at the logs to work out why we get faults every so often.

                                                  Right. Back to the Real World for a bit.

                                                  Comment

                                                  Working...
                                                  X