bug report

[hgjf2]

This morning.
The site FRACTALFORUMS was encountered many times this error:
Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
That's I told other users why mustn't posting too many videos at FRACTALFORUMS, that videos has much megabites and can make FRACTALFORUMS servers to crash.
 
Sorry for report.

[Sockratease]

This morning.
The site FRACTALFORUMS was encountered many times this error:
Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
That's I told other users why mustn't posting too many videos at FRACTALFORUMS, that videos has much megabites and can make FRACTALFORUMS servers to crash.
 
Sorry for report.

Hi.  Bug reports are always welcome and there is never a need to apologize for making them!

Even in cases like this where your guess as to what is causing them is 100% wrong!  

That is simply not the way the internet works.  You are seeing a server error.  Embedded videos have a near zero impact on our server - they do have an impact on your browser and can crash that, but server errors are an entirely unrelated issue.  We can embed as many videos as we want from sites like youtube, vimeo, etc - and 99.99% of the impact is on their server shared with your browser.  This site does not host any videos and so there is no way the embedded videos can be causing this error.

I know we have had this conversation before and I am sorry that you do not believe it - but facts are stubborn things.  They do not cease to be true just because you do not believe them.

The server issues can be caused by many things, mostly heavy traffic due to many users all trying to use the site at the same time - not many embedded videos.

I do not think we have a "dedicated server" - meaning this site is on a server with many other sites also hosted by the service we use to host the forum.  So it could actually be coming from some site not even related to ours!  

This is actually among the most common of errors because it can have so many causes.

But remotely hosted videos are not even possible causes of this particular error since the videos are not on this server!  The worst they could do is crash your browser, but that would never generate a database error.

I hope this is sinking in.  There is no need for anybody to cut back on embedding videos from other servers because of this error!!  

So please stop misleading others, and yourself, that embedding any number of videos has any impact at all on our servers.  It only shows that you do not fully understand how the internet works in regards to servers and local vs remotely hosted content.

Really!


EDIT - I get this error here periodically too.  The only solution is to go do something else for a few minutes and try again.  It never takes long to get back to normal since it is mostly a traffic issue and often not even traffic on this particular site.

I suggest just getting a cup of tea and relaxing when this happens, then try again   

[cKleinhuis]

hi there, what sock said is correct

what annoyes me is that although we have no the largest hosting package that this issues still occur, i need to talk with the hoster why this keeps happening, it might be that they misconfigured web/database acces or something, whereas web acces is unlimited but database acces has a bottle neck of limited access, something like that i believe

BUT: it usually clears itself, and high serve load is not a bad thing

[M Benesi]

Haha... is server load because of Andreas announcing the release of new M3D? 

 

[cKleinhuis]

no rather more due to general activity (new posts) and a more loose restriction regarding robots, most of the traffic is generated by these hungry beasts, average impression of real people is 1000 impressions a day in average lower compared to november, if you take then into account that the forum counts like 70.000 impressions a day, this is clearly due to hungry robots, if it is going to happen too often in the next days/weeks the robots restriction will get more tighten, but i genuinely want to include the gallery in the robots to let people find some nice 4k images stored here, so, at the end it comes down that the server could use some even bigger extensions and load balancing for database and web parts, but this is not going to happen any time soon ... or just the database should handle more connections, i am going to talk about some slight improvements with my hoster before end of year

when talking about bots - some of them seem to prefer night-time but since this is such a huge international crowd here it can happen to anyone, anytime ....

[cKleinhuis]

i installed a new modification to get informations about the bot counts, now you can see below of the online users a list of active bots in the last 15 minutes with individual counts, by the time of writing this reads:


Google spider (3), Feedfetcher-Google, Yahoo spider, Google AdSense spider (3)


which comes down to 8 spiders actively consuming server bandwith/connection/computing time

[zebastian]

70000 impressions a day should be managable. In fact More than 1000 impressions a second should be manageable with a proper Server setup.
I would say FF outgrew this hosting Konfiguration.
The question is how portable is your Server code.
Would you be able to move to a dedicated root Server and would you like to?

[DarkBeam]

Haha... is server load because of Andreas announcing the release of new M3D? 

 

The evil developer strikes back 

[cKleinhuis]

70000 impressions a day should be managable. In fact More than 1000 impressions a second should be manageable with a proper Server setup.
I would say FF outgrew this hosting Konfiguration.
The question is how portable is your Server code.
Would you be able to move to a dedicated root Server and would you like to?

i think so as well, it is not too much to do but the server hangs, it seems that the biggest 1and1 hosting is not performing very good, i do not want to have any trouble moving the webserver to another hoster, or something it is very easy how it is and i want to continue with that i am going to check the customer service on their side

[3dickulus]

a more loose restriction regarding robots, most of the traffic is generated by these hungry beasts,

on my server I see many more bot accesses, they only ID themselves sometimes, like when accessing robots.txt, I see accesses from the same IP but IDd as browser sifting through my site, (nobody can click that fast) I black list IPs based on behaviour not based on the ID presented. BingBot is one of the worst, so is Baidu and yandex, ignoring rules in robots.txt completely. Then there are the hack bots looking for vulnerabilities, imho, you need to block bad behaviour IPs with rewriterules and a bit of php code. All "bad" accesses on my server get redirected to bad_conduct.php which adds a block rule to .htaccess

I would be happy to share my rewrite rule list, collected over a few years of patient observation, it blocks exploit vulnerability hunters, SQL Injection via _POST, SQL Injection via QUERY_STRING, and others...

if you are tracking via HTTP_USER_AGENT your numbers are about 0.1% of real bot accesses.

[cKleinhuis]

that sounds reasonable, the .htaccess file here on the forum is already quite huge, but ip filtering for annoying bots seems to be quite important
baidu is already blocked completely (although i am not happy about removing a chinese bot completely )

[3dickulus]

I'll post you the apache virtual host config and the php file that adds a rule to .htaccess, it should be pretty (httpd server admin) obvious what to do with it, but do check the rules carefully, I have a few things in there that you may not want or may need refactoring due to your particular setup, I also block and turn off logging for baidu