Slowdown on VPS package

[Notawiz]

Hello,

Before opening a ticket, I thought of submitting this matter first in the forum.

A long while ago, I noticed that there was always a lag time when I consulted our website online.
Since it is pretty complex, I first thought that my way of writing my source code was slowing down the page loading. But upon close examination, once some page started to load, it was completed very swiftly. (did some tests with microtime() comparisons etc.)
Then I also noticed the same delay when working with phpmyadmin, and that had nothing to do with my php source code.
Still pursuing my investigations, I found the same delay when retrieving email messages, or when connecting to the ftp server and even when accessing our HSPc control panel.

I have to say that I use a 2048 kbits/sec broadband connection, and no dial-up modem.

But anyhow, I thought that it could have something to do with the way our French trafic was accessing the backbones before crossing the ocean or something like that, and left the problem aside. Till recently several users from all over the world started also to complain about this phenomenon of a "very slow server" as they stated it incorrectly (because once responding, he is not slow).

In fact, it looks as if each of our http requests is put in a buffer queue before the server handles them and sends his http response (change into ftp or smtp/pop etc because of the same behaviour).

Now this might be a standard setting for some free mail servers like yahoo and the lot, where buffers are frequently used to balance the server load, but as far as I recollect it was not stated in the VPS package specifications that buffer queues were used on the http or ftp protocols of the package.

Can you throw some light on this behaviour please, and let us know what causes this delay between page requests and reponses?

Thank you in advance.

[WHSP-Jarrod]

Hello,

Before opening a ticket, I thought of submitting this matter first in the forum.

A long while ago, I noticed that there was always a lag time when I consulted our website online.
Since it is pretty complex, I first thought that my way of writing my source code was slowing down the page loading. But upon close examination, once some page started to load, it was completed very swiftly. (did some tests with microtime() comparisons etc.)
Then I also noticed the same delay when working with phpmyadmin, and that had nothing to do with my php source code.
Still pursuing my investigations, I found the same delay when retrieving email messages, or when connecting to the ftp server and even when accessing our HSPc control panel.

I have to say that I use a 2048 kbits/sec broadband connection, and no dial-up modem.

But anyhow, I thought that it could have something to do with the way our French trafic was accessing the backbones before crossing the ocean or something like that, and left the problem aside. Till recently several users from all over the world started also to complain about this phenomenon of a "very slow server" as they stated it incorrectly (because once responding, he is not slow).

In fact, it looks as if each of our http requests is put in a buffer queue before the server handles them and sends his http response (change into ftp or smtp/pop etc because of the same behaviour).

Now this might be a standard setting for some free mail servers like yahoo and the lot, where buffers are frequently used to balance the server load, but as far as I recollect it was not stated in the VPS package specifications that buffer queues were used on the http or ftp protocols of the package.

Can you throw some light on this behaviour please, and let us know what causes this delay between page requests and reponses?

Thank you in advance.

[snapback]289[/snapback]

Hi Jan,

We've removed the "ratebound" setting from your Virtual Environment and have re-synchronized it with the hardware node, and it now appears to be responding at a much more reasonable rate.  Please test this again and let us know if you are still noticing slow throughput from the VPS.

Jarrod,
WebHSP Support Team

[Notawiz]

Hi Jan,

We've removed the "ratebound" setting from your Virtual Environment and have re-synchronized it with the hardware node, and it now appears to be responding at a much more reasonable rate.  Please test this again and let us know if you are still noticing slow throughput from the VPS.

Jarrod,
WebHSP Support Team

Hello Jarrod,

Thanks for the fast answer.

By the way, what's a ratebound? Could not find it on google.

When testing the ftp access, the difference is noticeable, as well as for the mail server. I tested those first, because I'm certain that in these cases it has nothing to do with "spaghetti code" I might have produced occasionnally.

I'll keep listening to our user feedback to see if this has the expected effect also for the web pages. Let's agree that if I remain silent about this, the "lag" mostly disappeared.

[WHSP-Jarrod]

Hello Jarrod,

Thanks for the fast answer.

By the way, what's a ratebound? Could not find it on google.

When testing the ftp access, the difference is noticeable, as well as for the mail server. I tested those first, because I'm certain that in these cases it has nothing to do with "spaghetti code" I might have produced occasionnally.

I'll keep listening to our user feedback to see if this has the expected effect also for the web pages. Let's agree that if I remain silent about this, the "lag" mostly disappeared.

[snapback]292[/snapback]

Hi Jan,

We use the term "ratebound" to describe the guaranteed network speed value that is available to be set on VPS' running on the HSPComplete system.  It may be a term that was originated internally, however, but I'm not sure, sorry for any confusion that may have arisen with it

Sometimes, if a VPS exceeds any of the resources set in the "Bean Counters" file (/proc/user_beancounters), the system can set it as "ratebound", slowing things down with the intention of correcting this.  The variables are set as follows in that file:

PRIMARY PARAMETERS:

    * avnumproc - The average number of processes and threads.
    * numproc - The maximal number of processes and threads the VPS may create.
    * numtcpsock - The number of TCP sockets (PF_INET family, SOCK_STREAM type). This parameter limits the number of TCP connections and, thus, the number of clients the server application can handle in parallel.
    * numothersock - The number of sockets other than TCP ones. Local (UNIX-domain) sockets are used for communications inside the system. UDP sockets are used, for example, for Domain Name Service (DNS) queries. UDP and other sockets may also be used in some very specialized applications (SNMP agents and others).
    * vmguarpages - The memory allocation guarantee, in pages (one page is 4 Kb). VPS applications are guaranteed to be able to allocate additional memory so long as the amount of memory accounted as privvmpages (see the auxiliary parameters) does not exceed the configured barrier of the vmguarpages parameter. Above the barrier, additional memory allocation is not guaranteed and may fail in case of overall memory shortage.

SECONDARY PARAMETERS:

    * kmemsize - The size of unswappable kernel memory allocated for the internal kernel structures for the processes of a particular VPS.
    * tcpsndbuf - The total size of send buffers for TCP sockets, i.e. the amount of kernel memory allocated for the data sent from an application to a TCP socket, but not acknowledged by the remote side yet.
    * tcprcvbuf - The total size of receive buffers for TCP sockets, i.e. the amount of kernel memory allocated for the data received from the remote side, but not read by the local application yet.
    * othersockbuf - The total size of UNIX-domain socket buffers, UDP, and other datagram protocol send buffers.
    * dgramrcvbuf - The total size of receive buffers of UDP and other datagram protocols.
    * oomguarpages - The out-of-memory guarantee, in pages. Any VPS process will not be killed even in case of heavy memory shortage if the current memory consumption (including both physical memory and swap) does not reach the *oomguarpages barrier.

AUXILIARY PARAMETERS:

    * lockedpages - The memory not allowed to be swapped out (locked with the mlock() system call), in pages.
    * shmpages - The total size of shared memory (including IPC, shared anonymous mappings and tmpfs objects) allocated by the processes of a particular VPS, in pages.
    * privvmpages - The size of private (or potentially private) memory allocated by an application. The memory that is always shared among different applications is not included in this resource parameter.
    * numfile - The number of files opened by all VPS processes.
    * numflock - The number of file locks created by all VPS processes.
    * numpty - The number of pseudo-terminals, such as an ssh session, the screen or xterm applications, etc.
    * numsiginfo - The number of siginfo structures (essentially, this parameter limits the size of the signal delivery queue).
    * dcachesize - The total size of dentry and inode structures locked in the memory.
    * physpages - The total size of RAM used by the VPS processes. This is an accounting-only parameter currently. It shows the usage of RAM by the VPS. For the memory pages used by several different VPSs (mappings of shared libraries, for example), only the corresponding fraction of a page is charged to each VPS. The sum of the physpages usage for all VPSs corresponds to the total number of pages used in the system by all the accounted users.
    * numiptent - The number of IP packet filtering entries.

By removing ratebound from your VPS, normal speed and functionality has been restored.  If anything else happens, let us know!

Jarrod,
WebHSP Support Team

[Notawiz]

Hi Jarrod,

Wow, that sounds pretty tough. I went looking that file user_beancounters, but silly me, found it empty.

I was wondering what could have caused our VPS to have exceeded his resources.
Just a wild guess perhaps: the lookup of the DNS that was implemented (function gethostbyname() of PHP) to be able to run the site on the previous Shared cPanel AND the new VPS at the same time before the switch in May?

Thing is I'm busy rewriting part of the code to delete that DNS/IP query on every page call. So if it were that, the "overload" will soon disappear also.

But even then, we don't have that much trafic yet, and will probably never have due to the very narrow market niche we are doing business in.

Anyhow, if you have a clue of what causes the VPS to exceed its allocated resources, and that it is something I should/could correct, I'd be glad to hear.

[jamesy]

Hi Jarrod,

Wow, that sounds pretty tough. I went looking that file user_beancounters, but silly me, found it empty.

I was wondering what could have caused our VPS to have exceeded his resources.
Just a wild guess perhaps: the lookup of the DNS that was implemented (function gethostbyname() of PHP) to be able to run the site on the previous Shared cPanel AND the new VPS at the same time before the switch in May?

Thing is I'm busy rewriting part of the code to delete that DNS/IP query on every page call. So if it were that, the "overload" will soon disappear also.

But even then, we don't have that much trafic yet, and will probably never have due to the very narrow market niche we are doing business in.

Anyhow, if you have a clue of what causes the VPS to exceed its allocated resources, and that it is something I should/could correct, I'd be glad to hear.

[snapback]295[/snapback]

Hi Jan,

I checked the bean_counters file, and it looks like the following resources have been exceeded:

resource: shmpages
failcount: 58

resource: numfile
failcount:  831

The failcount is the number of times the resource was exceeded beyond the limits of your account. As for what's causing it, it is difficult to determine. Send us in a ticket, we may be able to increase your resources individually.

Regards,

James @ WebHSP

[Notawiz]

Hi James,

After all this time I still don't understand what causes this resource outage.

Our website may indeed use rather complex code, but that should only affect the execution time of a script, and put only a strain on the webserver.
But the strange thing is that the whole bunch slows down, web server, mail server, ftp, mysql...

Since the starting date of this topic, all the DNS lookups have been removed, also.

What also puzzles me is that I have never been able to see that bean_counters myself. I mean, I see that in our file system, but always empty (0 KB).
Your collegues have emptied it regularly, but it keeps filling up again within days.

I was just wondering it there is some way to track back the application or page triggering each failcount, so as to narrow down the cause, and be able to address it with the most suitable solution, instead of just resetting the thing once in a while.
Those failcounts must come from somewhere, mustn't they?

Anyhow, thanks for caring about this overall slowdown.

Kind regards.

Jan

Hi Jan,

I checked the bean_counters file, and it looks like the following resources have been exceeded:

resource: shmpages
failcount: 58

resource: numfile
failcount:  831

The failcount is the number of times the resource was exceeded beyond the limits of your account. As for what's causing it, it is difficult to determine. Send us in a ticket, we may be able to increase your resources individually.

Regards,

James @ WebHSP

[snapback]381[/snapback]

[jamesy]

Hi James,

After all this time I still don't understand what causes this resource outage.

Our website may indeed use rather complex code, but that should only affect the execution time of a script, and put only a strain on the webserver.
But the strange thing is that the whole bunch slows down, web server, mail server, ftp, mysql...

Since the starting date of this topic, all the DNS lookups have been removed, also.

What also puzzles me is that I have never been able to see that bean_counters myself. I mean, I see that in our file system, but always empty (0 KB).
Your collegues have emptied it regularly, but it keeps filling up again within days.

I was just wondering it there is some way to track back the application or page triggering each failcount, so as to narrow down the cause, and be able to address it with the most suitable solution, instead of just resetting the thing once in a while.
Those failcounts must come from somewhere, mustn't they?

Anyhow, thanks for caring about this overall slowdown.

Kind regards.

Jan

[snapback]384[/snapback]

Hi Jan,

The reason you can't see the beancounters file is because it actually exists outside of your server and your user doesn't have access to it. It is a process that runs on the hardware node above your Virtual Environment.

I have been monitoring the hardware node, and particularly your VE, but the load is low at the moment. It is difficult to troubleshoot since the issue is not occuring at the moment.

Let us know if it continues to be a problem.

Cheers,

James @ WebHSP