Hardware Failure - defender.whspn.net

[jann]

I'm a designer in San Diego. I just checked Adel's victoriangoldfields.com.au and its up YO and hi to Ann also.... and the person in Oklahoma.  I think its happening.

[Adel]

I'm a designer in San Diego. I just checked Adel's victoriangoldfields.com.au and its up YO

Shhhhhh   nobody is supposed to be looking at that site yet. Now I'm gunna have to put a splash page up.      That site was never down.  It's on another server which is why I've been using that email address for the forum.     

We've had 144 accounts effected by this.  It's been a nightmare.   I've just been told our 't's are back online.

'B's 'B's give me some 'B's.

[john]

landspro and tsikes,

there were some issues restoring many "l" user accounts.  I have resubmitted those so they get restored.

barbjl,

I will review the c's for your domain.  It may have also failed.  Your other "b" domain is in the other "b" domains that were missed as previously noted and there is a restore job submitted.


It is apparent that an audit of the restores is needed.  I will be doing that as we are getting near completion to ensure all user accounts are restored.

Another update shortly.

~John

[greeny]

I can ssh into my primary account now and actually get into my home directory, but web pages still return a 403 Forbidden error.

[m1rk]

Hi John,

Problems with these two sites :

from b'  - buganim-law.co.il   ( not found )   and from d' dekel-law.co.il  ( forbbiden ).

Can you check please ?

Thank you,
M1rk

[john]

Hi,

The last group of user accounts is being restored and their range is sou...  -->  wiz..., so some s', t's, u's, v's and some w's

I have restore jobs submitted to restore the b's that were missed and the l's that were missed (both previously noted in this thread).  After those are finished I will compare what should have been restored against what was restored and identify exceptions.  Those that were missed will be restored.  I don't expect that to be a large list but that will take some time and can only be done after the restores in queue are completed.  Otherwise I will be comparing to a moving target.

Now this is important.  If your site has been restored and you are seeing issues like 403 Forbidden issues and the like.  Please log those in a support ticket or email support@webhsp.com.  Quite honestly I am not the one who would be fixing whatever the issue is, our technical staff would be and that is the fastest and best way to get them working on your problem.

After we get everything restored we will turn up email, and we have all been waiting for that, and that is the most important thing remaining to complete.

More updates coming and we get this restore completed.

~John

[john]

greeny and m1rk,

I will look into the forbidden errors.  greeny I know you have a ticket already on that one.

buganim-law.co.il should be restored shortly as it was part of the missed b"s group.

~John

[jann]

John, did u say the "R"s are done, cause still have no mail or site coming up.

[Adel]

John, did u say the "R"s are done, cause still have no mail or site coming up.

we have some rh...  and  ri....  showing.

[john]

Hi,

r's should be done but I need to do an audit and that may show some have been missed.

Once all the restores are done, we will turn up email. 

~John

[greeny]

I found the cause of the 403 errors I've been seeing.  It is a permissions issue.  Comparing an account that works with one that didn't, I discovered that my home directory was restored as 700, and needed to be set to 711 for any web pages to be accessible.

It also appears that some directories under public_html have permissions issues that need to be fixed.

But to the degree other accounts are seeing the same issue, this is what probably needs to be fixed...need to make sure that directories include world execute permission.

[jann]

Still no Rogov site showing. I'm not checking cpanels, just going to the site.

[landspro]

I have nothing...  No home page, no forum, nothing.  It's like a disappeared or something.  Much less getting to cpanel or MySQL...

LAS!  Hopefully, they will finish the L's soon...

[barbjl]

John, when do you want us to submit trouble tickets?  I did so a couple of hours ago and got a "thank you for your patience" response, so I assumed that they weren't dealing with these issues yet.  Thank you, Barb

[Adel]

Thank you John.

And thanks to your team for all the hard work.  

We only have three rogue sites not showing now.  I've sent those in a ticket for your audit.

[m1rk]

Hi John,

The website forbidden problem is still there - dekel-law.co.il
I tried FTP and the error says unable to read. Strange problem only with this simple website.
Does it also make problems with dekel-law.co.il email accounts ?

Thank you,
M1rk

[john]

Hi,

OK, the restores have finished.

The 2nd round of restores of the 'l's encountered the same problem that caused them not to restore the 1st time so we are looking into whats going on there now.

We are going to go ahead and start the audit while that is investigated and resolved.

I am escalating the Forbidden error and will get that addressed ASAP.

More updates to come, but we are real close to turning up email and that is a key milestone at this point.

~John

[Mussel Bound]

Hi, everyone - Mukilteo, Washington here. I see we're spread far and wide. Sympathy to all of you in miserably hot weather while you deal with this!

Has anyone else had email sending issues, particularly before the crash? Mine stopped sending mail about two weeks ago; I put a ticket in Wednesday night (delay was medical issue).  
It started right when they announced they were moving us from the tempest server. Right after the tech said this new server was having some problems Wednesday night, the big crash happened.

Sites are back now (main one is lobsterdude.net) and all the old mail is catching up. Still won't send, and they're still attributing it to the server crash.

I've tried port numbers 25, 26, 27, 465, 567, 577, 587 and 995, with and without authentication, before and after the crash, and even tried smtp.lobsterdude.net in frustration today. I've set it up on three different computers than my own, all with different ISPs than mine, same result - receives but won't send.

Mac mail program, it just says "unable to send using mail.lobsterdude.net" and offers the drop down menu of my other servers.

Thanks for any insight or advice anyone has!

[john]

Hi all,

I have gotten the restore of the "l" account completed.  So all restores are finished and I just need to audit everything to make sure no accounts were missed.

I will be turning up email shortly as we should be all set on that now.  I just need to touch base with the techs to make sure all is good from their perspective.

A big THANK YOU goes to greeny who figured out the Forbidden issue.  M1rk I have corrected that for dekel-law.co.il and that site now loads.  I will be reviewing all accounts to identify others.  Thanks greeny!

More updates to come.

~John

[john]

Hello,

Email is now turned on, so you should be able to send and receive email.

The issue with the Forbidden websites has been corrected.  If you are still having that issue get a ticket into our support desk so someone can investigate.

I am now undertaking the audit of the restores to make sure no accounts were missed and restore them as needed.

~John