Seriously, what happened?

[Datawiz]

I can't find a thread that discusses why were were down.

TIA

[ApexOversteer]

Welcome back, how ya been?

[Mike Mercury]

About 5 days into it... I asked the LiquidWeb group for any info. All they would say is:

"there is no "DNS A" assigned to it.

[Datawiz]

Originally Posted by ApexOversteer

Welcome back, how ya been?

Awsome Apex, and you?

I raced and won in the Lamborghini in a tuxedo today.

See the R8 thread.

[Y2Kvert4me]

I'm currently at work, will post all the details when I get home late tonight. The short version is our nameserver domain had expired, and getting that resolved was a real pita.

Actually, it's still not truly resolved, but I found another way around it.

[NEED-A-VETTE]

Originally Posted by datawiz

I can't find a thread that discusses why were were down.

TIA

We were all right here. WTF were you?

[MikeB]

Having never gone through it, this must be what banned camp is like.

Now, I know.

[VITE1]

It was down? Then who the hell was I talking to?

[JRD77VET]

Originally Posted by NEED-A-VETTE

We were all right here. WTF were you?

Since I'm not on fakebook, nobody shared the workaround

Great to feel wanted

[Chuck A]

when i tried to log on my virus protection went crazy , i was
very pissed off indeed
thank God it wasnt a virus, glad to be back

[Sneaks]

Originally Posted by VITE1

It was down? Then who the hell was I talking to?

And what were the voices telling you?

[VITE1]

Originally Posted by Sneaks

And what were the voices telling you?

Drink more Vodka and play more golf.

[Ms.Gem]

Originally Posted by Sneaks

And what were the voices telling you?

Sometimes it's best not to ask such questions.

[NEED-A-VETTE]

Originally Posted by JRD77VET

Since I'm not on fakebook, nobody shared the workaround

Great to feel wanted

Give me your home number. You know, in case it ever happens again. I promise not to call you 15 times in a row at 3:00 AM.

Unless you're over 60. In that case, 3:00 AM is probably your normal wakeup time.

[_Nomad_]

Originally Posted by Sneaks

And what were the voices telling you?

Mine tell me to kill everyone and escape, but I can never tell if they're serious because they all sound like Shultz from Hogan's Heroes.

What do yours say to you?

[Y2Kvert4me]

OK, here's the really long version of what happened. I'll preface it with a bit of history.

Many of you recall the "exodus" of 2011, when a lot of you discovered this forum and joined. The increased traffic overwhelmed our old (shared) hosting plan and we had to upgrade.
In Jan '11 GB set us up with a dedicated server, and configured it with future expansion in mind. The server got it's own domain, vbfhosting.com, and for the past 5 years, it has hosted 2 websites...Thevettebarn.com, and gbvette.com.

On Fri 2/26, I received an email from Dixie, indicating site problems and that he couldn't access the site. At the time, site was working fine for me, others were here too. I did a check of the site and server control panels, saw no errors, all seemed well, so I dismissed it as a possible localized network outage at first.
By Fri afternoon, I did notice a lot less members than normal here, but I had to go to work, where I don't have any 'net access other than my phone. I checked in occasionslly, and by Fri night it was obvious there was a big problem and when I got home I too could no longer access the site.

Sat morning. I replied to Dixie, who offered to help, and I took him up on that offer, and gave him access to the server control panel so he could look around. With my hectic weekend work schedule, I just didn't have the time to investigate it in-depth.
I also sent off an email to GB Sat am, as I suspected this might be the result of something expiring or perhaps the bill didn't get paid. I hadn't talked to Gary in a very long time, hadn't seen him on here in probably a year, so I had concerns of his well-being too.
On Sat, Rob replied to me with a screenshot of our DNS report, which indicated evidence of the problem, but at the time I wasn't sure what it was telling me. I had already checked the status of the vettebarn .com domain, it wasn't expired....But it's nameservers showed no IP address.

When I woke up Sun morning, the thought hit me to check the vbfhosting.com domain, and sure enough, it had expired 1/15/16 and was in redemption period, domain speak for "way past due, we're cutting you off'.
So now I had found the root cause, but what to do about it? All these hosting accounts were setup by GB, I don't have access to all of it. And I still hadn't received a reply from him.
So Sun morning, I decided to reactivate an account we had with Cloudflare, a CDN (content delivery network). We had used them briefly in 2012 in attempts to mitigate the DDoS attack. Setting up a site with them only requires changing to their nameservers, so I felt that would bypass our expired ones.

On Monday morning, I received a reply from Gary. I filled him in on what I found and told him what I had done to get the site back asap. Since he thought I had a fix going, he took no immediate action, other than opening a support ticket with Liquidweb, the data center that hosts our server. Since I had found the cause, they agreed with my assessment and didn't do any further investigating either

By Mon afternoon, Cloudflare still hadn't activated our account, so I opened a support ticket with them. I received only a canned autoreply, telling me to make the nameserver changes I had made Sunday morning, not much help. By late Mon night I still had not received any personal response from Cloudflare, so yeah I was pissed and since that was only a workaround solution anyways, shifted my thoughts to getting that expired domain renewed and fixing it proper. Now, even though these accounts are GBs, I have access to enough of them to know his typical username/passwords. So I dug a little and found GoDaddy was the registrar for that domain, so I went there to try and hack into his account.
No luck there, but of course I didn't try too many times to possibly get the account locked and make things even worse.
So at 3am Tues morning, sent GB another email urging him to get that domain renewed, since my cloudflare idea wasn't working, and preferable to just keep things as they were.
When I woke up late Tues am, GB had replied with a copy of the invoice showing he renewed the domain for 5 years and paid the $80 redemption fee to bail it out of domain jail.
So then it was another day of wait and see. Lawyers might not accept it, but the plain reality is all DNS changes do take time to propagate the internet. ISPs cache DNS records rather than do a lookup query each time a website is visited.

At 4am Wed, I did a whois and noticed the vbfhosting domain had emerged from redemption status and was again active registered. That too took time to take effect, and it had to happen before the site would be accessible again.

Thurs morning, still no site, so I reopened the ticket with Liquudweb to have a tech smarter than me figure out why we were still down. He said Hostmonster (the registrar of the vettebarn domain) wasn't showing our correct nameservers and told me to take it up with them .
I went to our Hostmonster account, and it still indicated the vbfhosting nameservers as they should....But though I didn't change them, trying to save that setting generated an error message:
"Nameserver doesn't exist and cannot be saved'
So another email to GB asking him to contact Hostmonster support and find out what that error nonsense was all about.

Late Friday, no response from Gary, no indication he pursued it with Hostmonster, so fuk it, I went back the Cloudflare idea. I changed the designated nameservers to theirs, and again reactivated this site on their service. This time it did at least show as active in a timely fashion, not "pending" as earlier in the week. But again, another 24hr wait to see for results. Sat morning, afternoon, nothing. I had to work Sat evening/night, so when I got home @ 1am, I looked deeper into the Cloudflare control panel, and discovered their "auto detect" DNS setting had the site pointed to the server's IP address, not the IP of this (child) website. They are different.
So I corrected that entry, and a few hours later is when I saw the first report that the site was again accessible. Finally.

[Craig]

That's what I figured happened...

[Black94lt1]

Originally Posted by Craig

That's what I figured happened...

Me too Anyway, thanks for all the work that went into bringing the site back

[Giraffe (He/Him)]

Yup. Makes perfect sense.













Said no welder ever.

[Ol Timer]

Originally Posted by NEED-A-VETTE

Give me your home number. You know, in case it ever happens again. I promise not to call you 15 times in a row at 3:00 AM.

Unless you're over 60. In that case, 3:00 AM is probably your normal wakeup time.

You know, some shit's just not funny.