Our hardware failed this morning and we’re stuck waiting on OVH to investigate the support ticket I filed. They claim their support is 24/7 but live chat / phone is just business hours, so worst case we might be down until tomorrow =(
https://status.lemmy.ca/ has an update, but not much to say really except “It’s down until it’s not”
Edit: I’m assuming we’re down until I can call / chat with them first thing in the morning, so expect to be back up early tomorrow (depending on how long the parts replacement takes).
Edit 2: I called them when I woke up at 6:30 PST and hopefully they’ll fix it soon now…
They claim their support is 24/7 but live chat / phone is just business hours
It’s increasingly common for “24x7 support” to mean access to their KB/FAQ and the ability to raise a ticket, while the actual support is a skeleton crew working regular hours.
Not unlike “your call is very important to us…” 😅
You can send us email at any time of day or night!
Just changed all my maintenance messages to
It’s down until it’s not
Thanks! I’ll bookmark that.
Weird to think what happens when you lose a site you visit on a regular basis. I can understand the whole instance going down but us users just get lost when things get turned off. There’s no way to know what happened, why or when or if anything will return.
I had to resurrect my old lemmy.world account to figure out what happened. I guess we have to hang on to two or more unique instance accounts in case our main instance goes down.
Stay well guys and I totally appreciate the admins of lemmy.ca and all the work they do. I’ll patiently wait and look forward to seeing it online again.
At least it’s not like Reddit when the website goes down you can’t use it at all.
I keep trying to open the lemmy app on my phone out of habit.
I did that five times before remembering I had an old account on this instance that I could switch to to post the question and see what communities I used to be subscribed to.
It feels shitty and I was doing the same thing all day. I had to resurrect my lemmy.world account just to get back online. It’s made me realize that I should maintain both accounts for shut downs like this. And now I’m also having to do more in maintaining my Mastodon account too.
They posted on the admin chat that their hosting provider had a hardware failure and is working on providing a replacement server.
It may not get fixed untill tommrow since they are not getting any response from their service provider.
That’s an error message from Cloudflare, their tunnel provider. The server is probably unreachable. Either they’re doing some maintenance, or it’s a hiccup or something went wrong. You need to wait for a moment. Or maybe a bit longer if they’re facing any issues.
Wow, they’re really offline for a long time now. Did OVH really mess up that hard? I guess hardware in a datacenter fails every day? Wouldn’t they be prepared for that and just restore/migrate to a different machine, then?
Yeah that’s usually how it’s supposed to work, but sometimes it doesn’t work properly for a variety of reasons.
Yeah, it’s speculation at this point anyways. I mean I’ve only ever seen webservers with a RAID. So one harddrive failure shouldn’t affect it too much. Then most people rent a VPS and that usually runs on some cluster. Even if a whole machine goes down, it’s supposed to come up again on some other machine automatically within a few minutes. And then you should have tech support of a big hosting provider answer within some timespan. I hope someone will write a summary of the events and link it somewhere. Maybe a lot of things went wrong. Or it’s some kind of error in the specific setup lemmy.ca did.
It wasn’t a hard drive failure but something more severe, since the remote console is non-responsive. I just called OVH to yell at them and press on the ticket, so hopefully should be resolved soon.
You can host a lemmy instance through a cloudflare tunnel??? :0
This very message is coming to you through a tunnel!
Yes of course, as almost any other website.
