Slack Is Down | Hacker News Reader

Slack Is Down

slack-status.com

190 pointspolyrand51 comments

Minor thing that bugs the heck out of me: I can't login to Slack, it's a total outage for me, but on their status page [0] their copy says, "Something's not quite right."

In reality, there are probably folks freaking out internally at Slack with their hair on fire, and probably tens of thousands of people are locked out of their business comms. Saying "something is not quite right" trivializes and downplays the severity.

No! Something like, "We have a real problem" would be much more appropriate and not pretending like "oh it's no biggie, we'll take a look later."

[0] https://slack-status.com

It seems that Slack being fully down and unusable only reaches the "incident" category on their scale - I'd hate to see what counts as "outage"

Yeah, and the incident details indicate call it "degraded functionality" when it seems broken for everyone across the board. Desktop app, website and mobile app all non-functional.

Oddly, my mobile app still has me logged in and seems to work, but the desktop app switched to its stupid 'oopsy daisy, something not quite right' screen on its own.

Can you send messages or see anyone online?

I think it's AWS or something similar related. Seems to be specific to some but not all locations.

classic SLA hacking

Their status page still says "Uptime for the current quarter: 99.996%".

I worked as a sysadmin during a major outage a few years ago. A bug completely destroyed a table in one of the customer databases; we had to restore/rebuild the table. It took hours. We reported it to our leadership; they handled the comms. When the message got translated and posted on the company's status page - well, it sounded like we were doing our customers a favor. Nothing said was technically untrue, but I was amazed at the spin from "We fucked up and we're working to fix it" to "We're doing this all for you; you're welcome". Ever since then I've been skeptical of status pages.

To be fair, their status page reported problems super early. It might even be automated. Usually you have to search Twitter for accurate information on outages.

It really didn't -- we (Slack customer and integration) had our first internal reports of issues at 09:59 ET and the status page was updated at 10:27. For this broad an impact, that's a long time.

At a previous company that focused on small-business finance, we had an error page that said something like "We couldn't process your request, but we're working on it! In the meantime, here's a video of a puppy." with a link to a Youtube video of a puppy.

PEOPLE HATED THE PUPPY.

"I can't process SALES, i'm losing MONEY and you want to give me a VIDEO of a PUPPY?"

We removed the puppy.

Our company of >100 people is all remote and all our internal comms are through Slack.

The silence is deafening, surprisingly so, when Slack goes down like this. It happens so rarely (to the Slack team's credit), but when it does, it feels just like when the power unexpectedly goes out.

We totally forget how much we rely on it until it disappears.

Consider a Signal group or some sort of backup comms channel.

Edit: Email can have latency, which is why I did not recommend it.

Email is always there as a backup, but no one in the company uses it anymore, to the point where we have to remind people to check it incase an external vendor reaches out.

Slack has provided so much additional functionality that trying to cram day to day working back into email just doesn't work anymore.

I had to pull everyone I could into newly created Gchat "Spaces" (Did anyone else just discover that even existed? I didn't know about it) -- though it's hard to get the message out to go there when normally that's the kind of thing I'd tag @channel in #general.

And my other go-to communication method is grabbing people's phone numbers from their slack profile. D'oh!

Purely coincidence but we just posted about emdash, our Slack/Zoom alternative on Shown HN: https://news.ycombinator.com/item?id=43184362

Check it out @ https://emdash.io

oooh; i certainly will!

My entire company is unable to access it.

Good reminder that we need a backup real time messaging app.

> Good reminder that we need a backup real time messaging app.

I'm quite enjoying the quiet. There's still video or email in an emergency.

HN and IRC! ;-)

Four hours and still broken for me. Not sure if we will get a postmortem but very curious what went wrong.

Good luck to SREs over there! (I know, they would say luck is not a strategy)

It must have been one hell of a morning. It took them 5 hours to identify the cause and everything is still broken for me.

> Our investigation is still in progress with regard to deprecated functionality for Slack features such as workflows, threads, sending messages and API-related features.

I'm not sure what their status page is talking about, is sending messages a deprecated slack feature?

Pretty sure that's a typo for "degraded". But it's funny.

And someone, somewhere, has further mangled "deprecated" into "depreciated" when reading it aloud, I'm fairly certain.

What do you mean messaging has been depressed?

Me! I'm the notorious mangler!

Just kidding. But there was a sizable fraction of time where I really hadn't heard the word "deprecated" and read it mentally as "depreciated" like an asset.

Their status updates look clearly like AI-generated blurbs saying the same thing with varied phrasing, hour after hour. Thanks for the slop? I know writing status page updates is annoying, but “a human is paying attention to this” is the specific thing that status updates are trying to convey, so trying to get info from this page felt dispiriting today.

Edit: this would also explain why some details of the updates were nonsensical.

Yes, it's been replaced by an LLM.

Everything seemed fine at around 9:00 AM PST (UTC-8). A few minutes later, I made the mistake of reloading Slack after a few messages began failing to send. Haven't been able to access my company's Slack workspace since. At the time of writing this post, it's now 11:00 AM PST.

Emailed my manager explaining the situation. Hopefully they can understand. Also plan to ask others about Slack alternative for situations like this. For now, we are relaying things through e-mail.

You can tell Teams, gchat, discord, and other have taken away from Slack's dominance. This isn't even frontpage.

I think this post could have improved visibility by prepending "Tell HN" to the title. That's what my eyes were searching for on the front page.

It's interesting to me that it has so many points (140) and comments (31) and it's still not on the front page.

I can't access the desktop or web versions but my phone app still seems to be working, at least intermittently.

Based on the status report, it seems like it was related to a database disaster and mention recovering shards. Publicly they say they are using vitess[0] so I wonder if there was some issues there.

I wonder if there will be a more detailed post-mortem, would be super interesting to learn more!

[0] https://slack.engineering/scaling-datastores-at-slack-with-v...

This is another remainder that crucial business communication should be based on self-hosted solutions to be independent on third-parties and for privacy reasons. There are good and even more lightweight open source Slack alternatives: Nextcloud Talk, Mattermost, Zulip, Element.io, Jami, to name a few. Relying on a third-party provider is asking for trouble. Plus giving away your private business conversations to feed AI and NSA is another thing.

I bet 90% of those installations are not self-hosted, they are using a hosting provider anyway.

Another alternative is https://www.rocket.chat/ .

Came to post the same. The mobile client still seems to work for me though but the electron client is down.

Edit: the mobile client loaded messages but also fails to send.

It also seems like all integrations that use Slack webhooks are working. But the ones which are like a dedicated app are failing with the same `fatal_error` message.

Hopefully slack will stay down long enough for me to get in a few good ski runs ;)

All my apps are returning an auth:test fatal error:

   AuthorizeError: Failed to call auth.test API due to Failed to call auth.test due to fatal_error

The slack app management page also seems down

Some slack app functionality seems to be back with commands and link previews working but message events still seem to be broken

I honestly thought I took down Slack when I posted a little too long message only to realize it went down globally.

PSA as a stopgap, mobile has worked for me for most of the day.

Will be an interesting post mortem to read...

Slack status reports that workflows are affected now (11:08 EST)

Slack is back up for me.

Just went down for me

same

Reply on news.ycombinator.com