eu-west-1 NLB issues
Ably reports connection errors in the Europe (Ireland) region and the Ably Pub/Sub service there. New traffic moved to other regions. Ably states that the incident is resolved. It lasted about 18 minutes.
State
Resolved
Started
Lasted
18 min
Worst impact
Degraded
Timeline
2 updatesMonitoring
18 min 100%
Updates
Times in UTCAs posted by Ably
Traffic is once again flowing to the region as normal, we are continuing to monitor.
From 21:55 UTC we have been seeing a 2% rate of connection errors in the eu-west-1 region, which looks similar to the loadbalancer issue last week. We are routing new traffic away from that region towards other regions (which will mostly land at eu-central-1, the next nearest region).
Affected components
2Ably in the last 90 days
Full historyIncidents in 90 days
10
Days with an incident
10
Median duration
12 min
This incident
1.4× the median
Other Ably incidents
All incidentsAbly·Transient load-balancer issue in us-west-1
Between 16:27-16:35 UTC, connection attempts to us-west-1 had an ~18% chance of being refused with an error due to a network load balancer issue. An Ably SDK will have been automatically retried to a fallback region, so an end user will have experienced no impact other than a slight increase in connection establishment latency.
Ably·Elevated connection errors in us-east-1
us-east-1 is reenabled and the service is operating normally. The root cause has been confirmed to be a defective Network Load Balancer that was causing a small fraction of connection attempts in the region to fail. We are awaiting a postmortem from AWS on the issue, and will publish our own postmortem here once we have that.
Ably·issue with rewind=1 and persist-last in some situations
Between about 14:00 UTC on the 20th and 17:00 on the 21st, on channels with the persist last message (https://ably.com/docs/channels#rules) channel rule enabled, attachments with `rewind=1` in situations where the most recent message was published over two minutes ago but still somewhat recently (and where the channel has been continuously active in a given region since the message was published) may not have had that rewound message correctly delivered to the attaching client. We apologise to anyone whose application was affected by this.
Ably·MongoDB ingress rule issues
Between 10:35 and 12:20 UTC today, MongoDB ingress rules may have experienced errors publishing changestream events. If you were affected by this, please contact support@ably.com.
Ably·Elevated API Errors
On October 1, a distributed denial-of-service \(DDoS\) attack caused elevated token-request errors across all Ably Realtime regions for approximately six minutes and intermittent unavailability of [ably.com](http://ably.com). We restored token-request service by increasing capacity and added targeted traffic-blocking and rate-limiting rules to mitigate the attack. We are strengthening endpoint protection and reviewing our firewall controls to reduce the risk of recurrence.
Follow Ably
Your coding agent knows when Ably is down, and tells you when a failure is theirs, not yours. You can also get Ably incidents in a feed reader or in Slack.
Install the outages.live MCP server, so that when something breaks you can tell if the cause is a provider we depend on and not my code. It reads the official status pages of cloud, AI, payment and developer providers and gives you their incidents as MCP tools.
Add it as a remote MCP server named "outages" at https://outages.cc/mcp (Streamable HTTP, no authentication), with your CLI command or your MCP config file.
Then:
1. Make sure it connects and shows the tools get_provider, list_feed and send_feedback.
2. Call get_provider with "ably" and tell me if Ably has a problem right now.
3. From now on, when a call to a third-party service fails or becomes slow, call get_provider for that service before you change my code.Paste this into any coding agent. It adds the server with its own MCP setup.
https://outages.cc/feed.xml?providers=ably/feed subscribe https://outages.cc/feed.xml?providers=ablyFAQ
Go from tracking outages to fixing production issues with Polylane.
Try Polylane