BlackTree Security · Infrastructure · Automation · AI

Railway Traces Customer Secret Theft to a Build Isolation Flaw

Railway’s postmortem describes how an attacker broke part of the isolation around its cloud builds on 20 October and later decrypted some customer environment variables. The hosting provider says the attacker was removed on 21 October. It estimates that variables in at most 1.85% of workspaces, about 100,000, could have been decrypted. The logs narrow down the possible workspaces but cannot identify every variable the attacker decrypted. Railway incident report · Initial disclosure

A build foothold led to cloud credentials

Railway says a previously unknown flaw allowed the attacker to read some files on a host running customer builds. One of those files gave the attacker a credential used by build infrastructure. Railway revoked it and restricted what the builders could reach. The attacker later used an incorrectly configured identity federation rule to obtain a short-lived cloud credential. That access helped them reach a system with both a route to the encrypted variable store and credentials that could decrypt its contents. Railway’s attack timeline

Railway also found that the attacker read the stored histories of 353 builds, including sealed variables, and decrypted linked-account tokens for 1,852 users. Railway says it reset those tokens and notified the affected users. It found no Railway account sign-ins using credentials taken in the incident. Its investigation indicates the attacker focused on AI provider keys, which can be used directly with the providers. Railway’s customer impact assessment

Check the services those secrets could reach

Start with any notification from Railway and the workspaces and linked accounts it identifies. Revoke or rotate potentially exposed credentials at the issuing service, update their replacements in deployments and check provider usage and audit logs around the incident. Pay particular attention to AI billing, linked repository access and secrets present in build histories. Record when each old credential stopped working so a later usage spike can be traced to the right account. Railway incident guidance

Update, 12 November 2026

Railway said customers had incurred at least about US$740,000 in unauthorised AI usage. It offered to help customers pursue refunds from providers and cover eligible charges where the provider would not refund them. The company asked customers to submit claims by 18 November and said it would try to honour later submissions. Railway compensation bulletin

Leave a Reply

Your email address will not be published. Required fields are marked *