don't forget today's NodeJS security update addressing its own RCE bugs!

is the google outage related to a security incident?

all popular, free, ungated software will be poisoned; like kesler syndrome but for the world of open source software.

Sounds like a reasonable choice of a language.

I myself have written a small llm proxy in Go, doing nothing but injecting the real API key to outgoing requests while itself validating virtual keys it generates, and it works great.

If you don't use any LLM router, it's infinity times faster, and infitesimally as targetted.

I didn't realize they've all such added that, this was a change we made to our own projects last year when these first started - glad to see it's common.

Risk is ensuring enough security researchers are finding these and quarantining these before that default cooldown lapses.

We have 7d cooldown on packages on an internal pypi proxy but that blocks security updates also, which we now have to manually exclude from the cooldown.

I’m unsure how to automate this properly, maybe an LLM is the best starting point.

I wonder if in some cases it's not so much that people are updating too quickly, but rather, their deployment pipelines do not have pinned / frozen versions attached to their builds.

The last time I looked for information, I remember seeing that the median supply chain attack was in place for multiple months to years before being discovered. Cooldowns shorter than that seem like snake oil.

Then we’ll find out about malware 24 - 48 hours later.

I use safechain for this, because there is always gonna be another package manager configured wrong. https://github.com/AikidoSec/safe-chain

I’d imagine there were probably fixes for critical CVEs that were included …. A lot of companies are on a 24 hour timeline to get them out.

That's not what's being said, at all. They are simply telling their enterprise customers that there is a different build pathway for the enterprise builds and that pathway wasn't impacted by this issue.

Also if you don't like organizations fixing vulns without releasing the open source first, you're going to be upset when you learn how every major vuln and associated patches are distributed long before the fixes hit the repos. A timely example of this is PHP, literally today.

Note: I work at LiteLLM

There exists a critical vulnerability which affects only Enterprise LiteLLM deployments, and does not affect OSS deployments.

We have provided our customers with patched releases distributed via a private registry and are allowing them some time to update their deployments before we release the fix in our public registry and disclose the vulnerability details. Releasing the fix publicly in an open source product would be a de-facto vulnerability disclosure.

it's just the poor styling of the selection highlight colour.

man, people are going to be sorely disappointed in what's free on the internet in a couple of years.

Agents and malware will eradicate all goodwill left.

Good by internet.

Not sure about a service, but PMG enforces dependency cooldown and sandbox on package install. https://github.com/safedep/pmg

Socket.dev

Tell em Tom Zubiri sent ya (I don't get a commission, just good karma)

This one ? https://news.ycombinator.com/item?id=46909037

I use it, I don't cooldown, proper version pinning for me.

Bytesafe (https://bytesafe.dev). Pure Dependency Firewall, not a package host, with usage based pricing.

What's interesting here is that this will only affect those who did not migrate off LiteLLM after it was attacked the first time.

I'm not saying everyong should migrate after there's a vuln, but I'm definitely saying that. Unless it's something huge and foundational like a programming language (PHP gets a second chance), just migrate within the next couple of weeks.

I didn't even have it installed the first time, so the ideal programmers, (read, your competition), actually suffered 0 vulns from this dep, if you suffered 1, that's already pretty bad, being burnt twice should be a wake up call. If you get hit 3 times by the same dependency, it's a lost case.

Yes if you use LLM to analyse everything. Not really if you analyse diffs, with heuristics to select candidates for LLM/agent based analysis. But at the cost of missing out novel attacks.

Welcome to the AI era, every week in the near future

There are over 8 billion people on Earth, and some subset of the computer systems that they all use are going to have issues, every day.

It's really not as bad as the accumulated headlines and exploitative hype make it sound.

It's similar to the mass hysteria and moral panics over things like child abduction. Media reports make certain dangers seem much more important than they really are, when considered in context.

So they sell advance access to information that will allow me to hack other people's LiteLLM instances?

Seems a little unethical

If that's right then attestations wouldn't have saved anyone here either. They prove which workflow published the wheel, not what source that workflow checked out. A private pipeline building from the same-looking repo still gives you a perfectly valid attestation.

What would catch it is someone rebuilding from the public tag and comparing the result, which almost nobody does for the python wheels. I have been treating attestations as the answer for my own project's releases, so this is a good reminder of where they stop.

This isn't the name of an entire package, but just a single file included in the package. .pth files are part of Python's import system. At startup, the standard library `site` runs automatically, and configures the (initial) places that Python will look for code to satisfy `import` statements. By design (because some people occasionally find it very useful), arbitrary Python code can be run from these[0]. The system searches the entire site-packages folder for these files.

The compromised build packages up a .pth file in such a way that installers will treat it as a "data file"; but because Python programmers are accustomed to looking for data files at paths relative to the current source file, installers put .pth files in places that will easily be picked up by the `site` scan.

So the point is simply that, once LiteLLM is installed, malware injected directly into the final package (not a dependency) runs at startup in the compromised environment even if you don't use the LiteLLM code explicitly.

Nothing "creates a directory from the build file". The attacker-modified source includes the entire "cook" payload directory and also modifies the build file so that the build system would include its contents. No dependencies are involved (unless you're thinking of LiteLLM as being a dependency for something else).

The Python ecosystem has been working on mitigations for this. Starting in 3.15 the .pth file system is being reworked[1], but a deprecation period is necessary. Also, the PyPI repository has an opt-in system called "Trusted Publishing"[2] which is supposed to allow verifying that a wheel was built from the appropriate source directly using GitHub's CI mechanism. (Opt-in because of the long tail of developers who don't use this or any other CI system and shouldn't be expected to learn them for a small hobby project.) However, since the attacker did use the CI system from a separate GitHub repository, and compromised multiple maintainer accounts, I imagine it was possible to defeat that protection as well.

If you don't need the `site` mechanism for your project (of course, this needs to be tested per project), it can be disabled by passing the -S flag to Python[3]. This has always been available, but awareness seems to be low.

Historically, Python packaging has overall had a pretty lax security posture, and new attempts to improve that are often delayed by the consensus process and by backward compatibility concerns.

[0]: It's possible to put arbitrary code directly in the file, but rather unwieldy. Interestingly, this payload is neatly organized: the .pth file simply runs an import statement and appears to keep hundreds of lines of code in two actual Python files: one that works directly, and one that sets up a sub-hook called `sitecustomize` (another explicit provision of the `site` configuration system).

[1]: https://docs.python.org/3/whatsnew/3.15.html#whatsnew315-sta...

[2]: https://docs.pypi.org/trusted-publishers/

[3]: https://docs.python.org/3/using/cmdline.html#cmdoption-S

Sarcasm?

Every harness thinking right now… "if we want do something like this we won't even have to compromise anything."

The compromise appears somewhere in the build and deploy system.

Yeah, thinking that after the last supply chain attack, people would be smarter than ship things with LiteLLM.

Is that you Max?

What are you talking about? Even dependency cooldown of 24h is enough to protect the consumers from these breaches. The attack did not have any novelty, used a newly registered look-alike domain for C2 and a PTH technique which is now known.

Sandbox comes in for the unknown ones and yes, its a pain to get the profile right.

I mean, didn't Linux shells invent pop-up fatigue? I thought we (yes, we) all used to -y flag preemptively so the upgrades and installations/setups go through. Linux really did do it first, the agents are just more needy and their popups more verbose now, but not exactly new.

VM isn't a silver bullet.

Fuchsia solved that

No no it's really important that I have 5-factor authentication on every website and it requires me to go through the whole login process again every time I break eye contact.

No, you can block this with sandboxing tools. For my LLM agents, I use a tool called Nono, whose macOS implementation is built on the same foundation as sandbox-exec. One of the key features it supports (and it's not the only tool that does this) is proxying and the injection of credentials via the proxy. For example, when I run Codex at work, Codex does not have access to my Azure AI Foundry API credentials. It sends out requests via dummy credentials, and the proxy injects the real ones outside the sandbox.

The other thing that proxy does is let you restrict network access in various ways. So you can run something like LiteLLM so that it doesn't have access to the internet at large; it can only talk to Azure AI Foundry or AWS Bedrock or whoever your inference provider is. You sandbox LiteLLM separately from your agents, and you give it access to much less. The software firewall stuff isn't perfect but it is a significant defense against things like this.

There are also lots of other restrictions that are easy to do which mitigate this. Nono blocks access to the root of $HOME, then selectively allows access to paths within it, and also explicitly blocks access to common paths to secrets. So it's pretty easy to make sure that access to important things are blocked: SSH credentials, GnuPG socket (and local Unix sockets in general), local OS keyrings, etc.

The way I use tools like this is exclusively by writing policy, testing it, and revising where the policy is inadequate. I never get approval fatigue because I don't do approvals.

You're right that attacks like this are hard to defend against, and that we're used to tool sprawl, and also that LLMs can do things like install tools we don't expect them to themselves! But there are ways to anticipate and mitigate attacks like this. Sandboxing tools (including contained and VMs) that support credential injection are genuinely useful for this.

Yes it’s messed up. I was thinking about pi agent and the extensions getting updates every three hours.

Sandboxing should be built in to your windows manager like: https://decentstat.es/posts/shai-halud-nix-housing/

This is seriously one of the reasons I see Rust as quite suspicious(with its 200-300 dependencies in all builds), unlike golang where you can actually create a full app with no dependencies.

This software should be rewritten until it has no dependencies other than the compiler. Like it should use no libraries at all other than crypto.

Also put it in rust ….

TIL! this is cool

Yeah it's not totally unlike people who consider learning that they've been hacked is worth more than $x and so, even if they're not into cryptocurrencies, they leave the equivalent of $x on their computer in a "software wallet", in an obvious place.

It's not a guaranteed you'll know you've been hacked, but it's still quite a flag to leave. And there are services doing the equivalent of what you posted, but for cryptocurrencies: they contact you through email, SMS, snail mail even, if they detect that "coins at address ..." ever move.

Canary token is great to tell you've been hacked.

There are ways to try to prevent this from happening in a first place. I wrote kow (keys-on-the-wire) to fix this. Apps only see placeholder tokens. The proxy injects real key from the vault into outbound requests for allowed hosts.

Author here. Apache-2.0. [1] https://keysonthewire.com [2] https://github.com/inflightsec/keys-on-the-wire

This is awesome, why did I just learn about this! Gonna add these all over my homelab

Hi abletonlive,

I appreciate the feedback. We work hard to make every release safe and secure. The attack was novel, and I'd love your feedback on how we can improve our ci/cd further.

https://docs.litellm.ai/blog/security-incident-oct-21-2026#h...

Déjà vu for both of us, huh?

Hi threecheese,

Pinning is a good idea.

So far, we believe it came from the github oauth cli tokens which are embedded in claude code tool responses - https://docs.litellm.ai/blog/security-incident-oct-21-2026#g...

It's something we're still investigating. I'll update my thread on this post, as we learn more!

Indeed, I don't think I'm upgrading ever again. I'll stuck with 1.04.2 for a year or two. Feels like a Russian rulette at this point.

And buried under the lede is that one of the compromised people was one of the organization owners. So it seems like the culture of insecurity comes from the top, which is unfortunate especially bc they keep acting like "this could've happened to anyone guys"

there's another one this summer that didn't get as much attention (or any at all) which allowed exfiltrating master provider keys with any authenticated user/virtual key

https://nvd.nist.gov/vuln/detail/cve-2026-84377

I looked at the code a year ago and decided I will always avoid it.

Original articles says it was likely a comprimised maintainer machine?

That's pretty hand wavy. What did y'all have CC doing that it had your GitHub CLI OAuth token in its context at all?

Stuff like this is why my coding agent is a coding agent not shell agent that basically own the machine. Mine has no tool for shell use narrow tools specific things like ripgrep, finding files, and deleting files that limit the paths to explicit workspace directory.

I take care of commits, push/pull, building wheels, and publishing wheels myself. It really doesn't take the much time and the coding agent frees up a lot of my time.

It's not a novel attack: it's novel negligence.

The best improvement would be to blame yourselves instead of Claude.

Amateur hour again.

What about adding a human in the loop? So the actual releases to the package registry would be done manually after reviewing the package?

If you can use uv, try the --exclude flag, e.g.:

    uv pip install -r requirements.txt --exclude excludes.txt

    pandas
    scikit-learn
    matplotlib

    numpy

EDIT: You can also configure this for uv directly in a pyproject.toml file:

    [tool.uv]
    exclude-dependencies = ["numpy"]

That just shifts the risk