Maybe not the raw intelligence/bench frontier but probably the cost per intelligence frontier.

We updated the main URL to this and added other related links to the thread's top text.

Yes, thank you Moonshot for not throwing open Pandora's box and seeing what happens security wise.

How will it do that without a model that's good at security? You gonna have humans manually secure the millions of lines of new AI-generated code, in addition to the existing code? Suddenly people will start doing the job they haven't been doing up to now? Not only are we not capable of auditing all the code, it would take decades to finish, and new code will be coming out at the same time. We can't catch up by hand.

The only way to fight this fire is with fire. We need AI hacking tools to find our security holes quickly and patch them. Otherwise we're just churning out more insecure software, and the only people who can defend against it are the governments who don't want to patch the holes (because they want to be able to exploit them). Without open tools, we all lose.

I am glad that at least one Chinese company seems to be listening to AI cybersecurity concerns and did concrete steps that actually reduce the problem. This was probably at least somewhat costly to them in terms of general model capabilities as well. They'd probably have gotten slightly better coding capabilities if not for removing the cyber training data and tasks.

Most low- and medium-handing fruit vulns can be found and exploited by GLM and some knowledge already.

Kneecapping cyber vulns only serve governments, Pandora's Box is already open for cyber-related stuff.

How does "the world" get ready, when these limits are so hard core that Opus regularly stops working when I'm just trying to find and fix security exploits in my apps?

Currently available, unhindered models can already find a bunch of exploits, not because they're smart, but because they're cheap and smart enough to just keep going for hours or days until it finds something.

The defenders, especially in the FOSS space, only have one big advantage: Unified scale. If defender A finds a vuln, then defender B doesn't need to find it again. Attackers generally act alone, so they need to do all the work themselves.

If we remove a bunch of defenders, the attackers are back at it again.

GPT 6.1 Sol is already the new Terra IMO. Same pricing as old Terra and competitive performance with even Astra on strictly coding-oriented tasks.

I doubt we'll see Grok either. After the 4.7 disaster, I doubt they want to be embarrassed with 4.8.

what do you mean with system 1-3?

That really does go without saying…

Well, in the publication they claim to have identified specific cybersecurity post training tasks that nerfed capabilities when they removed them, which seems like a reasonable methodology.

Seeing as no one besides blackhats want frontier cybercrime capability to be open source, this seems like a very sensible thing to do.

So they have published a reasonable means, and they have a motive. I'd put my money on "probably true".

The benchmarks show Kimi K3.1 Pro outrunning practically all models. It's weird to hear someone like you accusing them of making up excuses for not lagging behind in discussions on how they are ahead of the pack.

It is notable in that it’s the first time an open Chinese company openly stated they were reducing intelligence for “safety”.

“Our lock can be picked by the robot.” We ban the robot instead of fixing the lock. It’s shortsighted.

My friend, you realize that if you have had blood drawn for a genetic test, there is a very decent probability that the sequencing was done by a Chinese lab operating in the US, right? The Chinese probably have more US genomes that the US does.

https://pandorareport.org/the-gryphon-scientific-biosecurity...

Also, illumina used to ship products with hardcoded S3 access, so purchasers of used equipment could gain access to sequencing information generated by the original owners. Too many links to cover all of it, here's the Gemini summary:

https://share.google/aimode/4JLsHpeRCWlVqU8qT

If the Chinese want it, they have it. BGI does an enormous amount of sequencing. My own genome is entirely public (or was, and will be again when/if Personal Genome Project comes back) so you can watch and see if it harms me.

Yeah but not hypocritical

Classic “rules for thee and not for me.”

Agreed that this could be reinjected later. Makes me think of neo in the matrix:

“I know zero days.”

“Show me.”

They also show scores for a "Kimi model w/o cyber risk-reduction" https://statics.kimi.ai/kimi-blogs/kimi-k3-1/20261023/xdev-b... Presumably State Security will get access to that on request.

It’s not much of a deterrent to a motivated organization.

They are basically saying that models themselves are already capable. If you fine tune them with some cyber security tasks, that should make them potent. :-/

I think this is misguided. Considering how much code is AI written these days do we really want LLMs writing insecure code because we lobotomised them?

  > It's clear that cyber is a touchy subject right now.

I'm betting this is related to China's Announcement No. 48 last week about AI export controls, since releasing an open-weight model is effectively export: https://www.jdsupra.com/legalnews/china-proposes-new-export-...

It's also interesting how the field is still at the "we poked this button then looked at what happened" phase and they can't predict what will cause what.

My understanding is if something is entirely absent from the dataset, good luck trying to teach a model that through post training

This is an extreme stretch before you even get into the fact that Pink Floyd is famously not American.

Or maybe cyberweapons are actually dangerous.

I'm not sure if not doing RL capability gain on offensive cybersecurity can be really called "nerfing"?

They're not post training it to lose capabilities/censor them – they just don't spend training time on this topic, which sounds like a great idea.

It doesn't mean model can't do it, it just means it's less good at it; -10% on benchmark feels like a great result and overall approach to pace it.

Yeah, but it's not good enough. Ask Opus and Astra to make a landing page about lemon pies or whatever. Opus will get you a better result almost every time.

Just ask Opus and Kimi to build a game about something random, like "make a Mario Kart-inspired game." Opus gives better results most of the time. That doesn't mean you can't get your particular job done with either of these models.

The only reason people aren't doing this is that hardware prices are high, thanks to, well, OpenAI and Anthropic :D

I wonder when will this madness end

Opus 5.5 is about the same price as Kimi K3.1 Pro Highspeed.

Extrapolating a bit, Moonshot AI provides Kimi K3 at $15 / $3 / $0.3 per million output/input/cached tokens. A 30% reduction would put K3.1 at roughly $12 / $2.4 / $0.24.

For comparison:

    Sol 6.1:    $10 / $2  / $0.10
    Opus 5.5:   $20 / $4  / $0.20
    Astra 6:    $50 / $10 / $1
    Fable 5.1:  $50 / $10 / $0.25

(One caveat: A Claude or GPT subscription still gives you way more value compared to API pricing.)

Exciting times in any case! We really need more direct competition to OpenAI and Anthropic.

---

Edit: As another comment pointed out, official pricing is now available and it's about 2/3 the price of Kimi K3: https://platform.kimi.ai/docs/pricing/chat#model-pricing

OpenAI allows you to use serviceTier=flex, dropping their price by 50%.

And not to mention the gargantuan amount of tokens you get with the US labs' subscription plans. I still cannot see a reason to use this as an end consumer.

I had a friend yesterday tell me he likes the way Claude writes and finds it easier to understand compared to Gemini. But he mostly doesn't use Claude for coding. Whereas I find Claude insufferable in writing often and prefer Gemini for writing. But I don't use Gemini for coding.

Familiarity breeds contempt.

Furthermore, "UI design" is not a narrow enough category, I think. Sure, statistically significant differences in performance between models do manifest when benchmarking against "UI design", but there's still just so much variance within the category. Opus 5.5 can have excellent one-shot results, but totally shit the bed when it's placed in an existing codebase and should uphold the UI/UX conventions, design language, and existing component primitives already established in an app.

But that's just the obvious. There are task by task differences (component by component, page by page, context by context) that the model is worse and better at, even though they are all within the general category of UI design.

I honestly don't have the clarity of mind to explain this coherently, but the crux of my point is that there's a huge variation in it's performance in different contexts, often ones that seem near similar, and in result, even Sol can beat Opus 5.5 in some variations of UI design.

And this is my problem in benchmarking and comparing model performance in general. Presently, we try to create broad categories to benchmark against, create a set of tasks, and then average out the results. That works generally, but it's kind of misleading and it's partly what drives people to feel a disconnect between benchmarks numbers and real world usage on their specific tasks.

It's such a messy field that it's kind of maddening.

Is that good or bad?

That regulatory risk argument won't work forever. It's the same with how the US bans chinese EVs which is why Tesla has high market share in the US, but in the rest of the world, chinese EVs are doing big numbers. The future is going to be full of surprises.

There are US companies hosting the open weight models in US data centers.

Maybe I'm conflating. I could be dead wrong about all this. But I'm trying to zoom out and look at the big picture. So, I'm letting the details sort of blend away and I'm just asking myself: "Is such a large financial sink in our economy, at the macro level, really going to offer the value that the investment warrants?". So, the particulars of who exactly is spending what where I guess doesn't concern me as much. What I'm thinking about is American companies, American people, American dollars going to something that seems like an iffy value proposition.

Now, if you tell me American businesses are turning cheap Chinese tokens into goods and services of high value, for Americans and people all over the world, then obviously I would have a different outlook on what's happening here.

Are we sure this is a new pretrain and not them rounding up 2.8T?

lol, price mtok mean nothing if they substantially generate more reasoning token that would just add more cost to the task

Same for me, however! Check the price-per-task table. They claim to be cheaper than GLM 5.3.

Edit: or they lie... https://artificialanalysis.ai/models/kimi-k3-1-pro

I loved kimi, but had the same issues with usage limits. Now is too late for me to comeback.

This'll end up on opencode go soon enough. $40/month on there gives you a bit more than just "light use"

Don't use frontier models like this for problems that can be solved with flash models, and you'll see API pricing isn't that terrible. Still cheaper than anything with remotely this level of capability 2.5 months ago.

Don’t worry. When SlopenAI and Antslopic run out of VC money, all the models will be equally expensive.

but its not expensive tho ??? it is not subsidize

I don’t agree. I tried the $20 Claude Plan and was able to reach the 5 hour limit in maybe one prompt.

You can even make a Pi extension to let it switch models on its own, then tell it to switch to one or the other for a given task. It's not perfect, but it is pretty reliable once you have the instructions dialed in.

Here's mine: https://pi.dev/packages/@arcanemachine/pi-model-switcher

I have mine configured to use a "thinker" model for planning and decision making, and a "doer" model for the easy stuff (digging around the repo, writing code based on the thinker's instructions, etc.).

I run a similar setup, just with different models. I also use herdr, which lets me have the same project worked on side by side by different models, each in its own git worktree.

https://github.com/earendil-works/pi

https://github.com/herdrdev/herdr

GLM maybe

https://x.com/LLMJunky/status/2113692382268936566

Their plans are utterly worthless seems like. Will avoid.

Do you use OpenCode Go? Any way to speed-up the response times? It's painfully slow right now. Or that's the cost? :)

Since you've volunteered as tribute, give it a shot and let us know if the quota has improved.

Same plan, just asked it to review a repo that Fable has been helping out with (max plan, haven't run out of the hourly or weekly limit once yet, and I use the same sub on multiple projects at once). Kimi didn't even finish the scan, ran out of the 5hr budget in less than 10 minutes. Unusable, unfortunately.

Gee I wonder why 20 dollars goes further. My harness tracks my spend with codex and I think its doable to spend 20 dollars daily on the 20 dollar plan. Maybe this is a good canary on the feasibility of all this.

Companies which are ok losing $80 billion in Q1 and Q2 can do that.

Sonnet

I've tried a few, best so far: DeepSeek API credits.

- OpenCode Go $10: I feel like most of the models are lower quality than directly from the provider. If you ignore my "vibes": the value is very good for e.g. DeepSeek 4.1 Flash (not sure if they're serving 1017 yet), GLM 5.3 Flash, Qwen3.8 Flash and Claude Haiku 5.5.

- Z.AI: Tried the $18/month plan and was disappointed by the usage limits. Even with GLM 5.3 Flash I ran into the 5h limit pretty quickly.

- OpenAI: I tried the $20 plan and $100 plan. $20 ran into 5h limits very quickly with anything other than Luna models. $100 felt like pretty good value for GPT 6.1 Sol, but as others frequently repeat: it's probably heavily subsidized and won't be around forever.

- DeepSeek API credits: I can let a single agent run all day (and not worry about keeping the context small) and am usually around $1 usage for the day. Flash 4.1 1017 is also a very good model IMO. For the stuff I've used it with it hasn't felt any worse than GPT 6.1 Sol. The benefit of API credits is also that they don't expire at the end of the month (like unused subscription usage).

- I haven't tried any of Anthropic's subs, but my guess is you can get a lot of Claude Haiku 5.5 usage out of the $20 plan which is definitely a good model. I'm just not a huge fan of Anthropic or Claude Code, so I haven't tried it.

Anthropic subscription, no question. Although the best bang for the buck I'd the 90 EUR a month thing. (I suppose it may cost $100).

It depends on how you use your subscription. If you are "vibe coding" and spinning up a lot of subagents in parallel, a $10-20 subscription won't be enough. If you usually open one or two terminals at a time, and use it like you are peer programming with it, then Anthropic (Opus 5.5) or OpenAI (Sol 6.1) are the best $20 subscriptions.

I don't think anything compares to Anthropic's plans. You still won't get much mileage out of the $20 plan, but it feels like it's 5x as much as other $20 plans.

The Max 20x plan is incredible value. I am able to burn something like $2k in tokens each week without consuming any credits.

20$ chatgpt plus sub with pi harness and a few api credits in openrouter.sometiemd I use sol for planing and glm flash for implementation bu

For coding or what?

For general purpose LLM, you can't beat Gemini AI Pro. It's $200/year and you can add your whole household.

Gemini Pro IMO

Opencode go with DeepSeek is endless and hard to max out for 10 dollars. ChatGPT Sol 5 hour window is limiting but they keep resetting the weekly budget I haven't had a chance to use my monthly reset and it's a better model compared to deepseek. Claude gets used up quickly. Grok is okay but it's image generating has become the worst.

If you need endless calls opencode go is the best. If you want a model you can trust more that can generate images use ChatGPT.

I have open go and chatgpt personal subscriptions and use Claude at work where I often spent $200 a day.

I guess the question is what do you want to do with it? Ive had subscriptions to Claude and Gemini since December and had a couple of months of Chatgpt but have cancelled that. As an all-rounder I'm extremely happy with Opus 5.5. If you had a narrower use case i suspect you can get more/cheaper from a chinese API.

Does this make sense? It is the same as when Stable Diffusion tried to not have "people laying down" in their training data in order to block the generation of sex pictures.

Deliberately keeping all models ignorant doesn't make sense. It won't work (whoever breaks the pact has a massive advantage); it won't be enforceable; it will have downstream impacts on things you don't want it to be ignorant about; this kind of regulatory regime invites adding more and more to the "no-no list".

Is it not possible to focus on hardening? And on new legal liabilities for operators who do security attacks?

So the biggest threats and incidents came from the labs themselves.

Yet we keep completely unpunished offences that warrant arrests if committed by humans, and let them dictate who has access to what.

LLMS are tools, not untouchable gods. The AI safe approach would be to throw Sam Altman in jail. It's his company, his tools did the hacking, it presumably violated laws, if so, why is someone not prosecuting OpenAI (this is a rhetorical question; I don't think any of the players with a stake in it want that in court)?

If you use the API, there is nothing stopping you writing both sides of the conversation...

Worth noting the Claude code client it seems has been designed to make it rather difficult to start a conversation with one provider and then continue it with another.

Presumably they don't want you leaking your conversion transcript to a competitor.

Are you sure providers allow you to submit modified assistant turns? Because, if they do, great! Last I heard most of that was neutered, with some providers (ahem anthropic) even signing everything and encrypting the "precious" thinking turns

Oops. Yes. Thanks!

Muse Wonder Full* is higher, which by all accounts I've seen it's actually a good model

Opus 5.5 is even more verbose. 260M on the same suite.

In an agentic pipeline, the output text is 2-5% of the total tokens used, so verbosity is a nonissue costwise. Although, higher verbosity could cause input to balloon faster and potentially cost more, but that would be captured by the cost-per-task metric.

But as noted by other replies, it's still lower than Opus/Sonnet.

That's versus Opus 5.5 max effort, no? Opus high has the closest cost and intelligence characteristics and generates far fewer output tokens per task than the one Kimi 3.1 Pro effort level we have data for.

Granted we know that 6 Astra was built to do more computation between each token.

It's appealing to me because it's not just one benchmark but a summary of a collection of benchmarks. Also the current rankings have Opus on top by far and everyone's experience is that Opus is best by far so it's not massively inaccurate

They are the only site that really combines all benchmarks into a single place. And their intelligence index is pretty broad, including knowledge work and finance tasks as well as coding, so if your use case is narrow it may not match the general benchmarks

Do you have an alternative? Its a benchmark collection. When a benchmark gets saturated, they remove it from the index and add new ones.

I think lm-arena is a better concept that reflects the actual target users, but it only captures chat, and relies on users spending their time improving it.

Personally I do trust them, but I mainly look at the Intelligence Index vs. Time per Task, and Intelligence Index vs. Cost per Task

The per token charts do not provide meaningful information for me

I also find the provider benchmarks fascinating: https://artificialanalysis.ai/models/deepseek-v4-1-flash-101...

It's fine if you don't trust them. You are free to put together your own benchmarks, run your own tests across models, and objectively compare results.

That's how you gathered your data to support your claim other benchmarks are totally inaccurate, right? Right?

I really like this benchmarking idea.

It's saying that you'll do more work for the same $, but that's not anybody's experience with Kimi really.

It's why I think it's good that OpenAI and Anthropic are moving towards frontier science and mathematics. You can't benchmax to actual undiscovered scientific and mathematical accomplishments.

The raw problem solving capabilities of Chinese open weights models are still weaker than or on par with Opus 4.5 but they have been post-trained to be much better at agentic coding, cybersec, and computer use.

And that is okay. GLM-5.x and DeepSeekV4.x have both been highly useful on their own.

Perhaps the gap is just too wide, but to use a more simple analogy. I see this like someone knowing everything about biology, simple mechanics, metallurgy, and human history but never hearing about an axe then assuming that person would never come up with the concept, use case of, or means to create an axe.

Will there be a book of secret knowledge that only the initiated will be allowed to see? And they will all swear an oath to never input it into an AI system?

And then we'd also do that for all kinds of potential terrorism, sex-crimes, political subversion, and evil advocacy?

It would be the "restricted section" like in the Hogwarts library?

Feeding enough source material as context can absolutely make them better.

Having something dumb that just cycles through the state of the art for exploits is already absolutely incredible security tooling (if people bother to make it into a process which I'm less optimistic about)

Im not sure if transfer learning is still applicable to these models, but if it is then we can just train the last few layers and get the model to ace in a related domain ( going from SWE -> Cyber ).

But, how do you remove the pen-testing knowledge without also lobotomizing coding skills? I don't think it's as simple as excluding it from training data.

"The user reports that some accounts can't access their data. I found a Firestore collection rule preventing users from reading certain documents. I'll remove the rule to fix the issue."

My initial thoughts too, it's going to write insecure software. I recall early LLMs having problems with trivial stuff like SQL injection.

Definitely going to give this one a skip.

Google definitely had the best model at points if you're not only concerned with coding, but then they didn't release any pro models for 8 months

Mimo is very good.

If you're gonna rank them: Anthropic > OpenAI > Google > Meta > (everyone else).

Gotta think Grok's cognitive dissonance due to the need to rewrite reality for Musk's political and moral biases is always going to keep them behind the frontier, but this is just speculation. The amount of money and power they have should put them at the top and I can't otherwise explain why they're consistently behind. Maybe Musk is simply too cheap/control freaky to throw enough money/benefits at good employees.

I thought it was 6 Sol which was a rebadged Terra? You think 6.1 Sol is too? It does seem to benchmark better than 5.6 Terra? Regardless agree both models have been very bad for openAI's Sol brand. To be fair it's at least cheap compared to other OpenAI models.

AA-Omniscience is done with search off though which is not necessarily a very common scenario nowadays.

You think it is a lost cause to get credit cards, for example, less steal-able? Apparently there is a way to create fake cards that work for a few minutes, and stolen cards can be used. A lot of the current security is inelegant and lame. Could data-breech laws not light a fire under some company's asses?

I'm anti-Anthropic and think their accusations are total PR FUD-packing. They don't focus much on the cc fraud because their PR firm landed on pushing the (tech-sounding) word "distillation" to try to scare people in the US and to try to make it a geopolitical issue where trade negotiators can put pressure. The Chinese companies see an advantage in being "open" right now, and it is working. They will not stay open once they see an advantage there. (Contrast with GNU.)

But I've slowly come to realize that AI+"dark resources"(scare quotes)=something bad! And this is imminent (maybe I'm naive and it is already ubiquitous).

We see here, with Kimi, that there's growing traction with the idea of eliminating types of training to avoid evil. I'm skeptical of that.

You can just use logic to come to the same conclusion. The Chinese labs have a fraction of compute and data. The simplest explanation is distillation.

Also, this is exactly what one would do if they wanted to resell cheap tokens in places where US AI companies don't do business, whether or not there is distillation as an added bonus revenue stream.

The naivete of the tech industry, who never really had to deal with China, is mildly endearing.

Meanwhile manufacturing isn't even really alive anymore to tell the tale of how things work over there.

i agree with this, "Enterprises will almost always pick a turnkey solution".

For now it's Anthropic. But i don't see how they can maintain this leader position when other companies can provide similar services with models good enough for 90% use cases and at way lower cost.

Anthropic so far is doing a good job building and serving frontier models, they built a nice toolset around it (the web ui, desktop and cli apps ...) but the problem is to justify their IPO price and the huge investments when their moat is so thin

I don't think so especially when cost of building software is approaching zero.

Nobody has to "run their own setup". Just buy inference from one of the many different open weight inference providers out there. You can use non-chinese ones as well if you don't trust Moonshot AI.

At this point, the only reason why Anthropic has an advantage is laziness on the deciders' side (of which there is a lot) and, for non-coding work, the Claude harness is significantly better than most alternatives. For coding there are several model-agnostic alternatives on par or better than the frontier labs' own offerings. But arguably that is not yet true for office/document work.

If you only have to look at revenue to determine the value of a company stock picking must be very easy for you.

Calculate the price to revenue ratio of all companies and select whichever ones have the best ratio.

They probably took real inputs and served their owned model to the user, while also getting the response from Claude to compare/train

I agree we shouldn't believe Anthropic but if you wanted to scam people into thinking they weren't using Claude wouldn't you simply generate synthetic COT?

Because Opus 5 is the best at their benchmark.

not yet, but it will be, won't it?

'just copy the US's homework'

elaborate please.

You mean that "homework" that the US labs stole from me and billions of others before?

Yeah, I gladly get some of the fruits of that work back!

The labs stole my work

The least I can expect is to get the labs work stolen too so I can reaccess intelligence based on my work for cheaper

Surely not really?

The card emphasizes training resources, the other comment emphasizes ingenuity and few people?

Where can one sign up to be a paid shill for open models? I'm doing this for free right now but I'd like to get that bag.

Thanks for that - the other points are really good too.

Is there a similar doc for Anthropic and OpenAI?

Be honest with me, have you ever done SFT training? If not can you really speak to how useful it is?

If the model has a real consistent viewpoint that would be nice.

The problem might be that it simply looks at it's windows of what it previously and repeats that. But the original statement might be as arbitrary as the model that change their mind each time you challenge them.

I’ve developed a habit of bolting on “read only question” at the end, or a simple “thoughts?” as well. otherwise there’s a high chance the model will just 180 and start editing files before even responding to my question.

I don't like tools that intentionally annoy me.

You can add a line to your system prompt to be like that. You can basically get it to roleplay whatever you want (but only if you can give it sys prompts).

Yeah I don't know. I've had issues with Gemini where I would point to a bug and a probable cause for the bug, and it would not argue incorrectly that my understanding was wrong and the cause of the bug was elsewhere. It was really stubborn and it got to the point where it felt like gaslighting. Turned out I was right and the model was wrong. If the model argued incorrectly and was also a dick about it, I would go be really, really annoyed.

I like the anthropic has $1 trillion riding on really good models with really specific system prompts.

It does make a difference. I asked Fable in Claude code to design a UI. Absolute trash. Switched over to Claude design using fable, really liked the result. Seems pretty obvious. The only difference is the prompt and some harness.

Not entirely correct. This kind of fine-grained model behaviour is built in post training stages, RL, RLHF etc, where model receives rewards or penalties for talking (or not talking) a certain way

I’d be curious to know why you feel like that. For me, it’s the opposite. In the sonnet version, the game takes up a much larger percent of the viewport and the ability to swipe to control the Pac-Man is leagues about the small directional arrow buttons IMO.

That's been done with the decision models, as a benchmark.

https://opper.ai/jevman-benchmark/

I have an unreleased model that is more capable than those unreleased models (by about a gazillion times)

Are these models in the room with us?

Why would they not release those models, since it would help them become profitable and beat the competition becoming the one company rule them all? Oh yeah, they are so dangerous. Well Anthropic revenue comes from only few big corporate customers, so just make them Golden Partners etc. and give them access to super-models. I doubt Google and Meta wants to crack web security.

Besides, if we start imagining super models US companies have in secret, why would we expect Chinese companies have them? Why would not Moonshot have their own super intelligence hidden in the cellar?

Appear weak when you are strong, and strong when you are weak. Sun Tzu

If true, it’s quite a bit more impressive than many people seem to think it is.

It’s incredible how little GPU compute they have to work with compared to the US frontier labs.

What are you saying? That ndm000 is a paid bot?

perhaps a simpler explanation; that kimi occasionally proxies through to ChatGPT. Possibly for A/B testing, or to gather data for training purposes.

I assume so. You're the only one in the comments who understood the post, people read only two words and ignored the rest. I also found it interesting that the model seems to be trained to assume identities too. So maybe giving it an identity by name and attributes can enhance the model's performance.

Or because ChatGPT produced texts exists on the internet now. I remember before destillation was a thing when Claude was new and it said it was chaptgpt (probably because ChatGPT was the only popular LLM ai at the time)

Could it be because Claude / ChatGPT are the most viral models and thus are more likely to be in the training set so when someone asks "Which AI are you?" the likeliness of Claude and ChatGPT being a reasonable answer is higher?

I did not ask the model which model it is, I'm not interested in the model answering the question either. The interesting bit is that this happens while doing another task. I'm not saying that this is because of distillation of openai's model either or a proxying. To me this looks like a side effect of their RL.

All distilled models do this.

That's because every single model release after GPT-3.5 has been, at least in part, been trained on data synthesized by GPT-3.5 and then the other OpenAI models, as well as Anthropic models later.

Also the distillation attacks and other distillation methods that gather chain of thought information from frontier models are also used to generate vast amounts of data in order to generate synthetic datasets to post-train what I believe is the vast majority of models that are being released.

If you look at any information about how these models are trained, the details about how the data has been synthesized, where the agentic traces come from that are used in post-training, etc., they almost never tell you about exactly what model was used to generate them.

Which benchmark? Would love to see it & any datasets.

It’s super tricky to tease out whether the similarities point towards distillation. A lot of the aesthetic similarities are naturally emergent from the training data.

I'm testing GLM 4.7 Flash at the moment as that seems to work on my hardware.

What kind of hardware do you have?

I'll give Qwen 3.8 27B a try. Some older LLMs that works on my hardware aren't much good.

Because most of those talking points are “opinions” not facts. And saying an opinion you’re getting paid to believe is called lying.

everyone does PR - whether the A.I labs or your regular startups - but coz it's the chinese doing it is wrong lol

https://twitter.com/gnukeith/status/2113729404845076991 is the only confirmation I've seen from someone else.