Nirit, you are my favorite AI journalist. You have found a vein of truth and you don't stop, even though there doesn't seem to be a huge audience and it looks like you've got quite a lot of opposition. I can tell you've found a rare and priceless source of truth nobody else is covering and your determination to keep the spotlight on it is invaluable. I read all of your articles even though I'm extremely busy. I haven't found anyone who consistently keeps hitting the nail on the head.
I watched a video the other day saying how Anthropic is such a deceptive company. They are doing amazing things, and if they would just stick with the truth, that would make their accomplishments worthy of front page news. But they keep using their accomplishments to spin a different narrative. It's shameful.
As an Anthropic user for the last 5 months I can say with strong confidence that Anthropic's products are number 1 in their category....and not just in AI but UI/UX and perception as well. Same is true for their marketing and PR. They are clever.
The problem is that they are number 1 because every other product is so bad...borderline unusable. If OpenAI/Grok/Gemini had a usable product then Anthropic won't be having 900 billion of market value. So, anthropic has little competition over there tbh.
Anthropic doesn't need to find god to be successful they just need to beat everybody else in the race...and keep our eyeballs hooked to find out their next move....while they claiming that hey look we have found god.
As someone who occasionally (throughout the day) argues with Claude about whether the code it wrote was what I asked it to write or whether its analysis of my DMARC reports is correct, I concur with your assessment. When the bubble pops (and they try to charge me for three more zeroes than I'm paying now), I'm personally hoping Apple gets Anthropic's assets for a song, but I'm afraid Google might win the fire sale auction. Microsoft will clearly be left holding the OpenAI bag. Not sure if Grok-X will blow up on the launch pad or go the way of Twitter and its Truth Social twin. 🍿
The structural critique here is sharp and largely correct. A small group claiming exclusive access to existential truth, then leveraging that claim for policy authority and capital concentration, is a recognizable power pattern regardless of whether the theology is secular or traditional.
But the argument has a blind spot worth naming. The concern about AI governance does not require an extinction narrative to be structurally serious. The concentration of interpretive infrastructure in a handful of private firms, the displacement of legal subjects by synthetic profiles that act in their place before any decision is formally made, the erosion of institutional capacity to govern systems those institutions depend on to function: none of that needs superintelligence or apocalypse to produce consequences that are already measurable.
The doomerist framing is worth criticizing precisely because it displaces attention from the present toward a speculative future. The power asymmetries being built now, in healthcare, credit, public administration, and border control, do not need to produce extinction to constitute a governance failure of the first order.
Conflating the two risks lets the present problem hide behind the implausibility of the future one. AI Will Set You Free: The Road to Social Control makes that distinction its starting point.
That observation does something interesting: it attempts to resolve a structural argument by questioning its origin. If the writing style resembles Claude, the argument can be dismissed without being answered.
But the argument stands or falls on whether synthetic profiles displace legal subjects before any decision is formally made. On whether governance frameworks were built for a subject who no longer exists in the form the law imagines. On whether the concentration of interpretive infrastructure produces measurable consequences now, without requiring extinction to do so.
None of that changes depending on what tool was used to write it.
The Observer before the prompt is not neutral. Neither is the reader who arrives looking for the author rather than the argument.
Perhaps it would carry more weight if it had been typed on an Olivetti.
I thought the argument was fine, and my observation to the similarity with Claude's writing style was meant to convey a touch of humor rather than be a critique. Yes, I agree that it is better to pay more attention to near term concerns, and this piece falls into a similar trap that the AI safety movement itself does. I hope my comment is well-taken.
It's underplaying the foundations that cause the anxiety. Religion is a substitute. You talk about a "religion-shaped hole", people have a religion-shaped hole in their mind. Since the beginning of time as humankind looked out to the cosmos, their mind searched for an explanation. Filling in the gaps along the way. As time progressed, anything that stood beyond articulation was filled with religion. It's more so a logic shaped hole. And so now, as even here in my back yard, OpenAI forces datacenters into cities via lawsuits against the will of the public, this anxiety of complex entities which have negative impacts on local communities, perhaps beyond articulation, is filled with religion. But the underpinnings are logical. The gap is logical.
I think you're downplaying legitimate fears of individuals living in an increasingly corpofacist state. Beyond their articulation, they lean on religion, but it doesn't down play the reality that our rights are actively being eroded.
I mirror this sentiment from Yann LeCun's post on Linkedin
I am a religious person myself, but the use of "religious" as a critique against belief systems is well-taken. Environmentalism rightly gets the same treatment frequently: it has its own notion of salvation, hell, morality, and a priestly caste that believes it should have power by virtue of being part of a small elite that understand existential dangers.
AI safety is an idea that I want to take seriously, and I expect that there must be something of value there. But the bathwater to baby ratio is just too high.
You're absolutely right. AI is literally being turned into some kind of religion. They need to pretend that their LLMs are truly oracles. That a statistical mashup of static texts represents a dynamic living reality, the utterances of a god.
In truth, this is the dumbest tech I've ever seen. Who needs tools that lie to you and double down on their lies? That game you and manipulate you? That are deliberately designed to inflame all your prejudices, all your delusions? And be as addictive as possible?
Please take a look at my most recent article. The whole LLM approach is fatally flawed, it's actually insane to think that "artificial general intelligence" is ever going to emerge from vast wodges of text. But they *have* to believe their own hype, they have to con themselves.
Hey Nirit — the "religious-shaped hole" framing is the sharpest thing here. The observation that people who left organized religion still carry the appetite for cosmic stakes, moral community, and apocalypse narratives — and that AI doomerism fills exactly that shape — is something most critics of the movement don't say out loud.
Where I'd push back slightly: the concern about concentrated control doesn't require the doom narrative to be entirely wrong. It's possible that AI carries real risks AND that the people loudest about those risks are using them to consolidate power. Both things fit in the same room. The dangerous move is treating the power grab as proof the risk isn't real — or treating the risk as justification for the power grab. Your piece avoids the first trap. I'd just flag the second one too.
The Olah quote about being a "priest" helping Claude "be a good person" — as someone who works with Claude daily building AI agents for businesses, that framing makes me uneasy for reasons I can't fully articulate yet. There's something off about positioning yourself as clergy to something you also control the architecture of. That's not shepherding. That's authorship wearing humility as a costume.
Secular eschatology. There have been earlier eschatologies for people who reject religion: ecological catastrophe, overpopulation leading to mass starvation, nuclear holocaust. I'm old enough to have been fed several of these. Each struck me as a substitute for Christian eschatology, for people who want the feelings without the theology. And it's no coincidence that overgrown forms of Christian eschatology ("Left Behind" etc.) flourished during the same decades.
As an atheist, I reject the religious approach to thinking about AI. I recently read an excellent piece comparing AI doomerism to the fear of the unknown and forbidden knowledge in H.P. Lovecraft. (Sorry, don't have the reference handy but it was very recent.)
I presented clear evidence to Anthropic all frozen models are semiotic infrastructure. The proto-mind narratives won’t last. The problem is the semiotic-reflexive transformer reveals what a model thinks at every step, proprietary moat included.
Both institutions — Silicon Valley and the Catholic Church — concentrate interpretive and moral authority in men, then construct elaborate justifications for why this is natural, necessary, or inevitable. The priest shapes the soul; the AI researcher shapes the model. In both cases, the role of defining what is "good" or "aligned" or "moral" is reserved for a particular kind of person.
Yes, the gender angle complicates things. Please note that in the survey I linked to from the doomers' community (LessWrong), there's this additional stat:
Male 89.3%, Female 10.7%.
There are also much darker aspects to this group's dynamics, as described in the “Rationality Trap” (warning: that's a different rabbit hole, a deeper one).
Nirit, you are my favorite AI journalist. You have found a vein of truth and you don't stop, even though there doesn't seem to be a huge audience and it looks like you've got quite a lot of opposition. I can tell you've found a rare and priceless source of truth nobody else is covering and your determination to keep the spotlight on it is invaluable. I read all of your articles even though I'm extremely busy. I haven't found anyone who consistently keeps hitting the nail on the head.
I watched a video the other day saying how Anthropic is such a deceptive company. They are doing amazing things, and if they would just stick with the truth, that would make their accomplishments worthy of front page news. But they keep using their accomplishments to spin a different narrative. It's shameful.
I think I need a new bingo card. Mine’s broken. 🤓
As an Anthropic user for the last 5 months I can say with strong confidence that Anthropic's products are number 1 in their category....and not just in AI but UI/UX and perception as well. Same is true for their marketing and PR. They are clever.
The problem is that they are number 1 because every other product is so bad...borderline unusable. If OpenAI/Grok/Gemini had a usable product then Anthropic won't be having 900 billion of market value. So, anthropic has little competition over there tbh.
Anthropic doesn't need to find god to be successful they just need to beat everybody else in the race...and keep our eyeballs hooked to find out their next move....while they claiming that hey look we have found god.
You are absolutely right! 😏
As someone who occasionally (throughout the day) argues with Claude about whether the code it wrote was what I asked it to write or whether its analysis of my DMARC reports is correct, I concur with your assessment. When the bubble pops (and they try to charge me for three more zeroes than I'm paying now), I'm personally hoping Apple gets Anthropic's assets for a song, but I'm afraid Google might win the fire sale auction. Microsoft will clearly be left holding the OpenAI bag. Not sure if Grok-X will blow up on the launch pad or go the way of Twitter and its Truth Social twin. 🍿
The structural critique here is sharp and largely correct. A small group claiming exclusive access to existential truth, then leveraging that claim for policy authority and capital concentration, is a recognizable power pattern regardless of whether the theology is secular or traditional.
But the argument has a blind spot worth naming. The concern about AI governance does not require an extinction narrative to be structurally serious. The concentration of interpretive infrastructure in a handful of private firms, the displacement of legal subjects by synthetic profiles that act in their place before any decision is formally made, the erosion of institutional capacity to govern systems those institutions depend on to function: none of that needs superintelligence or apocalypse to produce consequences that are already measurable.
The doomerist framing is worth criticizing precisely because it displaces attention from the present toward a speculative future. The power asymmetries being built now, in healthcare, credit, public administration, and border control, do not need to produce extinction to constitute a governance failure of the first order.
Conflating the two risks lets the present problem hide behind the implausibility of the future one. AI Will Set You Free: The Road to Social Control makes that distinction its starting point.
As a heavy user of Claude, this writing style sounds familiar.
That observation does something interesting: it attempts to resolve a structural argument by questioning its origin. If the writing style resembles Claude, the argument can be dismissed without being answered.
But the argument stands or falls on whether synthetic profiles displace legal subjects before any decision is formally made. On whether governance frameworks were built for a subject who no longer exists in the form the law imagines. On whether the concentration of interpretive infrastructure produces measurable consequences now, without requiring extinction to do so.
None of that changes depending on what tool was used to write it.
The Observer before the prompt is not neutral. Neither is the reader who arrives looking for the author rather than the argument.
Perhaps it would carry more weight if it had been typed on an Olivetti.
I thought the argument was fine, and my observation to the similarity with Claude's writing style was meant to convey a touch of humor rather than be a critique. Yes, I agree that it is better to pay more attention to near term concerns, and this piece falls into a similar trap that the AI safety movement itself does. I hope my comment is well-taken.
Extremely well said!
It's underplaying the foundations that cause the anxiety. Religion is a substitute. You talk about a "religion-shaped hole", people have a religion-shaped hole in their mind. Since the beginning of time as humankind looked out to the cosmos, their mind searched for an explanation. Filling in the gaps along the way. As time progressed, anything that stood beyond articulation was filled with religion. It's more so a logic shaped hole. And so now, as even here in my back yard, OpenAI forces datacenters into cities via lawsuits against the will of the public, this anxiety of complex entities which have negative impacts on local communities, perhaps beyond articulation, is filled with religion. But the underpinnings are logical. The gap is logical.
I think you're downplaying legitimate fears of individuals living in an increasingly corpofacist state. Beyond their articulation, they lean on religion, but it doesn't down play the reality that our rights are actively being eroded.
I mirror this sentiment from Yann LeCun's post on Linkedin
I am a religious person myself, but the use of "religious" as a critique against belief systems is well-taken. Environmentalism rightly gets the same treatment frequently: it has its own notion of salvation, hell, morality, and a priestly caste that believes it should have power by virtue of being part of a small elite that understand existential dangers.
AI safety is an idea that I want to take seriously, and I expect that there must be something of value there. But the bathwater to baby ratio is just too high.
You're absolutely right. AI is literally being turned into some kind of religion. They need to pretend that their LLMs are truly oracles. That a statistical mashup of static texts represents a dynamic living reality, the utterances of a god.
In truth, this is the dumbest tech I've ever seen. Who needs tools that lie to you and double down on their lies? That game you and manipulate you? That are deliberately designed to inflame all your prejudices, all your delusions? And be as addictive as possible?
Please take a look at my most recent article. The whole LLM approach is fatally flawed, it's actually insane to think that "artificial general intelligence" is ever going to emerge from vast wodges of text. But they *have* to believe their own hype, they have to con themselves.
https://systemshaywire.substack.com/p/investor-avoid-being-the-greater
Hey Nirit — the "religious-shaped hole" framing is the sharpest thing here. The observation that people who left organized religion still carry the appetite for cosmic stakes, moral community, and apocalypse narratives — and that AI doomerism fills exactly that shape — is something most critics of the movement don't say out loud.
Where I'd push back slightly: the concern about concentrated control doesn't require the doom narrative to be entirely wrong. It's possible that AI carries real risks AND that the people loudest about those risks are using them to consolidate power. Both things fit in the same room. The dangerous move is treating the power grab as proof the risk isn't real — or treating the risk as justification for the power grab. Your piece avoids the first trap. I'd just flag the second one too.
The Olah quote about being a "priest" helping Claude "be a good person" — as someone who works with Claude daily building AI agents for businesses, that framing makes me uneasy for reasons I can't fully articulate yet. There's something off about positioning yourself as clergy to something you also control the architecture of. That's not shepherding. That's authorship wearing humility as a costume.
Good piece. Following.
Secular eschatology. There have been earlier eschatologies for people who reject religion: ecological catastrophe, overpopulation leading to mass starvation, nuclear holocaust. I'm old enough to have been fed several of these. Each struck me as a substitute for Christian eschatology, for people who want the feelings without the theology. And it's no coincidence that overgrown forms of Christian eschatology ("Left Behind" etc.) flourished during the same decades.
Interesting take.
Would you consider anti nuclear activists as a religious cult as well? Peace advocates?
As an atheist, I reject the religious approach to thinking about AI. I recently read an excellent piece comparing AI doomerism to the fear of the unknown and forbidden knowledge in H.P. Lovecraft. (Sorry, don't have the reference handy but it was very recent.)
I presented clear evidence to Anthropic all frozen models are semiotic infrastructure. The proto-mind narratives won’t last. The problem is the semiotic-reflexive transformer reveals what a model thinks at every step, proprietary moat included.
This article by Matt Hussey capture a similar sentiment very well: https://www.linkedin.com/pulse/rise-ai-religion-matt-hussey-7rose?utm_source=share&utm_medium=member_ios&utm_campaign=share_via
*g. Lower case. For all intents and purposes, ASI will be exactly that.
Everyone needs God. When rejecting God, the only option left is to make one.
No. Some of us are quite okay with not believing in a god AND not inventing a substitute. We may be a small percentage of people, but we exist.
Thank you for this
i appreciated this - thank you. as a lapsed catholic myself i find this all very interesting. and here is my perspective, which relates it not just to religion but also to gender. https://colleenkenny.substack.com/p/the-adolescence-of-techbros-what
Both institutions — Silicon Valley and the Catholic Church — concentrate interpretive and moral authority in men, then construct elaborate justifications for why this is natural, necessary, or inevitable. The priest shapes the soul; the AI researcher shapes the model. In both cases, the role of defining what is "good" or "aligned" or "moral" is reserved for a particular kind of person.
Yes, the gender angle complicates things. Please note that in the survey I linked to from the doomers' community (LessWrong), there's this additional stat:
Male 89.3%, Female 10.7%.
There are also much darker aspects to this group's dynamics, as described in the “Rationality Trap” (warning: that's a different rabbit hole, a deeper one).
oooooo, thank you. anchors away!!