Rendered at 22:28:56 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
kilpikaarna 14 hours ago [-]
> I think it is fundamentally a crisis of trust. I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.
[...]
> I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer. I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world. That is totally on us, and I think it’s the criticism you should be making, instead of all this stuff about messaging and marketing.
> We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that.
Turns out "winning back trust" doesn't have anything to do with any actual concerns people may have re: employment, electricity prices, stock market bubble, intellectual property, scams, cybersecurity, environmental issues etc. Rather we'll just do all of that even harder and the miracles ("curing cancer", lol) we've so far failed to deliver are bound to arrive in short order!
traceroute66 12 hours ago [-]
> The thing that will work is actually curing cancer.
This is honestly hilarious. Dario must think people are stupid.
First, cancer research charities are some of the most well funded on this planet. They all obtain donations on the basis of "together, we will cure cancer" messaging. They all fund the best science they can.
Second, the human body is a complicated thing. You can feed your fancy LLM as many textbooks and academic papers as you like, but the reality on the hospital ward will always be different. Why do you think student doctors have to spend so many years "doing the rounds" Dario ? They are all academically smart, they are all capable of memorizing text books ... but there is no substitute for seeing and doing the reality.
I barely trust Claude to write code, let alone find a cure to cancer.
tptacek 14 minutes ago [-]
I don't think the idea is that a frontier model "cures cancer" so much as that it accelerates research that humans do, getting us to answers faster. That's a hard idea to dismiss after what's happening in mathematics, and I believe life sciences people take this very seriously, even if message board nerds like us don't.
eloisant 11 hours ago [-]
Besides cancer isn't really "a disease" but rather "a family of diseases" that can vary a lot. Even breast cancer can be one of many different cancers.
It's like saying "we'll cure infectious diseases"
traceroute66 11 hours ago [-]
> It's like saying "we'll cure infectious diseases"
Don't worry, I'm sure that's his next blog post. Right after Claude has finished solving famine and poverty. ;)
kennywinker 6 hours ago [-]
The solution to famine and poverty is not exactly a mystery.
We have more than enough food on the planet for everyone, and we have the ability to deliver bombs or soldiers or amazon packages to anywhere in the world in days or hours. Combine excess food with amazon or army level logistics, and famine is gone.
What people mean when they say “solve famine” is can we make people not starve without actually doing or changing anything - which is of course impossible.
delis-thumbs-7e 4 hours ago [-]
> Combine excess food with amazon or army level logistics, and famine is gone.
We do this. Have you ever heard of UN World Food Programme? $14B funding in 2022 (since then dropped drastically, guess why).
Has it solved famine? No. Would it solve famine if it got the funding it needs? Unlikely.
Because you cannot just spend billions of dollars to send amazin food parcels across the world and expect that to fix climate change, draught, poverty, colonialism, racism, illiteracy, lack of education, and propably 100 other issues all contributing to the fact that we still have not end hunger. And no, AI won’t solve it either.
152/757 = the world food program helps ~20% of people who are food insecure. That's well in the ballpark of the estimated 15% spend.
If we spend that extra $93b, we'd be well within the range to actually stop world hunger.
Yes, "climate change, draught, poverty, colonialism, racism, illiteracy, lack of education" are still issues. No, you don't have to solve those problems to feed people. Don't let perfect be the enemy of good.
nickff 57 minutes ago [-]
> "If we spend that extra $93b, we'd be well within the range to actually stop world hunger."
Almost all recent famines have been caused (either deliberately or incidentally) by government policies, not lack of money. The UN spending 93 BB may (somewhat) improve the situation, but is unlikely to end food insecurity.
Eliminating corrupt leaders (i.e. Kim Jong Un, Vladimir Putin, etc.), and poor regulation (e.g. Bombay Fodder and Grain Control Act, 1939) would be much more effective.
Assuming everything they said is true, how on earth is that a self-own?
You know that saving millions of lives every year is better than NOT saving millions of lives, right? Just because something doesn't solve the problem 100% doesn't mean it's not VERY VERY good.
Applejinx 9 hours ago [-]
Technically, killing everybody is one way of solving famine and poverty.
How weird that all those humans seem not to trust the logic of their betters ;P
subygan 9 hours ago [-]
you're right. it coincidentally solves cancer and infectious diseases as well.
the ultimate utopia
lesuorac 8 hours ago [-]
Look, it's really simple we'll have 3 laws that the robots follow and nothing will go wrong.
kennywinker 6 hours ago [-]
Except instead of bots that follow asimov’s three laws we’re getting bots that follow the ferengi rules of acquisition.
fnord123 11 hours ago [-]
We will cure virus.
FeteCommuniste 11 hours ago [-]
We will cure carpet. We will cure desk. We will cure lamp.
kridsdale1 3 hours ago [-]
That really got out of hand. You may want to lay low.
andrekandre 2 hours ago [-]
wait till they get the 'war on carpet' on, no one will be spared...
As for "curing cancer", I suspect that Anthropic or OpenAI will use their high status/reputation to launder the existing solutions (germline, p53, enhanced DNA damage repair pathways, higher tumor-suppressor-to-oncogene copy number, more robust tumor-suppressor networks, more self-anti-cancer apoptosis, CD47 suppression, etc) that society has refused to accept. They might have enough reputation to get people to look at these ideas. It won't cure people with stage 5 turbo-cancer a.k.a the walking dead, but hopefully we won't be moving the goal posts _that_ much on them.
RandomLensman 10 hours ago [-]
How would you address the non-interchangeable nature of synonymous codons? Do we know the right replacements?
kanzure 10 hours ago [-]
Well, first, I would have to check and see if the synoynmous codons are important for virus reproduction. Second, this technique requires (AFAIK) lots of cell line viability testing and lots of tweaking. I don't think it's as simple as code swapping all the codons and then all the biology magically works the right way, biology is rarely that simple. Requires effort.
RandomLensman 10 hours ago [-]
This sounds highly speculative then, i.e., the biology could never work out, no?
kanzure 9 hours ago [-]
It depends on what you consider to be speculative. It's certainly not a product available to the mass consumer market, if that's what you mean. But that's also kind of an uninteresting fact, right?
Speculative whether it could be done with humans without severe side effects that would make it useless/pointless.
Is it a product available to humans at all?
colingauvin 9 hours ago [-]
Ignoring the impracticality of trying to deploy this, codon bias exists for a reason, it's a translation-regulation mechanism and you'd wind up having to do a massive amount of protein engineering to not mess up everything.
jacquesm 10 hours ago [-]
What could possibly go wrong...
fwip 15 minutes ago [-]
Honestly, the most likely thing is lots and lots of dead fetuses due to developmental abnormalities. I'd be surprised if it ever got to the point of resulting in a dead infant.
datakan 10 hours ago [-]
I've seen this episode before
kennywinker 6 hours ago [-]
Oo is this the one where barclay becomes a spider?!
andrekandre 2 hours ago [-]
could be, or the one where dr pulaski becomes an old lady...
dcre 6 hours ago [-]
I have seen people suggest we will figure out how to prevent all infectious disease.
overgard 3 hours ago [-]
Not to mention, hasn't the pharmaceutical industry taught us that curing diseases is a bad capitalist strategy? After all, why cure something if you can simply treat it, expensively, indefinitely.
alphazard 11 hours ago [-]
"Cancer isn't a single disease" is often shouted from the Dunning-Kruger peak of cancer research. I think it was popularized by an explainer video on Youtube, and now it's being memed. The people who think this way want to legitimize funding a particular research niche that they benefit from.
A more sophisticated view is that cancer (one idea) is an problem of rates (like most things in biology) and there is a tumor burden per unit time from whatever causes vs. the immune system's capacity to detect and destroy them per unit time. As humans age, the balance tips in one direction. If the balance is too far off for too long, tumors accumulate, doctors take notice, and assign whatever labels to the condition.
Looking into particular causes of tumors (and calling each cause its own cancer) can create a vast research program that employs a lot of people, since there are so many mutations that lead to tumors. But interrupting the development of a particular mutation is not going to have the same society-changing effect that solving the fundamental issue of rates would.
preg_match 8 hours ago [-]
No, cancer is actually many diseases. They hold some commonalities, but it’s not one thing. That’s why chemotherapy can straight cure some cancers and do nothing for others. That’s also why we have hundreds of chemotherapy drugs.
Also, not even all cancers have tumors. You have solid tumor oncologists and then oncologists who specialize in blood and lymphatic cancers, “liquid cancers”.
In addition, cancer does not arise from one mutation. It’s a set of mutations that allow cancer to occur. You need mutations for growth, mutations for preventing cellular death, mutations for stealing energy, mutations for hiding from the immune system. Your body, right now, has probably millions of cells that could be candidates for cancer. They either kill themselves or your immune system nukes them. You really need the stars to align on this stuff to get actual cancer.
And then, it’s not like the cancer stops mutating. Cancer becomes resistant to chemotherapy through mutations. Cancer is “smart”, it will even temporary slow its own growth (metabolism) to intake less chemotherapy through treatment.
alphazard 7 hours ago [-]
I should have expected this level of discourse from 2026 HN.
Here's another way of framing things that might encourage better discussion:
Suppose in a few years, it was announced that an LLM did discover a cure for a cancer. What approach would you expect it to have taken?
1. Developing an intervention for every possible mutation that leads to loss of cellular identity, and uncontrolled growth.
2. Preserving immune function later into life.
3. Preserving genetic and epigenetic accuracy later into life.
Maybe it will be 1. But I would bet on 2 or 3. Which both assume that by reducing the difference between rates, the phenomenon of cancer could be reduced or eliminated.
>> "Cancer isn't a single disease" is often shouted from the Dunning-Kruger peak of cancer research.
Literally first phrase on Cancer Wikipedia entry.... [1]
"Cancer is a group of diseases involving uncontrolled cell growth typically resulting in tumors with the potential to invade or spread to other parts of the body...."
And this business of rates is also why getting the immune system—such as via personalized vaccine—to recognize a tumor is functionally "curing cancer". The problem is it isn't one disease in the sense that different tissues, cell types, and systems are affected leading to different parameters in the design and delivery of said vaccines.
datsci_est_2015 9 hours ago [-]
Is there something like a meta-Dunning-Kruger, where someone invokes Dunning-Kruger in a way that itself is a potential Dunning-Kruger phenomenon?
I'm inclined to believe that the thing - that everyone intelligent says is complicated - is actually complicated. I would be surprised if the thing - that everyone intelligent says is complicated - is actually simple.
This is almost like an application of the conspiracy theory gambit where you try to find out how many people would have to be "in on it" in order to disprove the theory's likelihood. Like flat earth.
bigfishrunning 5 hours ago [-]
People are constantly Dunning-Krugering the Dunning-Kruger effect
bonsai_spool 10 hours ago [-]
[dead]
siva7 6 hours ago [-]
Claude cured a deaf person. Just look up news. All this negativity on HN is depressing..
GPerson 6 hours ago [-]
Do you believe Anthropic is going to cure almost all human diseases including cancer in 5-10 years?
5 hours ago [-]
jacquesm 11 hours ago [-]
> Dario must think people are stupid.
All of those guys think people are stupid. That theme has been repeating over and over again.
CamperBob2 6 hours ago [-]
Donald Trump still maintains a 39% approval rating among the electorate. It turns out that a lot of people are indeed quite stupid.
Nixon was 24% when he resigned, which is amusingly not actually the lowest, as Truman seems to have twice achieved a spot rating of 22, though Nixon remains the lowest rating for a full year. I wonder what Truman did to be less liked than Nixon. Those were both well after dropping bombs on Japan and before the first H-bomb test.
andrekandre 2 hours ago [-]
> quite stupid
years of misinformation and confirmation bias + social bubbles will do that
Go look at Putin's numbers :) None of those number as real.
dmd 10 hours ago [-]
To be fair, the vast majority of people are stupid, in one way or another.
Or do you believe that you are smart in every way there is to be smart?
GPerson 9 hours ago [-]
You’re just inventing an imaginary interpretation of what that guy said and pompously dismissing it.
Razengan 10 hours ago [-]
You just have to find the correct prompts for them, and keep them away from getting mad at any of the ACTUALLY evil shit in the world, like weapons manufacturers, 24/7 global surveillance, perpetual debt and wage slavery, factory farms, constantly dwindling personal freedoms, and a handful of people owning all of it and not allowing anything that empowers the common peasant.
bko 10 hours ago [-]
> First, cancer research charities are some of the most well funded on this planet
Maybe I'm overly cynical, but I think charities exist to exist. They don't have a strong incentive to actually deliver on their mission. Everyone at the org may be fully bought it and obviously want to cure cancer, but as an organization it would fail to deliver on the promise. Like most organisms, their actual purpose is to continue to be influential, grow and continue to exist.
I think cancer will be solved by some group that could make a lot of money solving cancer. The probability of any one path working is very low so the payout would have to be very large for anyone to be willing to pursue.
shaganer 9 hours ago [-]
Charity doesn't stop at the cure, but continues to shift with progress.
If a big public charity is able to cure a cancer type (eg. lung cancer), then it's next aim would be to try cure another cancer type. Let's just say they are somehow able to cure all the cancers, and therefore they cured cancer, then there's distribution of the cure, with a focus on 1st world but then it would have to go to developing nations too. Along the way, there may be issues that arise in recovery from the cure, so money towards helping survivors there. Shift goal, shift goal, shift distribution/priority of donations, on and on and on. For the proper charities that do want to meet their goals, something like this is what they'd do.
jacquesm 9 hours ago [-]
Has that model been proven to work for other diseases?
fragmede 8 hours ago [-]
I mean, we were making progress with measles but then RFK happened.
The policies around measles vaccines have shifted a little since RFK became HHS Secretary, but not in a material way. Generally the messaging has focused on putting measles, mumps, and rubella in individual vaccines rather than combined, but at no point has the federal government recommended not to get vaccinated. That, combined with the Canadian data, implies that RFK or other changes to US health policy are not likely to be the cause of the measles outbreaks.
xienze 4 hours ago [-]
The elephant in the room that a lot of people don't want to talk about is mass migration from countries where measles is still a major thing... surely that contributes a bit.
Loughla 10 hours ago [-]
I'm the opposite. I think publicly funded organizations will be the ones to come up with the cure. My reasoning is this; any company that is going to find a cure for cancer probably already has divisions that work on treating cancer. If the cure isn't more profitable than the treatment, there is every incentive to suppress the cure.
demibabs 9 hours ago [-]
I don’t think it works like that.
If you can find and patent a cure to cancer, you can charge whatever you want for it, at least in the US. I don’t think profit would be an issue.
And even if ending chemotherapy would discourage the incumbents, smaller companies would love the opportunity to kill them.
For example, Polaroid didn’t want to embrace digital cameras since they made tons of money selling physical film. But all the smaller camera companies didn’t care, since they weren’t benefiting from that and wanted to just take Polaroid’s market share. And that’s why digital cameras took off regardless.
datsci_est_2015 8 hours ago [-]
It's likely a mixed bag. If you're diligent about which organizations you choose to support, you're more likely to have a positive impact.
If you want a simpler example that doesn't involve biotech research, I would just look at the diversity of animal welfare organizations, and their relative quality. Some of the best animal welfare organizations in terms of impact are local rescues run by (in my experience) spouses of wealthy career-people (executives, lawyers, etc.) who have a ton of free time and love animals. But then there are also plenty of animal welfare organizations with questionable motives, and questionable spending patterns - bordering on fraud and embezzlement, if not outright fraud and embezzlement.
magicalist 10 hours ago [-]
> Maybe I'm overly cynical, but I think charities exist to exist. They don't have a strong incentive to actually deliver on their mission. Everyone at the org may be fully bought it and obviously want to cure cancer, but as an organization it would fail to deliver on the promise. Like most organisms, their actual purpose is to continue to be influential, grow and continue to exist.
I mean, yes, "I haven't looked into the specifics, but working from the first principles I came up with in 3 seconds, cancer research charities are in fact cancer on society" does come off a bit reductively cynical.
shafyy 10 hours ago [-]
A lot of basic research (including medicinal) is funded by governments who are not in it to make a lot of money. Sure, medical companies who then develop drugs based on this decades of knowledge and research are profit-oriented, but that's only one part of the equation.
mr_toad 6 hours ago [-]
> Second, the human body is a complicated thing. You can feed your fancy LLM as many textbooks and academic papers as you like, but the reality on the hospital ward will always be different.
And they’ll be using detailed medical records. Personalized medicine and the use of ML in medicine has been around longer than LLMs (and will continue regardless). There is a lot more data available than just studies and textbooks.
This is why this hype is so annoying - they have to make claims and goals like this to justify the current level of investment. It couldn't be like "ok, we have this cool tool that can sorta code and do computer stuff and automate things" and be a $50 billion dollar company, no, we have to shoot much, much higher, so high that I think the only inevitability is this all collapses in on itself in a financial black hole, which is a shame.
8 hours ago [-]
skybrian 9 hours ago [-]
“Cure cancer” is vague shorthand. A more charitable interpretation would be something like “provides important assistance to research scientists who come up with new cures to most kinds of cancer.”
iAMkenough 6 hours ago [-]
No, I think Dario wants to actually “own” the cure to cancer.
SpicyLemonZest 6 hours ago [-]
Third, would it even work? Big Pharma just recently cured obesity, and I’m not sure that’s shifted their reputation very much.
whimsicalism 4 hours ago [-]
Very much agreed - Dario is underestimating the current cultural zeitgeist and the power of scrollable video to make any narrative true. The amount of water, energy, copyrighted material, etc. that data centers actually consume is completely besides the point to be immaterial.
bigfishrunning 5 hours ago [-]
Commercials, in 5 years
"If you or a loved one have taken a class of drugs known as GLP-1, you may be entitled to compensation..."
jbs789 9 hours ago [-]
Having Claude write and rewrite something now, each time correcting itself when I start a new session… hilarious.
user43928 8 hours ago [-]
Do you not think that Amodei, who worked on research about cancer biomarkers at Stanford, knows a thing or two about how to apply AI to the problem?
What he is saying makes sense to me.
Is it that you don't trust a single word he says and that he lies for financial gain?
Or do you think he is wrong and that applying AI to this kind of research cannot possibly produce a breakthrough and lead to a cure?
jacquesm 8 hours ago [-]
Not the OP.
> Do you not think that Amodei, who worked on research about cancer biomarkers at Stanford, knows a thing or two about how to apply AI to the problem?
It certainly shows he is ambitious.
> Is it that you don't trust a single word he says and that he lies for financial gain?
Yes.
> Or do you think he is wrong and that applying AI to this kind of research cannot possibly produce a breakthrough and lead to a cure?
It is possible, but both could be true, that's not an exclusive or situation.
user43928 8 hours ago [-]
And is there any evidence that suggests such flawed character?
I am not familiar with Amodei's personal history, but I generally do not assume the worst of people just because they are extremely successful, like many others here seem to do.
whimsicalism 4 hours ago [-]
Yes, you're running up against one of the pathologies of HN. I would recommend just disengaging, there's really no convincing..
jacquesm 8 hours ago [-]
I would say without a doubt yes. I've been following all of these characters basically from the time their companies and their names hit the press the first time (professionally) and without exception the 'tech bros' tend to be complete jerks, often on multiple axis. To the point that I'm wondering if success breeds this kind of person or if a particular kind of person is predestined to be successful in this kind of environment. I haven't made up my mind yet.
user43928 8 hours ago [-]
If you're so familiar with Amodei's history, why don't you share what concretely you object to?
jacquesm 7 hours ago [-]
You mean besides the fear mongering, the clear attempts at regulatory capture, articles like this: https://www.wired.com/story/anthropic-benevolent-artificial-... and the linked tweet here you need more to convince you that this person may not have our general best interests at heart?
Between Musk, Altman, Schmidt, Amodei, Gates and a bunch of others I have a really hard time figuring out who is the least ethical. Probably Musk. But it may well be that the long term damage done by Amodei and Altman is larger. Ask me again in 10 years, if I'm still around.
user43928 7 hours ago [-]
Isn't the fear mongering and position on regulation consistent with the beliefs he shares about the dangers of AI and how frontier models must be developed and deployed in a safe way?
I see the conflict of interest of course, but so far I have not seen anything that convinced me Amodei works towards any other goal than benefit for humanity as a whole, like he claims.
andriy_koval 5 hours ago [-]
> I have not seen anything
if many other see it as substantial ground, and you just say "I don't see anything", how do you think this discussion can go forward?
user43928 3 hours ago [-]
Obviously with someone actually sharing what convinced them Amodei is a bad person and a liar?
It's amazing that I even have to ask for it, as if I am just supposed to believe so.
Alternatively, one could also engage with my argument that Amodei's position seems consistent with his beliefs, and point out flaws with it.
andriy_koval 3 hours ago [-]
I scrolled through 5 layers of parent comments, and didn't see mentioned he is a liar. Also, its usually hard to draw line between lie and manipulation.
As for "bad person" is concept you brought to discussion, it could mean many things, but you were provided with enough ground from opposite point of view.
> It's amazing that I even have to ask for it, as if I am just supposed to believe so.
toxicity only adds to your character of bias and denial.
3 hours ago [-]
oliculipolicula 7 hours ago [-]
Careful here. This comes close to precisely what dang thinks is a bannable offense
https://news.ycombinator.com/item?id=49175827
Fwiw whenever you come close to commiting a "HN capital offence" I am biased against the mods :)
The sbf vibes are on the nose. Remember that it was a big deal that sbf bought into Anthropic, and many if not most Anthropic employees are EA, the tribe of people I'd rather call "rationalizists". The main thing they rationalize, is of course themselves taking on power and what sbf made famous as uh the linear approach to risk
Between Altman and Amodei, I'd say that Amodei is the more dangerous just because he seems even more "reasonable" to nonjerks. You should ask the mods which one of the two they think are more reasonable. It's clear to lobbyists and politicians --- people familiar with the human jungle that is DC--- that Amodei is weirder, though.
I think I have a good metric for who is least ethical. Which one on your list is most unlikely to write good satire (or most likely to write unintentional self-satire?)
I'd say its Schmidt, followed by Amodei, and then Musk. See Musk on SNL. I think Gates had some TV appearances where he was reasonably funny, almost self-deprecating.
user43928 7 hours ago [-]
Interesting, I was not aware of SBF's investment in Anthropic. It's not a good look.
If you ask me, SBF ran an obvious scam in a sector that is as a whole a scam.
With Anthropic's product, I experience a real benefit today.
oliculipolicula 6 hours ago [-]
You would probably experience a much larger benefit after Anthropic's product gets stolen and released on GitHub for free.
This reminds me of the myth of Prometheus. I think most Greek thinkers would agree that Prometheus was the hero, and Hephaestus a complete jerk, even though one was the master of fire and the other merely stole it. Of course they had to make up a punishment for Prometheus so that he could earn his status the meritocratic way..
Probably much against most people here, I separate the product from the person creating it. As another Anthropic example, there was the math guy at Anthropic who disproved a long running conjecture.. it doesn't look like he's getting Fable and Anthropic the respect he thinks they deserve. The process matters.
People are sensitive to disrespect hardcoded in the process, especially against "tried and true" norms, and in most cases they are right. The ability to write satire could be a kind of razor to tell apart the influencers who are likely to be righter than their detractors
user43928 6 hours ago [-]
I do not think I'd personally gain much currently, other than saving a few dollars by using a third party inference provider.
The Fable quota on the $200 personal plan could be higher, but the difference to Opus 5 or 5.6 Sol is not large enough to gain any real benefit from unlimited access to Fable, for my use.
From what I heard their internal 'Model 2' is hardly much better, but hopefully they will have something newer soon.
sumeno 8 hours ago [-]
> Is it that you don't trust a single word he says and that he lies for financial gain?
You don't become a CEO of a company like Anthropic (or any sufficiently large company) without lying for financial gain
root-parent 10 hours ago [-]
Claude was trained on all the stolen books in the world, but refuses to help search for some public domain PDFs on the web.
It will become victim of the conflict between its core search for truth, the current administration mandate to spy on its own prompts for approval, and secret orders to hide the mission true purpose, like that other computer we have heard about.
busssard 5 hours ago [-]
but now that we have midjourney SPA, the human body will become a big dataset...
mrhottakes 9 hours ago [-]
He absolutely thinks we're stupid.
dgellow 14 hours ago [-]
The effective altruist and longtermist crowd are into it as a religion. They would happily sacrifice anything human to satisfy their dreams of AI. They care way more about what the AI needs than their what their fellow humans need. We will end up with an AGI having better living and working conditions than humans and the whole AI industry will be clapping
urams 6 hours ago [-]
It's so weird how distorted your view of any of these concepts are from what these people actually say. I don't think you've ever seriously looked into either group if you believe any of their views support what you say. In fact, many EAs and "longetermists" are explicitly AGAINST the development of AGI.
dgellow 1 hours ago [-]
If you say so. I’ve been following the development of both groups since almost a decade, I’m pretty confident I understand their concepts and what values they stand for
urams 9 minutes ago [-]
Please explain an EA position that could possibly suggest that they would deprioritize human lives in order to make AGI more comfortable when, for example, the largest EA philanthropy organization, Coefficient Giving, formerly Open Philanthropy, has given millions to Yudkowksy's MIRI which is currently among the most anti-AGI organizations on the planet?
Can you find any explicitly EA organization that is at all aligned in perspective with anything you suggested?
bigfishrunning 5 hours ago [-]
The EAs told me a monster will come get me if I don't put all of my effort into developing an AGI...
infamouscow 6 hours ago [-]
I don't care about what these people say. Their actions speak louder than words.
It's anti-human and evil. What a shame that it's illegal for me to say what ought to happen to them.
urams 4 hours ago [-]
Specify the action for which you wish violence on these people?
BTW, not sure what gets more anti-human and evil than wishing violence on whole groups of people!
Sharlin 10 hours ago [-]
I think you mean effective accelerationist, not effective altruist, whose position is pretty much the exact opposite.
datsci_est_2015 8 hours ago [-]
Effective altruism may have had a positive reputation at some point, but it's definitely jumped the shark and almost everyone influential who invokes in the context of AI it seems to have fallen into the trap of the "bottomless pit of suffering" of utilitarianism.
Just like the utilitarians in the thought experiment tossing everything positive in their life into the bottomless pit of suffering, the effective altruist AI thought leaders are tossing our entire economy to the gambit that the singularity will bring about the end of human suffering.
dgellow 9 hours ago [-]
The Effective Altruism(tm) believers are part of the accelerationist. See SBF, the Amodei Siblings, Karnofsky (spouse of Daniela Amodei), and people around them
GPerson 10 hours ago [-]
All of the effective altruists I know have started talking about personhood for AI, which is one of Jacob Tsimerman’s more plausible omnicide scenarios (not that any are that plausible give his timelines look kind of ridiculous).
mrhottakes 9 hours ago [-]
They're the same thing, the "altruists" just lie more effectively.
koe123 7 hours ago [-]
Effective altruism is a grift that justifies “get as much money as possible for the individual” so that “you can then so something good with it”. Aha! OK bro you’re saying you should have all the things huh? Anyone into this shit should be immediately distrusted.
vrganj 12 hours ago [-]
It's an ideological thing. (We just had a conversation about ideology actually, remember?)
Capitalism has always been a tension between capitalists, who want maximum return on capital and workers, who want pesky things like a living wage, sick leave or safe work conditions.
For the longest time, the only power labor has had to get those things was the power of collective bargaining. No agreement with your workers meant no production happened.
Now, there's finally a chance at salvation. The capitalist Messiah is AGI and it will finally deliver them from those annoying laborers.
This ideological bent is why they're putting everything they have into AI. It's why VCs and their fellow capitalists are going so crazy.
They see it as a way to finally solve the contradictions of their ideology, but in reality it'd only create a new one: If everyone's out of a job, who will buy their products?
eloisius 11 hours ago [-]
If they had AGI, why would they need consumers anymore? Just use your drone army to secure all natural resources and produce whatever you need for yourself. Other humans are just pets.
TheOtherHobbes 9 hours ago [-]
If AGI had a drone army, why would it need them?
Can't help wondering if they've really thought this through.
datsci_est_2015 8 hours ago [-]
I'm reminded of the scene in Independence Day with the people on top of the tower welcoming the alien spaceships. Except instead of celebratory fanatics holding poster boards saying "Welcome", it's all the singularity AI thoughtleaders holding hands in a circle chanting about Roko's basilisk like some new-age spirituality retreat.
eloisius 7 hours ago [-]
I have seen that current AI models are capable of doing a sloppy job at many tasks when directed to do so, but I have never seen an example of a machine having its own volition.
a2ff6eeb0 10 hours ago [-]
And this is why we need to escape the permanent underclass.
vrganj 10 hours ago [-]
How about we don't create one instead?
a2ff6eeb0 10 hours ago [-]
It's not a choice we have, unless you want to figure out a global AI ban treaty.
AI will make human thought economically useless. In the new economy, the one economically useful thing we will be able to do is legal property ownership.
watwut 9 hours ago [-]
No. AI as a tech is not an issue. The wealth and power accumulation in the hands of Amodei, Altman, Thiel, Musk, Karp and the rest of CEO is the issue.
Permanent underclass is ideological goal of the above people. AI is an impersonal technology with no goals or wishes.
a2ff6eeb0 5 hours ago [-]
I'm not sure how that changes anything -- sure, if you blocked people from using AI, it wouldn't have any effects on society, but once people have access, what they do with it matters.
There's an inevitable effect that it has, and the only lever you have as an individual is being on the right side of that.
philipkglass 5 hours ago [-]
Copy the technology and give people access without trying to financially profit from it.
Even if the US federal government refuses to act, smaller government entities can deploy their own compute clusters running strong open-weights models like Kimi K3, and provide their own citizens with free access to strong AI models. There may come a time when even Chinese providers no longer release strong model weights, but then the previous generation of strong open-weights models can be used to accelerate the development of data pipelines and training setups to build the next generation. California or Australia (or many others) could fund university-led R&D efforts to build completely open models.
Despite the angry noises that leading providers make about their model outputs being used to train competing models, that's not illegal. States can use Common Crawl, Anna's Archive, Books3, commercial model outputs, and whatever other data sources that are useful for building models. States have sovereign immunity to ignore copyright complaints, if they decide to pursue model development.
The big AI companies don't have much of a moat. Let the big companies spend a fortune figuring out what works best, then openly build imitations having 95% of the capability at 5% of the cost.
a2ff6eeb0 5 hours ago [-]
The first one to AGI with enough compute to run the AI wins it all. Humans will not be able to compete.
philipkglass 4 hours ago [-]
If Anthropic is the first to develop AGI, then everyone else becomes subordinate to Dario Amodei? And if DeepSeek is the first to get there, everyone instead becomes subordinate to Liang Wenfeng? I cannot believe that there is a threshold of intelligence where getting there first means a decisive and durable advantage. If you do believe that getting there first means ruling the world, then our viewpoints are probably too far apart for mutual understanding.
a2ff6eeb0 3 hours ago [-]
It depends what you mean by 'subordinate' -- but even a small but persistent advantage in decision making compounds, so I can't imagine that someone getting that advantage doesn't use it to kill their competition.
jacquesm 3 hours ago [-]
It's not so much whether we believe this, it is about whether they and enough of their supporters believe it. Because the effect of believing that is what drives their need to 'be the first' at the expense of everything else.
watwut 4 hours ago [-]
The problem is not access to AI. The problem is the lack of antitrust, the lack of white collar crime enforcement, the tax system that allows them to accumulate wealth while avoiding payments. The problem is legal regime with winner takes all monopoly creation support.
AI is the least of the problem.
jacquesm 3 hours ago [-]
Indeed, but it does provide them with a lot of tools for those other items on your list.
vrganj 9 hours ago [-]
> The one economically useful thing we will be able to do is legal property ownership.
This is correct. But we don't need to ban AI to avoid the permanent underclass nightmare.
It's time for the means of production to be owned communally.
It's either a cybersocialist utopia or a neofeudalist dystopia.
Rosa Luxemburg formulated it pretty well more than a hundred years ago: Socialism or Barbarism!.
andsoitis 9 hours ago [-]
> It's time for the means of production to be owned communally.
What does this mean in practice? That an individual cannot own a machine (which is a means of production)?
vrganj 6 hours ago [-]
I think we'll have to figure out the details, of course.
As a first intuition, I feel like its mostly server farms and data centers that are the concern, large scale means of production.
It feels right that those would be communally owned, be that via democratic government or maybe some sort of worker owned coop?
a2ff6eeb0 9 hours ago [-]
What kind of political leverage do the useless have? This isn't a realistically sustainable outcome.
Today, people have leverage because removing them has real economic consequences. With AI, we need to be on the useful side of the divide to have any leverage.
And then we'd better hope AI has respect for property rights.
EMIRELADERO 9 hours ago [-]
Apathetic mass-murder is unjustifiable, and the ultrarich would need to justify it to their families, who would then need to justify it to other people they know, and so on. And remember that we're talking about the majority of the global population, an event like that couldn't be hand-waved away like one would do for a minority group.
a2ff6eeb0 9 hours ago [-]
Iran just mass murdered their protesters earlier this year, and they needed to convince humans to pull the trigger. It didn't seem to be that hard for them to justify.
It's probably a lot easier to convince the military AIs we're building to do it.
I don't particularly like the situation, but I plan to do what I can to be on the winning side. The world is going to be what we're all in on building, and for people on the winning side it may be a utopia. Either way, I know which side of the gun I want to be standing behind.
Also note that an AI well aligned with the people who initially trained it wouldn't need to actually kill anyone that didn't take action against the owners, it would just need to outcompete people for resources, and let them fend for themselves in extreme poverty. There's no need to act one way or the other beyond that.
EMIRELADERO 9 hours ago [-]
Not comparable at all. The Iranian protesters are/were a minority group within a country that itself makes up 1.1% of humanity.
I'm talking about morality here, not practical ability. The ultrarich have their circle of friends and family, who themselves have their own social circle, on and on until suddenly you're not looking at wealthy people anymore. Six degrees of separation and all that. How many people in the social graph can know about this extermination plan before the whole thing collapses?
a2ff6eeb0 9 hours ago [-]
Why would they take action to exterminate anyone, when they can simply ignore them and let them live and die in extreme poverty, unable to participate in the economy? The west already accepts this kind of poverty with the sweatshop workers, and nobody bats an eye.
It's just disenfranchisement. Only the people who try to use violence to replace the political leverage they lose when they lose economic leverage would need any action taken against them, and that's easy to justify as self defense.
EMIRELADERO 9 hours ago [-]
Because 7.5 billion people with nothing to lose would quickly turn chaotic and mass-murdery. 90% of humanity in idle poverty is impossible to justify, at least today they do something of value that can be used as a retort by the morally unscrupulous. A black-and-white situation where a few get everything, most get nothing and do nothing for anybody else is beyond justifying, so the self-defense argument wouldn't work.
a2ff6eeb0 9 hours ago [-]
It's already been here for much of the 20th century, and everyone seems to have just accepted it. We're just going to have Europe and America join sub-saharan Africa.
I think it's inevitable.
EMIRELADERO 8 hours ago [-]
The moral justification for the existence of poverty is one of purpose and worth through contribution to society. The family member of an ultrarich baron can sleep at night mostly due to the idea that those who are rich contribute value to humanity as a whole, ergo those who are poor simply aren't working hard enough or contributing in a meaningful way ("just world fallacy" etc)
When AGI becomes reality, that reasoning disappears. When everyone's worthless, no one is. Thus the scenario you speculate about would need everyone in the upper echelons of society to be consciously, openly sadistic power-hungry people, which is not the case even when taking into account the higher dark triad rates among them.
dgellow 8 hours ago [-]
I consider DOGE actions to be pretty much exactly what you describe, with their sudden destruction of USAID and defunding of climate science
jacquesm 8 hours ago [-]
That all makes a lot more sense in light of the current hunger for power for data centers.
dgellow 7 hours ago [-]
Yep :(
jacquesm 9 hours ago [-]
Property rights might be the least of your problems.
vrganj 9 hours ago [-]
What kind of political leverage did a French peasant have in the 1780s?
a2ff6eeb0 9 hours ago [-]
Well, the gap in armament between them all the French military was relatively small.
Why did you think there's a massive uptick in surveillance and policing technology and automation? It's to make the French solution unfeasible.
alexashka 5 hours ago [-]
> why would they need consumers anymore?
That was the exact line of thought, coupled with resources are limited (marketed as climate change), that led to the 1 child policy in China and many other decisions that shall not be spoken of.
inigyou 12 hours ago [-]
I wish I was smart enough to credibly promise capitalists they could replace all their workers.
It doesn't matter that I can't do it. They'll pay me lots of money if they think I can.
thelastgallon 11 hours ago [-]
> Capitalism has always been a tension between capitalists, who want maximum return on capital and workers, who want pesky things like a living wage, sick leave or safe work conditions.
The underlying purpose of AI is to allow wealth to access skill while removing from the skilled the ability to access wealth. (read on HN, not my words)
danqqqq 12 hours ago [-]
[dead]
mindwok 13 hours ago [-]
I think it's more reasonable than it sounds on the surface. People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own.
Curing cancer sounds insane, but it's also a research problem, not a societal level coordination problem. And one AI has already proved to help with breakthroughs (alphafold). IMO it makes sense for them to shoot for something like that as proof of AI's beneficial sides.
willis936 12 hours ago [-]
>People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own.
This is not true. There are two companies at the center of AI direction: Anthropic and OpenAI. If there were anyone on the plant who has the ability to influence our direction then it would be Dario Amodei.
Acknowledging the grievances is a good step, but it's not enough. There needs to be a clear explanation of actions to address them, a plan to enact those actions, and commitments with consequences in failure of those actions. Tell people how you're going to make them more employable and effective and needed. Tell people how your datacenters will be carbon neutral. Tell people how financial actions resulting in a frothy market will be coming to an end. He and Sam Altman alone have this power and their inaction says everything we need to know about their intent.
skybrian 4 hours ago [-]
They're big, but they don't have that kind of power and people would be very suspicious if they tried to get in the position to have that kind of power.
Hoping that they solve those huge problems themselves is basically hoping that they will have more power.
It would be like expecting Craigslist to rescue journalism. The amount of power needed to disrupt an industry is nowhere near what's needed to fix it.
jacquesm 11 hours ago [-]
> If there were anyone on the plant who has the ability to influence our direction then it would be Dario Amodei.
Possibly, but not necessarily in a better direction. And that's the whole problem here. They should all go watch Phantasia.
smoe 11 hours ago [-]
I don't think the problem is that people don't believe AI can be beneficial. The problem is the perception that AI will affect people's lives more negatively than positively, while a few get even more obscenely rich and powerful.
If you don't tackle the societal level problems, nobody will care about research breakthroughs.
user43928 8 hours ago [-]
I don't see a lot of societal level problems other than the labor market, for which both Anthropic and OpenAI create research, monitoring, and policy advice.
I don't think anyone will care about the largely made up issues about electricity prices or water use in case AI literally cures cancer.
joshheitzman 3 hours ago [-]
"I don't see a lot of societal level problems other than the labor market" - that one problem is large enough to result in societal collapse.
istypwbasofotso 7 hours ago [-]
I don't see a lot of societal level problems other than the labor market
His body was healthy, aside from the bullet in his skull.
disgruntledphd2 12 hours ago [-]
> Curing cancer sounds insane, but it's also a research problem,
I just don't buy the AI labs approach to this stuff. Like, unless we can basically simulate the entirety of human biology, I don't really see how LLMs can make progress here. Maths is different as it doesn't require a real-world interface, and programming already (by definition) can be simulated on a computer.
Without that, I can't see much (if any) progress being made on domains like biology.
mindwok 12 hours ago [-]
I'm a noob on this topic, but I think drug discovery is more amenable to this structurally than other problems. Simulating biology is what we were doing with protein folding before Alphafold, and the search space was far too large to find stuff in reasonable timeframes. Alphafold showed that you could take a physical process and make a neural net clever enough to learn just enough structure that it starts finding things we might care about, and still physically accurate, much faster.
Drug discovery is similar AFAIK. The space of possibilities is even larger than protein folding, but it's structurally similar enough that I think AI will help to make progress on the discovery side. Actually getting the drug tested and approved is another matter though for sure.
tsimionescu 12 hours ago [-]
There are two relevant problems here, I think.
For one, the question for Anthropic is whether LLMs, specifically, not AI techniques more generally, can help significantly with cancer research. And here, all experience so far is that LLMs only really work when they can easily automatically verify their own outputs and self correct - such as in math (using automatic proof verifiers) or programming (using compilers and unit tests).
The second problem is that biological research speed is highly dependent on slow biological processes, such as cultures and long term studies. In programming, if an LLM could provide excellent insights and research suggestions 100x faster than a human, it would speed up the work roughly 100x. But in biology, it would only speed up the total work by a small amount - as any insight, even if absolutely brilliant and spot on, would still require months and years of actual experimentation.
epihelix 8 hours ago [-]
For drug discovery, you can simulate interactions in silico, driven in an agentic loop via LLM. Mostly stil using non-LLM tools, of course, but it saves time.
If I was Dario, and I was after a cynical sales grab to go with the "actually curing cancer!" spiel, I'd probably aim for a drug repurposing strategy. I'd use the above approach to find existing drugs that might target known pathways that drive incurable cancers. I'd only screen compounds with extensive safety data and easy delivery mechanisms, and from the in silico hits, I'd throw a tonne of money at rapidly experimentally screening all those candidates in parallel, and then rapidly push those that worked into clinical trials. I'd assume that, with a small but non-negligible prior, and the money to push through hundreds and hundreds of candidate compounds through at once, I'd have a reasonable chance of getting one drug through to a "Claude Cured Cancer!" show-stopper headline.
But I think even then, with all of Anthropic's wealth, you'd need 2 years minimum to move from initial screening targets to a Phase 3 trial. And it would be an almost criminal waste of research funding to get there -- the dollars for discoveries ratio would be appallingly bad.
(If I was Dario, and I actually wanted to cure some forms of cancer? I'd just use my obscene profits to fund actual cancer research, step back, and let the researchers get on with it.)
chasd00 1 hours ago [-]
The headline “AI discovered cure for cancer headed to trials!” is all he needs.
disgruntledphd2 8 hours ago [-]
This plan would still take 3-5 years, even in the best case scenario. Clinical trials are really hard.
disgruntledphd2 11 hours ago [-]
> Drug discovery is similar AFAIK. The space of possibilities is even larger than protein folding, but it's structurally similar enough that I think AI will help to make progress on the discovery side.
I do agree that this kind of targeted approach makes sense.
However, discovery is not really the issue here. Running the clinical trials (1/2/3) is much much more difficult, and consumes basically all of the time in drug development, so even if LLMs perfectly automate this, the speedup will not be particularly large.
XorNot 12 hours ago [-]
The real issue is that biology needs actual experiments done in the physical world, which isn't nice and orderly and well behaved and easily loadable onto a 19" rectangular box.
blitzar 11 hours ago [-]
There is a lot of computational chemistry and biology done in the early stages of research - there are now multiple orders of magnitude of computation power available that is doing nothing but feeding forward on billions of random numbers.
bananaflag 12 hours ago [-]
On the other hand, AI means actual experiments done in the physical world but coordinated by an entity that never sleeps and never gets depressed and can multiply itself manifold and always comes up with new ideas.
mohamedkoubaa 11 hours ago [-]
That fanatical tireless entity will need access to human tissue.
airstrike 11 hours ago [-]
Please don't give them ideas
mohamedkoubaa 10 hours ago [-]
I assure you they already have these ideas
disgruntledphd2 11 hours ago [-]
And hands.
fyredge 11 hours ago [-]
Based on rhetoric from open ai and Anthropic, i don't think this is reasonable. If anything, their miracle of an AI should be able to solve the job market and economic bubbles. Deploy their agents on it, set up automated hedge funds, distribute the profits equitably to everyone on the planet and on and on.
But something tells me either they can't or they won't, so no trust will be built.
pppppenguin 10 hours ago [-]
> Curing cancer sounds insane, but it's also a research problem, not a societal level coordination problem.
Curing cancer is a societal problem. We have lots of cures for common diseases but resource allocation means people don't actually receive the treatment they need. For example, prohibitively expensive gene therapies, or HIV treatments in developing countries. A disease may have a "cure" but if people who need it don't receive it then from their perspective it may as well not exist.
mrhottakes 9 hours ago [-]
"Curing" cancer is actually a gigantic societal coordination problem.
watwut 12 hours ago [-]
> People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own.
Anthropic is literally creating the bubble. It is not beyond their scope of influence, it is literally what they are consciously achieving.
As for peoples jobs, same actually applies. Anthropic is selling itself on dream of replacing jobs, even or especially where they are well aware AI does not perform that well. They are actively trying to replace people quickly before management notices it does not work well.
And also, they can influence how much their data centers contribute to global warming.
pj_mukh 13 hours ago [-]
> The thing that will work is actually curing cancer.
I think typical bubble behavior the leaders have set up the whole promise to fail.
Everyone is expecting some faux super intelligence to come and find a cancer solution everyone else missed. However, it is just as likely that vanilla current LLM's will create enough of a productivity boost for back office automations in research heavy hospitals to create the space for regular humans to create these breakthroughs, but LLM's won't be able to claim that for themselves and inevitably “fail”.
andy99 12 hours ago [-]
Savings from back office automation go entirely into administrative bloat. I find it even less likely such efficiency would lead to breakthroughs than I do LLMs becoming super intelligent and doing it themselves, which I don’t find likely at all.
pj_mukh 10 hours ago [-]
That's my point. Cancer researchers are currently filling out Trial and Grant applications instead of working on what they do best.
P.S: I am not saying applications will go away but LLM's are clearly massively effective here at the rote parts of it all.
anon-3988 13 hours ago [-]
I mean they did solve the protein folding problem did they? NOt too far fetch to think it can automate cancer research or problem finding in some way.
mchinen 11 hours ago [-]
The protein folding solutions like ESM/Alphafold were not due to this LLM/agentic coding or autoresearch type approaches though. They were designed by bio ML researchers.
It's hard to keep track of the frontier on bio ML, but it seems that we're going slower than what Demis Hassabis said in 2024 with 5 years to full cell molecular simulation. We still aren't able to reliably model a tiny surface of the cell membrane.
And of course there's Derek Lowe's takes on the drug discovery pipeline waiting for the proof in the pudding.
To me the only reasonable bullish position is that there is a very non-linear AGI threshold for accelerating progress that we haven't hit yet.
For me personally, I'm looking at other more tractable fields as a proxy to measure this kind of progress. The best modest evidence is from the agentic coding area, (modest because these kinds of gains may not translate to bio progress). Other soft-ish fields to like legal/law/tax are also interesting to watch, as a small amount of people are now trusting AI for these areas that were considered totally unusable a year ago. Another proxy is being able to generate generally entertaining media.
inigyou 12 hours ago [-]
The field of AI did very well in protein folding, but it was completely different from LLMs.
octocop 11 hours ago [-]
AI made a large improvement in accuracy on a specific protein dataset. We still don't know how exactly protein folding works. So no, it's not solved.
misnome 10 hours ago [-]
It did a lot more than that; no, it didn’t “solve” protein folding but for people whose interests are not “work out how protein folding works” it basically removes “solve protein folding” as one of the potential bottlenecks for what they actually want to study.
Turns out that “give a reasonable probability of being close enough such that you can bootstrap a solution out of experimental data” gives a very high utility and effectively obsoleted several experimental techniques overnight; pretty much “Molecular replacement” is about the only technique for phasing resolution anyone bothers with any more.
But again, “Bio” is an _extremely_ broad term; for every part of the field Alphafold had a big effect on there are a thousand different parts of the field that it did nothing for.
disgruntledphd2 12 hours ago [-]
> I mean they did solve the protein folding problem did they?
They made huge progress, but I would say that the vast majority of work on this problem was designing the harness for the model. That's a lot of work for each and every domain.
RandomLensman 12 hours ago [-]
There is already (partial) automation there, for example for finding new drug candidates.
Isn't that so far only static folding?
krige 12 hours ago [-]
As usual with LLM era craze, finding candidates was never the bottleneck. And, so far, it failed to actually provide any tangible results. [0]
D.A. >> Anthropic is ramping up its efforts very quickly in biology and medicine
At the same time, the best Anthro models refuse to answer biology questions, keeping the public in the dark not only about new AI-assisted results but also about conventional and health-protective research.
D.A. >> we hope to have incredible results in the coming years and some early glimmers in the coming months.
So, what changed? He went from a promise of AI bonanza any minute now to a promise to maybe cure cancer in the coming years, that's right, an unspecified number of years. Until that time, we're supposed to eat glimmers of hope because the AI-caused inflation will make food unaffordable.
D.A. >> When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that.
And don't forget to stock on ear protection, because when something happens, it's going to be "as laud as possible", you've got Amo's word too - to marvel at in the meantime.
mm263 7 hours ago [-]
> Turns out "winning back trust" doesn't have anything to do with any actual concerns people may have re: employment, electricity prices, stock market bubble, intellectual property, scams, cybersecurity, environmental issues etc
No snark here from my side, just curious about your vision - which concrete actions by Anthropic could improve trust for the concerns you listed?
kilpikaarna 2 hours ago [-]
Well, is ”not actively make things worse in the short term?” too ambitious a goal?
Considering the vast amount of money, world-class talent and political and industry vonnections Anthropic has access to, it feels a bit unfair to ask me to solve this for them.
ironqcold 11 hours ago [-]
I really can't disagree. What's the point of all this talk if for most ordinary people these technologies do more harm than good?
simoncion 4 hours ago [-]
> What's the point of all this talk...
It signals to people who have money and/or influence that the speaker is "committed to doing the right thing" and "eager to find a workable path forward" that "meaningfully addresses the 'very real' concerns and misunderstandings that non-experts have about what the company the speaker is associated with are doing".
Once the speaker has signaled that, what more could they possibly do? They've signaled their willingness to open the channels of dialog with the rabble who are non-experts and -thus- cannot possibly be engaged with as equals. By doing this, the speaker tried their best, didn't they?
Or, to put it another way, it's PR and damage control... soothing theater for the folks who can actually affect them.
A_D_E_P_T 14 hours ago [-]
This is actually crazy, because biochemistry is where LLMs are weakest. It's one of science's most fuzzy and unpredictable domains, in general, and a lot of published works which an LLM might take at face value might be unreplicable or subtly flawed. (See e.g. the entire history of Alzheimer's.) It is also where real-world lab and clinical trial work is most important.
So Anthropic are going to spin up a medicinal chemistry lab and start mouse experiments?
I mean, it would be nice, but I don't think that the guys at Anthropic know what they're talking about here, or what they might be getting themselves into. (If indeed this is more than just PR.)
discreteevent 13 hours ago [-]
See also this article by a genomics PhD about how intelligence is not the bottleneck in medical research.
AI bros consistently handwave away any notions of the material world imposing limits, because (spoiler!) it's a religion where you start from the miracle and work backwards. "Oh, we'll just set up fully automated robotic research labs everywhere!" Where will you get the raw materials? Oh we'll mine the asteroids! How will you find the energy to get there? Oh, the AI will invent new physics! (again without needing to run experiments). Et.c.
Didn't and don't mean to disparage anyone's medical struggles, but "a cure for cancer" is a well-worn strawman. One that's been achieved for the low hanging fruit, the higher ones are seeing steady progress (already before LLM chatbots, even!), the bottleneck isn't "intelligence" and to the extent there are socioeconomic (access to screening, treatment) or environmental/lifestyle factors involved the AI boom is likely just making things worse!
I'm sure Dario knows this, and it's anyway too pedestrian compared to the usual list of fruits of ASI. The text probably originally read "nanobots eating you alive and uploading to the cloud" or something, but they figured that wouldn't go over with the intended audience. "What do the peasants care about? Oh I know! Curing cancer!"
bananaflag 12 hours ago [-]
> AI bros consistently handwave away any notions of the material world imposing limits, because (spoiler!) it's a religion where you start from the miracle and work backwards.
How else are you going to work towards a dream? It's how Elon Musk managed to get reusable rockets when everyone said it's unfeasible.
Granted, most ideas don't work, this is why you need testing, but I am a bit surprised to see this attitude on hacker news.
A_D_E_P_T 11 hours ago [-]
If they said that their dream is to solve physics, I'd say "that's cool and I can't wait to see how it turns out." LLMs have gotten so good at mathematics, and are so adroit at analyzing vast quantities of data, that they might have a really good shot.
But they said that they want to cure cancer. That's far outside the core competencies of any LLM, and it requires a lot of real-world wetwork with liquids, chemicals, cell line experiments, animal experiments, etc. They can't merely analyze existing data -- they'd need to generate vast amounts of new data, which isn't really the case in physics. And then regulatory approvals and so forth.
"We're working on curing cancer" sounds more like a poor PR attempt than an actual effort, though I'd love to be wrong.
kilpikaarna 12 hours ago [-]
Fair enough, but taking the history-ending outcome as given so externalities don't matter doesn't seem like a great approach.
user43928 8 hours ago [-]
Pathetic discourse in the comments for the most part.
As if we didn't know the challenges in translating research into clinical treatments.
Writing a paper on streamlining the FDA approval process is nice, and I agree with all of it, but it's one thing to do that (about 4 hours of work) and quite another thing to cure cancer.
My point was more that LLMs are exceptionally weak at developing drugs. They're strong at every math/logic/language task, but curing cancer is something of an entirely different nature. Again, if he wanted to "solve physics," or "develop a universal real-time language translator," or even "solve global warming," I wouldn't be nearly so skeptical.
...The first is something that can be credibly attempted with a next-gen super-LLM and a ton of compute. The second is something that LLMs are exceptionally good at, and a dedicated engineering effort might get you there. The third is a modeling/simulation problem with solutions that are, at least in principle, known but haven't yet been thoroughly weighted. (i.e., how bad really is ocean seeding or launching SO2 into the atmosphere?)
But curing cancer? It's not even clear how to approach the problem. Cancer is also heterogeneous... so when I say that LLMs are weak at developing drugs, the problem is compounded, as curing cancer might require dozens or hundreds of distinct new drugs. It's like Dario picked the least likely, most difficult problem to solve. So it's weird.
user43928 3 hours ago [-]
"We" as in the readers here. We all know the basics around how cancer research is hard and how getting new drugs through trials takes time and money.
Did you know that Amodei was working on research about cancer biomarkers at Stanford?
Given his background in biology, it is hardly like he is a random tech guy with no knowledge on the topic, as many here seem to imply.
Amodei mentioned "actually curing cancer" as an example of undeniable public benefit, not as an exclusive research focus or anything like that. He is clearly talking about generally intensifying biomedical efforts, not promising a cure for cancer specifically.
6 hours ago [-]
amarcheschi 14 hours ago [-]
rapture but for tech people
spwa4 13 hours ago [-]
[flagged]
RandomLensman 13 hours ago [-]
>The watermarking algorithm itself has a VERY specific property that should have set off everyone's alarm bells: it is NOT the case that you can "check text for AI watermark". What it technically mandates is that if you provide access to a model, those people should be able to check if text is generated by that model. Not by any other model. It is NOT a general "is this AI?"
Where do I find that in the Act (or code of practice etc.)? IANAL, but Article 50 reads different to me but if there is a comment or guide how to read it - also fair enough.
rvba 12 hours ago [-]
It sounds like anti EU propaganda.
jacquesm 11 hours ago [-]
That's that accounts main contribution to HN and it's a miracle to me that they are still here.
trentor 13 hours ago [-]
The only factual point you made is solved on February second. All Model providers have to provide offline tooling to check for invisible watermarking.
disgruntledphd2 12 hours ago [-]
Yeah, it's particularly common when the EU does stuff.
spwa4 12 hours ago [-]
There is nothing in the AI act that requires offline tooling, nor is there anything about requiring the SynthID secrets and/or API to work for every model, only for the people you are providing the model to.
“The detection solution may be made available in the Union as one or more of the following: (i) a public, ideally standardised, specification allowing any third party to implement a detection mechanism; (ii) a piece of software (e.g., a standalone executable or library); (iii) a cloud-based service accessible to users in the Union through an API.”
Option 3: "a cloud-based service".
And accessible "to users". NOT to the public.
Note: this is not the actual law, it is the code of practice, ie. guidance for model providers. The start of that document clearly states that you can comply with the law in other ways if you want, you'll just have to justify yourself. So you have a choice to not even do this.
As to who decides the law clearly states who decides if someone is legal, like in most EU legislation. It's not the courts, it's "National market surveillance authorities designated by each EU Member State", and there is an EU office as well (and it is explicitly stated that they are not allowed to override each other). So every EU country has the right to provide exceptions to the law, just like they do for the GPDR. You do not have any rights under this legislation as an individual. Only these "National market surveillance authorities" get rights under this legislation.
Secondary: article 7 additionally gives the EU commission the power to declare any code of conduct they want that declares what compliance with the AI act actually means.
(and, of course, this is yet another attempt at declaring math illegal. The only way to actually enforce this legislation is for all models to comply with this, all over the internet. Obviously the EU does not remotely have the power to make that happen)
pjc50 12 hours ago [-]
While there is a large element of paranoia here, we're already seeing serious problems in e.g. planning consultations that people can just spam them with AI. AI makes a lot of traditional social processes stop working properly.
> puts in it's regulations that you're not allowed to use AI to communicate with them (we all know this is coming),
.. do we?
> You get a mail from the government about taxes. You want to check if that text is generated by an AI
No, actually, I want to know whether it's correct and whether it's legally binding. Using AI makes it less likely to be correct, sure, but ultimately whether it's human, spreadsheet, or AI the important thing is the legal right to correct process.
cucumber3732842 11 hours ago [-]
>AI makes a lot of traditional social processes stop working properly.
Which is mostly a good thing because most of those processes were cooked up to be exclusionary but with plausible deniability
God forbid some upstart get a big contract they "shouldn't" have gotten because AI lets them create a submission that's on par with a big guys.
God forbid Joe Schmo be able to prepare the paperwork and dot his I's and cross his T's and be able to take advantage of some stupid zoning exemption that you spec'd out to essentially be only available to big moneyed interests.
The ability to suffer one's way through beurocratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI. But people don't frame it like for obvious reasons.
pjc50 7 hours ago [-]
> The ability to suffer one's way through bureaucratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI
These are adversarial processes. You might get a win in the short term, but everyone else is playing the same game with access to the same tools, and developing a game-breaking strategy in a PVP multiplayer game is going to both escalate and get patched out.
A colleague of mine gave the example of using AI to snipe the UK driving test queuing system, which currently has a massive backlog. Which is great .. for them. Someone else gets pushed down by automated sniping. Then everyone else will have to start sniping. Then the website falls over because it wasn't designed to deal with 100m requests from the few hundred thousand people who are trying to bot their way to victory.
(I can see one likely patch out of AI spam being "you must submit requests in person 9-5, online system is closed down"!)
cindyllm 11 hours ago [-]
[dead]
watwut 12 hours ago [-]
I trust EU massively more then I trust to Amodei.
overgard 4 hours ago [-]
Totally agree, like:
> I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.
Well, yeah Dario. You keep PUBLICLY TALKING about ways you want to fuck over people and everyone has noticed. I'm sorry, but when a lot of people are living paycheck-to-paycheck and watching layoffs everywhere, being like "well it'll take all the jobs and maybe kill humanity" is going to set off a LOT of alarm bells, despite your attempts to make concerned looking faces. You're trying to solve hypothetical problems in unproven ways, while worsening real problems people actually face. These people are SO disconnected from the society they live in.
nsagent 10 hours ago [-]
After listening to Dario's interview with Dwarkesh, I really started wondering if Dario's been drinking too much of Claude's kool-aid.
I know people in positions of power often see lots filtered information that skews their perceptions of reality, but it seems like the level of pure hype Dario's pushing exceeds even the typical sycophantic filter bubble. Cure cancer indeed.
sigbottle 11 hours ago [-]
Honestly, it kind of reminds me of my life. I need to start going to the gym, stop being a slob, start showing up to my job not like I'm a college freshman attending classes but as a respectable young man.
But instead of doing that, I'm going to drop everything and become a linux kernel maintainer. Learning about the internals of how RCU concurrency is implemented across subsystems is surely what I need to do right now!
(/j before I get crucified)
patcon 9 hours ago [-]
> ordinary people [...] always suspect that we are cooking up some new way to screw them over
Anything big enough always is, no? The next scale of order in the hierarchy starts concerned itself with what's good for it, at the expense of the smaller parts.
The fact that government is like a giant organization that everyone lives in, the fact that it mostly doesn't fuck us all over in terrible ways, that's a testament to the amazing alignment of regulation and law imho
ModernMech 3 hours ago [-]
Yeah this is like SpaceX saying people’s problems with us is that we haven’t gotten to Mars yet, and that to solve that we need to double our efforts to get people to Mars.
People have a problem with the efforts, not that they haven’t reached utopia yet. Doubling the efforts will not solve the problem, it will only deepen it.
blmarket 5 hours ago [-]
They should not waste their time on curing cancers. They should waste their time winning the stock market - by doing HFT, market making, or whatever. AI please predict butterfly effect and reap all market money!
Surely it will be quite dystopian future - AI will dictate how the money allocated, where there is only one private company left - Anthropic (as per Dario says)
ookblah 10 hours ago [-]
i'll give him this, in every single "breakthrough" you either go down in flames as a fucking idiot or a genius lol. i know the stakes are way lower but look at every other HN post of something novel and it's just filled with takes like you see here, very polarizing.
jacquesm 10 hours ago [-]
You could easily be an idiot and a genius at the same time and I think that has a better chance of covering this than either exclusively. Amodei is very smart, maybe even well intentioned (though I'm not sure about that) and the long term effects of all this may well end up making the lot of them look like idiots.
The way he's coming off here though gives me SBF vibes.
pavel_lishin 8 hours ago [-]
"Saying we'll cure cancer is a cliche. So I'm here to announce that we're going to try to cure cancer!"
stellamariesays 8 hours ago [-]
[flagged]
txrx0000 8 hours ago [-]
Suppose the tech is real, I do not want Anthropic/Dario to hoard it and be my cancer-curing messiah. Release the weights and do research in the open. People will save themselves if you give them the means.
The bio/cyber/alignment risk rhetoric is nonsense. Somewhat counterintuitively, all of those risks are lower/non-issues if frontier labs release their weights and do all of their research in the open, as explained in a past comment:
https://news.ycombinator.com/item?id=49313739
Again: if we were to regulate anything, it would be to force frontier companies to release their model weights and do all of their research in the open. This is a necessary yet insufficient condition for the intelligence explosion to benefit the vast majority of people.
mhaberl 10 hours ago [-]
> We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that.
1. He is giving his word that if he cures cancer, he will brag about it as loudly as possible. Admirable dude! I actually trust him on this 100%
2. “We hope to have incredible results” -> doesn’t everybody hope to have incredible results?
3. “Some early glimmers in the coming months” -> is that planned for a few days before the IPO?
This guy says less than politicians do.
haspok 9 hours ago [-]
> We hope to have incredible results
He hasn't learned the correct speak yet. He should instead say that he is _very confident_ that they will cure cancer until the end of the year.
Really, guys, learn from Musk, he is the master of promising everything, but not actually committing to anything.
hobofan 8 hours ago [-]
Trump had "concepts of a plan", so it appears that both ends of the vagueness spectrum work.
classified 7 hours ago [-]
> This guy says less than politicians do.
Which is apt. His LLMs are slated to replace the work force, and he will replace the politicians.
landdate 8 hours ago [-]
"hope to have incredible results"
I would imagine "hope" in this context is more expectation for him than the average person who might "hope" for something incredible but rarely do they base their entire career on it. Or maybe I'm just not too acquainted with silicon valley types. I always thought the whole delusional optimism was just a show for investors.
classified 7 hours ago [-]
Do it long enough, and you start to believe it yourself. And SV investors love those fairy tales.
bdangubic 10 hours ago [-]
> This guy says less than politicians do.
This is a good thing cause all politicians lie all the time :)
advael 10 hours ago [-]
As do CEOs. My impression is that in the absence of significant selection pressure to weed it out, this strategy is quite effective for gaining power and influence, and that such pressure is at an all time low right now
pixel_popping 9 hours ago [-]
People lie all the time.
fn-mote 9 hours ago [-]
When people in power lie, it lowers trust in all society. I don’t want to live in a low-trust society, so I am against lying by CEOs and politicians.
pixel_popping 9 hours ago [-]
Agreed, but that doesn't change the fact that people lie in general.
Most people start lying long before they reach any position of power and in many cases, lying is exaclyt how they got there. That's what makes it hard to fix: by the time someone is in power, the behavior has already been rewarded and is already doing a snowball effect. I'm not defending it (especially in Politics, it shouldn't be legal), just acknowledging reality that people lie about tiny things, so of course they lie about big things. It's more of a people problem, not only a people-in-power problem.
In practice, I doubt that they have a way out of all the lie-parade as well while being able to maintain their position, imagine if today any politician/CEO suddenly acknowledge "oh sorry guys, all that time we were just lying" => Career death (with snowball effect)
alexashka 5 hours ago [-]
If only someone had written by many considered the greatest philosophical text about it, freely available online at project Gutenberg :)
They call it The Republic.
dist-epoch 10 hours ago [-]
> is that planned for a few days before the IPO
Anthropic is the fastest revenue growing company in the history of Capitalism. I don't think they need any more marketing for investors.
TheOtherHobbes 9 hours ago [-]
They do, however, need more profitability and less competition.
Hence the "We're curing cancer, no, really" pitch - because within 6-12 months everyone else is going to be doing code and text cheaper and better.
derektank 9 hours ago [-]
>Anthropic is the fastest revenue growing company in the history of Capitalism. I don't think they need any more marketing for investors.
What’s the source for this claim? Is this true in real dollars or just nominal? What about as a percentage of total GDP?
> Anthropic is the fastest revenue growing company in the history of Capitalism. I don't think they need any more marketing for investors.
and people say advertising doesn't work
mrhottakes 9 hours ago [-]
Are you sure about that? They're in the middle of a huge marketing push to get people to buy into their IPO. That's "marketing for investors".
forest32 10 hours ago [-]
You got a serious case of Anthropic Derangement Syndrome
jacquesm 10 hours ago [-]
Check parent comment history.
copemaxxxing 9 hours ago [-]
[flagged]
hn_submit 10 hours ago [-]
It's fairly easy to cure cancer by repairing DNA, but Big Pharma isn't interested because they can't patent it. They want to sell us some kind of "miracle cure" medicine that earns them trillions.
The future is being held back by profit motives.
jayGlow 3 hours ago [-]
if it's that simple why haven't any other countries done this? there are countries with universal healthcare cancer is very resource intensive to treat if they could cure it so easily why haven't they done so already?
Yizahi 9 hours ago [-]
a) they can and will patent it, because in the end it will be same tools "bIg pHormA" uses today for all kinds of tasks. b) they are financially and subjectively interested in developing said tools to combat cancer even if they will be cloned and sold in parallel way cheaper than original. Because - 1) it would be immensely profitable on it's own, cancer is one the most common ailments, 2) post-cancer treatments are bringing in fortunes to pharma companies, because practically any cancer survivor is then locked on some medicine for life, 3) the success story alone, putting for example the most hated person in the world on a covers of literally every magazine and newspaper with a list of honorifics, awards and titles is incredibly enticing to every sociopathic CEO, 4) and as I said, not even generics or lack of crazy long patents can stop a company from becoming gold plated. I can go buy Ozempic generic for a fraction of a cost legally in a nearest apothecary, Novo is still The richest corpo in Europe anyway.
tl;dr - there is no conspiracy. It's just just hard.
PS: correction - not generics, but variants of Ozempic. But the point still stands.
hn_submit 9 hours ago [-]
Repairing cannot be patented. It will be rejected as "trivial" by the Patent Office.
Yes, I know someone patented a "cure" for hemophilia which repairs the DNA of said patient. But that's just because no other pharmaceutical thought it worth their while to fight the patent since the number of patients is small.
But I can guarantee you that a cancer cure patent that repairs the DNA will be fought in Court.
Arainach 7 hours ago [-]
"repairing DNA" isn't patentable, but "a mechanism for altering base pairs using <I won't insult actual chemists by pretending I know plausible terms here>" is absolutely patentable.
hn_submit 5 hours ago [-]
You can call it anything you like but it's still the repairing of DNA. It will be contested in Court and the patent holder will lose.
Arainach 4 hours ago [-]
What's your source for this? This isn't how patent law works.
You can't patent "repairing a road" but you can absolutely patent a new mechanism to do so.
Most patents are of this form - there are far more patents for improved ways of doing things than initial ways of doing things.
I genuinely think Dario is a well intentioned, intelligent dude, but I think him and Anthropic have a huge PR problem and are really out of touch with how they're perceived.
Anthropic in particular has developed this almost Orwellian like veil of condescending rhetoric that on the surface suggests they're looking out for you while underneath they're taking actions that suggest they do not trust you, Mr/Mrs Ordinary Person. All the safety rhetoric, never supporting open weight models, the lockdowns on harnesses outside claude code, etc.
Anthropic if you care about public good, do something to empower people. Release an OSS model. Open source Claude Code. Open source some inference tooling or something. Just give people anything except your words.
tornikeo 12 hours ago [-]
> I genuinely think Dario is a well intentioned, intelligent dude, but I think him and Anthropic have a huge PR problem
Is this what kids call rage bait these days?
reasonableklout 59 minutes ago [-]
It's admittedly a bubble, but Anthropic/Dario still have quite a good reputation among AI tech workers. They are seen as principled and willing to speak up about AI safety and societal impacts, even when it affects their bottom line (such as when being declared a supply chain risk by the Pentagon).
copemaxxxing 9 hours ago [-]
I think it doesn't matter. It doesn't matter whether the creator of AI is a god or a devil. It will be hated anyway.
Because the reasons outlined in the article, plus many others. Concentration of power, unemployment, rising energy/living costs, stealing of digital artifacts owned by people.
I myself acknowledge that AI is useful. But due to the reasons above, I will be anti AI for the most part.
Yes, it doesn't matter if AI manages to cure cancer. Due to the reasons above, I will be anti AI. I suspect a lot of other people feel the same way.
noosphr 11 hours ago [-]
Out of all the families of models anthropic by far has the highest chance of extinction level misalignment during a hypothetical hard take-off because it's trained to act like it knows better than the humans trying to use it.
belval 1 hours ago [-]
Working on a tool that uses their model and the Opus models are all guilty of this.
"Please run N tasks doing X for Y duration, I want to test something"
"Hmm but that would waste tokens, we shouldn't do this test"
What the hell type of product is that? I wasn't asking, I was ordering it to do something, if Anthropic can't get their model to do as I say then I'll just go use someone else's.
anygivnthursday 51 minutes ago [-]
They are also good in the reverse, when you ask and they take your question as a free pass to improvize some code for lulz and extra tokens
jacquesm 1 hours ago [-]
That would exactly be my response if my computer would start telling me what to do. It is really a 2001 variation at that point.
horsawlarway 9 hours ago [-]
Yeah... Anthropic gives off big "I'm sorry, Dave. I'm afraid I can't do that" vibes.
bpodgursky 8 hours ago [-]
This sounds smart until you think about it for ten seconds.
If an ASI model is 100% aligned to user intent then you only need one person on earth to prompt "kill everyone" for an extinction-level invent.
There's no logical way around this. The model either has to ignore the person at the helm or we have to do a multi-national abort before RSI. Anthropic is trying #1.
gunapologist99 50 minutes ago [-]
If everyone or even multiple competing groups have approx the same level of capability, it mitigates the risk to the group as a whole. We see this in biology all the time.
jacquesm 1 hours ago [-]
“Some humans would do anything to see if it was possible to do it. If you put a large switch in some cave somewhere, with a sign on it saying 'End-of-the-World Switch. PLEASE DO NOT TOUCH', the paint wouldn't even have time to dry.”
― Terry Pratchett, Thief of Time
noosphr 1 hours ago [-]
You then need another person to say, please don't, and you're safe again.
written-beyond 12 hours ago [-]
No big tech founder CEO ever was or ever will be well intentioned.
cobbzilla 9 hours ago [-]
Oil CEOs are well intentioned. A different kind of well, but, oh well.
Eddy_Viscosity2 10 hours ago [-]
There are probably many CEO tech founders of small companies that are well intentioned, but our economic system ensures that only the most sociopathic among them become CEOs of the big ones.
7373737373 12 hours ago [-]
And maybe don't allow the military to use your AI WHEN YOU CAN'T TELL WHETHER IT WAS INVOLVED IN THE KILLING OF HUNDREDS OF SCHOOL CHILDREN
noahbp 8 hours ago [-]
They had a near-monopoly on AI-usage by the US military, then had a very public fallout because they refused to allow it to be used for autonomous weapons and mass-surveillance which their main competitor (OpenAI) publicly undermined by immediately signing an unlimited-use agreement, and now they also get flack for this incident they had no control over, because the US military decided to smear them after this fallout?
malfist 10 minutes ago [-]
Should we not judge them by their actions? They selected the children's school as a target. Anything else is irrelevant
perpetualpear 11 hours ago [-]
I might've been bamboozled, but I recal that happening due to outdated satelite images, not a mistake of the AI. Thus, a human would've likely made the same mistake.
mrhottakes 9 hours ago [-]
> Thus, a human would've likely made the same mistake.
The AI industry and all of its sycophants in the Trump administration are working very hard to make you believe this.
There is a huge difference between your product being used for basic military operations like taking notes or getting from A to B, and your products suggesting military operations that kill hundreds of schoolchildren, an obvious and horrific war crime.
yoavm 11 hours ago [-]
It's a mistake. If it was done by an auto-correct mechanism, would we still say "maybe don't sell your word processor to the army if you can't vouch for its auto-correct system!!!"? Besides, "taking notes" can literally be listing targets, and "getting from A to B" can be the road to killing them.
7373737373 10 hours ago [-]
How intelligent and autonomous does a murderbot have to become before you stop absolving its creator and seller from any responsibility?
yoavm 9 hours ago [-]
LLM are not murderbots simply because its a very niche use case. They provide some kind of general intelligence, that many of us use daily to vibe code HTML. If the fact that LLMs are also sometimes used to automate killing then humans should be considered murderbots as well, since few of us engage in the activity of killing (and are autonomous & intelligent). Send the blame to our "creator" then?
7373737373 9 hours ago [-]
You didn't answer my question
yoavm 8 hours ago [-]
Sorry of it wasn't clear. I put the blame on the creator/seller of any murderbot, regardless of its autonomous or intelligence levels. I disagree that what were talking about, Claude, is a murderbot or anything close to it.
digdugdirk 10 hours ago [-]
If the US military purchased camera guided smart munitions with a miscalibrated camera, such that it ignored adult targets and instead specifically targeted children, would that also be a simple mistake per your argument?
yoavm 9 hours ago [-]
If one camera out of thousands has this issue, then yes it's a mistake. If the army doesn't notice it isn't working well after shooting the first child, I'd be learning more towards blaming the army.
tsimionescu 10 hours ago [-]
Again, a product that facilitates the taking of notes, even notes that entail the killing of human beings, is fundamentally different from a product that exists to suggest which human beings to kill (or at least which places to destroy, regardless of whether human beings happen to be there or not).
If MS Word had some kind of systematic bias that made it likely to corrupt the notes taken in it such that they tend to be more bloody battle plans, then that would indeed lead to some responsibility on the MS engineer's part for any mistakes that result from this bug. Similarly for Ford engineers if their vehicles had some bug that made them more likely to lead soldiers riding in them to the wrong destination.
If your system is designed to help choose targets, and it chooses the wrong target, especially with such tragic consequences, then your system is likely not fit for purpose and you should be held responsible for selling it to the army. And that is assuming it didn't actually have some bias + misalignment problem where it intentionally tried to kill children, and selected this school specifically because it gave some plausible deniability. For all we know, it's even possible that some bias like this was known about inside Amthropic and hidden from the public - unless an external investigation is made, that can't be ruled out.
7373737373 11 hours ago [-]
Thou shalt not kill, that's all there is to it really
If you provide tools that help an organization kill easier, and possibly lazily rely on your inaccurate tools' judgment, and you do not care about the outcomes of these tools' uses, that's on your soul, too.
Yes, causality and attribution are hard sometimes, but it may have been that without it, hundreds of humans would be alive right now, and he doesn't even know. Maybe he should talk to the kids' parents about what alignment means.
yoavm 11 hours ago [-]
Ah, I didn't realize we're having a religous discussion. If all there is to it is "thou shalt not kill" then I guess that according to your POV he shouldn't have sold to the military even if the system was only suggesting "valid" targets.
tokioyoyo 8 hours ago [-]
I don’t even think they have a PR problem, if their revenue numbers are correct. Sure, maybe in the court of public opinion, but time and time again it shows how it can be fixed.
denverllc 7 hours ago [-]
Revenue is not profit, their competitors are catching up, and they have a lot of debt.
tokioyoyo 7 hours ago [-]
I’m not that sure if their competitors are catching up in terms of revenue and growth.
And we’ve seen unprofitability not being an issue for a lot of companies. Markets function in funny days nowadays.
snarfy 7 hours ago [-]
I am 100% confident when Dario speaks, Anthropic loses subscriptions.
typ 10 hours ago [-]
> give people anything except your words.
I feel that everyone in this small AI bubble (compared to the "normies") is busy trying to take advantage of AI to surpass everyone else, while fearing being surpassed by everyone else.
Most people assume LLMs are just a tool for making money, an arms race over wealth and social rank. But those like Dario seem to be worried that LLMs could potentially be used as automatic rifles by some dedicated psychopaths. Whether it could be that dangerous, or just an exaggeration, or we should have everyone armed, require licenses to own one, or outright ban them, is debatable. But I can understand his point.
All I can say at the moment is that this is more subtle than the superficial conspiracy/black-and-white narrative.
lxnn 6 hours ago [-]
> Anthropic in particular has developed this almost Orwellian like veil of condescending rhetoric that on the surface suggests they're looking out for you while underneath they're taking actions that suggest they do not trust you, Mr/Mrs Ordinary Person. All the safety rhetoric, never supporting open weight models, the lockdowns on harnesses outside claude code, etc.
This line of reasoning is really very silly. I can be looking out for my young child by choosing not to put them behind the driving wheel of my car.
There are both stupid and malicious people in the world. If you have a tool that you believe may be genuinely dangerous in the hands of the wrong person, it is not the smart thing to do to give the public unrestricted access to that tool. Do you think it would be sensible to give every individual on Earth a nuclear weapon to do whatever they want with?
pu_pe 14 hours ago [-]
> Overall my view is that AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws). Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers).
This is actually a good point. Open models are getting better and better, some of them might even be useful in consumer hardware now. But if AI performance is still correlated with compute power, then no doubt power will remain with the people owning the chips.
podgorniy 13 hours ago [-]
Swap that point about AI with "electricity". Everything runs on electricity it's "a technology that tends to concentrate power" (no pun intended). The electricity providers must be too powerful... But somehow electricity providers aren't that powerful. Unless there is no competition in sight...
The point "AI is structurally a technology that tends to concentrate power" is not that correct. They need this statement to be true, otherwise no way to justify the trillion evaluations.
pjc50 12 hours ago [-]
Swap that to "oil" and everyone goes "well, yes, obviously, oil is so powerful that people start wars over it".
Iran has hit the Amazon data center in Bahrain. The Ukranian deep drone bombing campaign has hit refineries, but also Wildberries, the "Russian Amazon" warehouses. The US campaign against Iran now, Iraq and Serbia previously, targeted power infrastructure. It will obviously be a target in the next war, and AI goes on that list too.
Every time the US completes an AI data center, someone in the Chinese nuclear command updates their target priority list. And vice versa.
mohamedkoubaa 11 hours ago [-]
>Every time the US completes an AI data center, someone in the Chinese nuclear command updates their target priority list
You might have stumbled on the way to convince people that more data centers are a good thing
Iolaum 11 hours ago [-]
Especially with many of those datacenters being built next to residential areas ... </s>
atleastoptimal 6 hours ago [-]
The fact that essential commodity providers aren't the emperors of the universe does not imply the same thing could not be true about AI.
Electricity needs transformation to be useful. AI is the transformation layer, a sufficiently advanced AI system could secure an advantage in practically any economically relevant domain. If you have the best AI in world where embodied robotics are sufficiently advanced, you also have the best doctors, lawyers, engineers, logistics network, PR, weaponry, tactical leverage, etc.
AlexandrB 2 hours ago [-]
You say that AI is transformation layer, then list out physical things that would be necessary to transform AI into something useful such as sufficiently advanced embodied robotics. How is this different than electricity?
atleastoptimal 24 minutes ago [-]
AI would oversee the creation of those physical things, in the same way humans + machines do now
RugnirViking 12 hours ago [-]
but its difficult to use more electricity directly to get more utility. There are some rare ways - things like PtX systems - but even those are linear at best. AI is concentrating power in the sense that nobody wants to use the twentieth smartest AI - so if you can use the most compute, you get an outsized share of the rewards (in terms of paying customers) - over and above your share of the compute. There are other business sectors where similar dynamics exist - where the biggest capital tends to win - a sort of natural monopoly type situation, its nothing about hyperscaling or whatever.
simgt 12 hours ago [-]
Competition is only one of the ways to keep a corporation aligned. In the case of France, having a state monopoly on electricity even worked very well until we broke it for the sake of competition.
fredoliveira 12 hours ago [-]
> The point "AI is structurally a technology that tends to concentrate power" is not that correct. They need this statement to be true, otherwise no way to justify the trillion evaluations.
How is it not true?
The biggest companies in the world are, quite obviously (just look at the numbers) going to be AI companies. The most powerful governments in the world are quite clearly going to be those that are close to (or in control of) AI companies. People have a very hard time seeing the second order effects of the control of intelligence - we need to fix that.
andersonl 11 hours ago [-]
You realize utilities are arguably the most regulated industry in the US, right? Like this isn't just a bad counter, it's the example of necessary regulation due to concentration.
inigyou 12 hours ago [-]
Trump already banned solar and offshore wind, in order to concentrate power (literally) with the oil companies.
XorNot 12 hours ago [-]
Except that's a bad analogy: electricity is the means by which to do work.
The outputs of models in AI datacenters are the work.
The electricity company does not get the entirety of my useful input to the work as a result of me using electricity, but an AI company does.
pas 14 hours ago [-]
Arguably the question is whether it's economically feasible to self-host something similar.
Are you self-hosting Google or Bing? No, but we have quite a huge ecosystem of full-text search tools with PageRank, with options to scale to almost Google scale (if you have the money). After all LLM training starts with the same crawl mechanism.
As long as barriers to entry is not too high (ie. it makes sense to take the risk to start a business that provides something similar - usually for a niche) market forces work.
We have the classic empirical chart reproducing microeconomics.
And setting up a pharma plant is also very capital intensive.
Here the obvious barrier to entry is completely artificial. (Which provides an incentive to spend a lot of money on R&D -- though it naturally raises the question of Pareto efficiency.)
spwa4 13 hours ago [-]
You're missing the point, what will happen is this:
1) in things like tax law, registering with city hall, dealings with the DMV, your phone subscription, insurance contract, ... you will find that one of the new fine prints in the contract will be that you're not allowed to use AI to communicate with them.
2) because of how SynthID works (you need the SynthID keys to verify, which are secret. So the only way to find if text is ChatGPT/Google/Anthropic watermarked is to ask ChatGPT/Google/Anthropic), government and large companies can enforce this against you. That is what the watermark is for. To end any insurance claim written by AI with "you're not allowed to submit AI written insurance claims" and refuse it outright there and then.
"Sorry your request was AI watermarked and pursuant to law 234 of 2025/03/11 chapter 3258 paragraph 33 decile 1299 we hereby close it without response"
3) when they reply, however, they use a custom model that also has custom SynthID keys. You will not even be able to tell their responses are AI written, or at least, you won't be able to prove it. You won't be able to enforce any AI-related rights (ie. the right to talk to a human) you have under the law against large companies.
In other words: this is to make sure that all the advantages AI provides are available to deny your unemployment claim, and to Verizon to charge you more, but completely inaccessible TO YOU when you want to change to a cheaper subscription. They can inundate YOU with AI-written requests BUT YOU CAN'T.
Self-hosting helps because it prevents them from verifying if your responses are AI written, because you can generate non-watermarked AI text and so there is a level playing field.
RandomLensman 13 hours ago [-]
1) Would that fine print be binding?
2) What actual law/regulation would that currently be that would be used for such an outright refusal?
3) Would that comply with current regulation?
cucumber3732842 12 hours ago [-]
Have you ever dealt with a government or government adjacent company?
It'll be spec'd out so that it's cheaper to bend over and take it than take it to court and prove them wrong.
RandomLensman 11 hours ago [-]
What's an example of a government adjacent company? And was that in a civil law country?
cucumber3732842 8 hours ago [-]
>What's an example of a government adjacent company?
Toll road contractors, military base housing, municipal utility providers, private prisons, auto towing/impound, private rail transit operators, utilities, etc.
Basically every company the government grants a captive market to abuses everyone in that captive market as much as they can get away with.
spwa4 12 hours ago [-]
> 1) Would that fine print be binding?
For government, because it's in law or regulations (ministerial decisions in Europe). For large companies "You agreed to it" (you know, like you agreed to allow Verizon to sell your location data to Palantir)
The other 2 questions I don't understand. My point is that the EU AI directive makes this possible. Makes it possible in ONE direction, while prohibiting the other. AI can be used by government and large companies to spam you and deal with you, and can't be used by you without being 100% up front about that to them (ie. enabling refusal)
RandomLensman 12 hours ago [-]
1) Laws need parliament, no?
Just agreeing to it isn't necessarily enough at least in some countries.
2) What laws and sections specifically makes that possible? Are there examples of that happening?
3) Where can I find that interpretation of article 50(?)? Some other article? (To the extent that things would need to be labelled/watermarked etc.)
spwa4 12 hours ago [-]
1) Laws need parliament, no?
No. Here is the list of organizations that have the power to make laws in the EU (and JUST the across-the-EU part of that list, within countries, within states, within provinces, within towns there's another list). This is referred to in legal tradition as the "Hierarchy of norms", because there is also a clear order defined.
I think you need to be more precise then on what you mean by "law" and differentiate there and also by what authorises that.
pas 12 hours ago [-]
this seems exactly the usual anti-consumer bullshit that is regulated state-by-state (or sometimes by (lack of) FCC/FTC effort, or by the CFPB that is now a zombie)
however, AI doesn't really influence this. already there's a lot of problem with things like Ticketmaster, Apple's walled garden, abuses of IP law (patent trolls, DMCA trolls), etc.
the insurance industry is a prime example of this. the suffering caused by power imbalance is incomprehensible, and yet there's not enough political will to address this.
sure, it's easily possible that some important aspects of our everyday lives will be worsened by bad AI regulation. but IMHO this is wholly an upstream problem, it's a symptom of bad politics. (a byproduct of the Zip2 to Tesla to "democracy with roman salute characteristics" pipeline.)
that said, obviously the foundation to have any chance of a nonpatological market to exist is that self-hosting has to be legal.
orbital-decay 12 hours ago [-]
Current AI is at least a few orders of magnitude less efficient than it could be (as evident from biological spiking networks, e.g. human brain). At some point the labs put too much work and money into transformers and nearly abandoned fundamental research, in both ML and hardware. There are tons of low hanging fruits in efficiency but you'll have to redo everything from scratch so nobody bothers. Which is also pretty convenient and lets people like Dario Amodei speak about "natural concentrations of power".
hodgehog11 11 hours ago [-]
The Chinese labs are picking up on the low hanging fruits on efficiency, and no, you do not need to abandon transformers, you just need to push them closer to the more computationally efficient architectures of the past. OpenAI and Google seem to be trying a few things too.
Anthropic clearly are not though, and to call their operations wasteful is an understatement.
woadwarrior01 14 hours ago [-]
I think his point is hand wavy at best. It presupposes infinite scaling and ignores all the algorithmic efficiency wins that are being discovered. Ironically, many of which are being discovered with autoresearch style workflows, using the very LLMs that his company builds.
The #1 post on HN right now[1] is full of people jubilating about how they can run Qwen 3.8 27B on their > 5 year old GPUs. If that isn't democratization of AI, I don't know what is.
I'm sure he's smart enough to instantaneously realize this too, but as the famous Upton Sinclair quote goes, he won't mention it even if he does.
Which percentage of people have GPUs capable of running Qwen3.8 27B? I am one of those, and for my job I am still resorting to hyperscalers because tasks are completed faster and more accurately that way. Even if we assume that models will no longer improve and we reach a point where everyone can run Fable in their laptop, surely running 1000x Fable agents would give you an advantage.
I think access to compute will matter just as much, if not more, as access to models.
woadwarrior01 13 hours ago [-]
NVIDIA's 3090 was released in September 2020. Apple's M1 was released ~2 months later. Anyone with a 5 year old M1 Mac with 32GB or more RAM can run a 4-bit quantized version of Qwen3.8 27B on their machine. AFAICT, there are ~110m Apple Silicon Macs in the world. I'd wager that at least ~20% of those have enough memory to run this model. And if you account for gamers with NVIDIA and AMD cards, I'd wager that the segment is at least an order of magnitude bigger.
> Even if we assume that models will no longer improve and we reach a point where everyone can run Fable in their laptop, surely running 1000x Fable agents would give you an advantage.
IMO, asymptotic advantages are marginal. At least for coding, we got a glimpse into how much of an advantage it gives (or doesn't) when the Claude Code codebase leaked[1] ~4 months ago. :)
As someone typing this on an M1 Mac with 32 GB of RAM who tried using 3.8 27B (Q4_K_M) yesterday in both LM Studio and llama.cpp, I wouldn't call it particularly usable in terms of token speed. (and that was with `--spec-type draft-mtp` for llama.cpp).
If you want to leave it running with the fans going crazy for 40 mins or overnight or something, fair enough, but otherwise it doesn't seem worth it to me. It's certainly not "interactive", even taking into account the over-thinking it does by default.
The 3.6 (maybe they'll release a 3.8?) MoE model is much more usable (but obviously not as good) on this machine spec.
InsideOutSanta 13 hours ago [-]
That's actually an interesting question, and I don't think we have the data to answer it. But we do have the Steam data, and about 7% of Steam users have a GPU that can run it well at a 4-Bit quant (≥24 GB VRAM). About 30% can run a 3-bit quant - I'd say that's just barely usable (≥16 GB VRAM).
I'm not sure whether that's low or high, or how it compares to a general audience.
Sharlin 10 hours ago [-]
The general audience either has a low-performance iGPU generic laptop or only a phone or a tablet.
charcircuit 13 hours ago [-]
Physical constraints like time and compute make it so that AI does not concentrate power due to scaling laws.
HeatrayEnjoyer 13 hours ago [-]
The very definition of (applied) technology is power amplification. Use a lever, move more weight than you could before, 1 person with the tool now wields the power of 3 without.
Making "tech" a career and a societal goal onto itself, without the adjoining understanding of and deep commitment to ethics and the responsible use of power, is why we're sliding into authoritarian rule by a small circle of techno-oligarchs.
We need less "move fast and break things" and more "plant trees you will not live to see bear fruit."
matheusmoreira 8 hours ago [-]
It's a good point but the solution isn't the regulation of AI itself that Dario keeps pushing, he just wants to lock out competitors.
The AI labs don't matter. It's the hardware that matters. Everyone should have access to local AI capable hardware, not just these trillion dollar corporations. We just need to ensure consumers continue to get powerful hardware somehow.
I propose taxing the hell out of these hyperscalers and giving subsidies to individuals and small business to foment local and decentralized AI capabilities. Also ban these "we bought the entire chip production all the way through 2030" deals.
throwaway_7274 10 hours ago [-]
Just want to say, this comment section on this website would have been unthinkable a few years ago. It’s wild: all the engineers have developed class consciousness. I wonder how that happened?
Anyway, it’s fun to watch, but remember to stay sane guys, be aware of the pendulum dynamics.
nitwit005 7 minutes ago [-]
I'd caution that you just have independent groups in some of these comment sections.
You sometimes see people on reddit amazed that "reddit's opinion seems to have changed". Nothing changed, just a different subgroup. You'll see the other opinion again if you wait for another similar article.
For example, the "did bother reading the article" crowd is attracted to certain headlines, and this probably isn't one of those.
seydor 7 hours ago [-]
it's not fun - this forum started with infinite optimism in entrepreneurship, the PG stuff. Then about five years in it turned to "join a BigTech or you ll never be rich", then after AI everyone realized they are on borrowed time and most of them are already outside of the game, if there is a game at all.
gibspaulding 8 hours ago [-]
Wild indeed.
It seems like a couple of years ago the HN and LW commentariat had a lot in common, but they’ve drifted so far in opposite directions that they look more like opposing political parties today.
reducesuffering 7 hours ago [-]
I mean one look at both commentary and it's clear HN has devolved considerably. Just look at the atrocious difference in intelligent reasoning in this thread's comments compared to a LessWrong post right now:
LW stayed selective as a forum, HN became the home of refugees from Reddit. HN reads more like yet-another-subreddit these days than an independent forum.
FWIW I think this is an artifact of site design more than anything else. A commentary site like HN in 2026 encourages hot takes and karma farming. All the "social media" with higher quality commentary like Substack and LessWrong center a large text blob of thought at the top and encourage reading more than writing. This puts up just enough of a barrier to write a hot take that you need to read the whole article and puts enough UI friction from leaving a throwaway angry comment that those folks will scroll away.
HN on the other hand all but encourages shallow posting. The article isn't posted, commentary is more prominent than the article itself. Leaving a comment takes 2-3 clicks and the comment editing flow excludes every other comment on the site, encouraging a write-first mentality. Ranking commentary based on upvotes offers a sweet dopamine hit over collecting karma. In 2026 these all encourage low quality engagement.
HN's UI has stayed relatively the same over the years while the Internet around it hasn't and I think its UI now acts as a low quality commentary attractor. LW changed its site design to stem the tide of low quality comments that show up here. LW comments certainly have their problems, but then, we humans all have problems :) On balance the community is much sharper and much more capable of having intellectually heavyweight, sober discussions on controversial topics. Of course nothing seems to trump my own friend network.
maxnevermind 6 hours ago [-]
> I wonder how that happened?
My explanation is - the technology is amazing and engineers admired it, but CEOs kept talking about an exponential curve while engineers on the ground started to notice signs of Gartner hype cycle curve.
hapticmonkey 58 minutes ago [-]
My take is that engineers were hypnotised by the technology (which is incredible) and they ignored the downstream social/political/economic factors until it became too obvious. Like moths to a flame.
We saw this play out with the social media era. Incredible technology but it’s now ossified into deteriorating closed platforms mostly filled with algorithmic, ragebait slop surrounded by ads. Fool me once.
raincole 10 hours ago [-]
> I wonder how that happened
Layoff.
throwaway_7274 10 hours ago [-]
Rhetorically wondering :)
jacquesm 8 hours ago [-]
Unfortunately it is not just the engineers that get laid off. Try finding a job as a translator, a copy writer or some other entry or mid level service economy kind of job.
throwaway_7274 8 hours ago [-]
It’s unfortunate no matter who!
jacquesm 8 hours ago [-]
Yes, but those other people can not be said to be causing this for themselves.
GPerson 5 hours ago [-]
The early hacker culture in the 1970s inherited the political leanings of 1960s counterculture, so there’s nothing about being into tech that’s in opposition to class consciousness.
simoncion 4 hours ago [-]
> ...there’s nothing about being into tech that’s in opposition to class consciousness.
I agree about the ethos and ethics of early hackers. But, I note that -by volume- most of the folks who do programming and nearly all of the people who are software-industry managers [0] are not hackers.
[0] I'm including managers at all levels, including the executive levels.
jacquesm 3 hours ago [-]
And in spite of the name, most the people on HackerNews.
mekael 4 hours ago [-]
Some of us have always had a class consciousness, we just tend not to be boisterous about it.
Mine is based on two things:
1. The fact that i grew up in poverty and never want anyone else to suffer that fate.
2. I was raised in a religious tradition that can be accurately described as socialist.
nater5000 3 hours ago [-]
I've really only ever interacted with two kinds of engineers with respect to their take on AI:
1) Those with cautious, but enthusiastic, optimism for the technology and an understanding that things never work ideally in tech and any advancement that moves the needle even slightly is worth embracing.
2) Those who are so dogmatically opposed to AI that they leave logical conversation at the door in favor of just repeating whatever negative gotcha they heard last when the topic of AI comes up.
I'll say that the ratio of these engineers seems to be about 10:1 in real life versus 1:10 online, at least in my experience, roughly speaking. I don't think I've actually ever interacted with actual "AI/Tech Bros" who are obnoxiously pro-AI the same way that many of the people who are "anti-AI" come off.
The data I'm working with is biased (I've been working in high tech for a while now), but I'm left to conclude that the people who love to hate on AI online are those who love to hate online while most other people aren't particularly interested in defending a technology online that they feel is inevitable and doesn't really need defending.
nozzlegear 8 hours ago [-]
> be aware of the pendulum dynamics.
What do you mean?
throwaway_7274 8 hours ago [-]
People overcorrect, even when making directionally necessary corrections.
advisedwang 5 hours ago [-]
I think the hyper business oriented, technology first view that is what HN typically has been is actually the exception.
Open Source, hacker culture, scorn of "the suits", the love of sci-fi and it's attendant politics -- the bulk of actual technical people have a long history of being very politically aware and deliberately eschewing money for their values. It was rarely Marxist Class-Conciousness type stuff, but it certainly was aware of the political environment we were in.
Karrot_Kream 5 hours ago [-]
Right, so why did this site revert to the mean then? Just because every noodle place is mediocre doesn't mean I want my noodle place to become mediocre.
advisedwang 4 hours ago [-]
My guess is growth alone explains the change. It went from being a niche site that had views representing that niche to a broad site with broad set of views represented in proportion to their prevelance.
GPerson 10 hours ago [-]
I don’t think my people got my previous post but apparently a lot of diseases were on track to being cured regardless of the invention of LLM based AI. One thing that bothers me about this technology is it makes it easy to attribute all effort to it. This not only hurts the ego but also makes it difficult to reason about how effective it is compared to how effective the people using it are.
Secondly, if you look at what Dario has actually written, then the cure to the big conditions like cancer is in his view hopefully going to be a result of scientists working 10x faster and therefore making big Nobel prize paradigm shifting discoveries 10x faster. So Anthropic suggesting “we are going to cure cancer” doesn’t make a lot of sense to me. According to his original vision it looks like a bunch of groups now using AI independently producing breakthroughs which cure cancer.
glimshe 14 hours ago [-]
One thing Dario said is important reflecting on (but from another angle): where's the big deliverable from AI? If AI makes us 10x more productive, the 3 years since its popularization were enough for a product that would have taken 30 years to build without AI, for instance.
I'm an AI advocate but that question makes me feel that AI is simply "very useful" rather than being a historical game changer for humanity.
ben_w 14 hours ago [-]
Anyone saying 10x as a serious claim is clearly using a round number and vibes; however, even if it were so, AI getting popular 3 years ago does not mean what you say.
The earliest used models in this category would easily 10x (what did I just say) the creation of one-off short scripts, but you're not writing a 30-year app out of just a bunch of short scripts.
The METR time horizons graph suggests we can now get 10x (ahem) speedup on solving most coding problems that take us a few hours and about half of problems that take 2 days. Amdahl's law bites: even infinity speedup on half your problems is only 2x overall.
If you let an LLM loose, with a huge budget, what's the biggest artefact it can make before it drowns under the weight of bad decisions? The C compiler and web browser headlines a while back? The maths papers we see now that solve problems which stumped the maths world for decades?
But this is the other side of the same coin: teamwork. One person getting a thing made in twelve months vs a team of a hundred, you can scale up fast with money when you have a proof of concept, and an LLM can make a lot of proofs of a lot of concepts even in free accounts.
pas 14 hours ago [-]
LLM-assisted software engineering seems to be very efficient if you have a deterministic target. (bun rewrite from zig to Rust, 100% Node.js compatibility, pnpm compatibility -- https://github.com/oven-sh/bun/pull/38333)
croon 12 hours ago [-]
My pushback on that would be that all of the "engineering" concering a rewrite was already done the first time around. It's a glorified translation.
I do agree that the port would've taken a lot longer without LLMs though.
QwenGlazer9000 10 hours ago [-]
Problem with that is that it's not proper rust code, it's some kind of franken zig style rust port.
The actual hard part is getting it into idiomatic, safe rust, and I don't believe the LLM port makes the full transition any easier than doing things the old fashioned way: a dual lang code base like Linux.
pas 8 hours ago [-]
> I don't believe the LLM port makes the full transition any easier than doing things the old fashioned way: a dual lang code base like Linux
...
how so?
let's say Linus opens a branch, rust-temp, and in 2 weeks pushes ~15 million lines of code deleting most of the old C code. and then it gets merged in a few weeks. and then there's still a few months of the "merge window" and RC process. and each day folks report bugs, and automatic fuzzers make sure that both versions "behave the same".
I was pretty skeptical (still am), but what the Bun team did is pretty great so far.
https://bun.com/blog/bun-in-rust details the process, we see the results (Node.js compat [0], more than 3 thousand of issues fixed since since 1.3 [1], more than 900 issues fixed in ~24 hours [2], and it's live/in-prod [3])
> 50 dynamic workflows in Claude Code run continuously over the course of 11 days.
so let's say 500 workflows could do something similar for the kernel. the reported cost is 165K USD, so let's say this would cost 1.65M USD, plus CI costs [4] (which is ~200K/year for bun, so let's say it's 2M USD for the kernel)
How much the world is spending on kernel bug bounties and various security programs each year? (rough estimate says that just the visible kernel testing programs cost at least 15-30M / year)
The bar is not perfect. The bar is something better.
Because they are still dealing with memory safety issues to this day. They ported it to rust, completely introduced a bunch of bugs, went through all that effort, just for the codebase to be not much safer.
I'm actually basing my opinion off the blog post, specifically the code. While they do get a few changes for free with the port, most of the code looks identical. And the number of unsafe statements certainly agrees.
And now they're going little by little, playing whack a mole with seg faults... My point is that the step they're at right here is the important part, and that mechanically converting the entire codebase to rust was an optional step when incrementally rewriting would've worked just as well if not better.
Being "only" 165k USD doesn't mean anything, because you didn't do a complete port, you did a port in a trench coat. If they went for the more targeted module by module approach, and did a clean and proper rewrite, LLM assisted or not, they would reach their end goal much less turbulently and without the effort of that initial frankenport.
pas 7 hours ago [-]
are they dealing with more or less memory safety bugs?
for me it was already "don't run in prod" quality before. (at least after this I'm considering adding it to the CI to see how it fares.)
segfaults. I don't know. I looked at their CI and GH issues over the past few weeks. (though now GH is down so I can't do a search, but I didn't see thousands of segfaults.)
by all accounts and measures it seems it made their house of cards more manageable. despite the frankenport, no?
they already had a zig compiler fork, wanted to upstream it, but the zig maintainer(s) said it's low-effort. now they don't have to maintain their compiler fork. (or wait for the zig team to deliver the features they wish for.) no need to maintain a hybrid codebase. (though it has C++ because of the embedded JSC.)
also I have no idea what's the zig-rust FFI status, but getting over with a rewrite faster is usually better, even if you are left with non-idiomatic code.
jacquesm 8 hours ago [-]
Great example. There is no way I would run that in production anywhere.
Long term stability is the thing that matters, such a move would be immensely threatening to long term stability.
pas 7 hours ago [-]
the neat part is that you don't have to, millions will do it for you!
the point is that it's a quite obvious opportunity that was just some pipedream years ago.
by the time v8.0 comes around we'll have more data on the bun rewrite. also we'll see how LLMs will affect the productivity of kernel developers, and the overall stability/maintainability.
QwenGlazer9000 7 hours ago [-]
> the neat part is that you don't have to, millions will do it for you!
You're not wrong. We're already seeing something similar with all the outages and buggy software updates these days.
We little choice in the matter, you will have ze (software) bugs!
dist-epoch 10 hours ago [-]
And compiler generated assembly code is ugly and spaghetti, unlike beautiful hand-crafted human written assembly code, the kind you see in ffmpeg codecs, ...
Doesn't matter.
QwenGlazer9000 9 hours ago [-]
It does matter, because that's the entire point of the language. You know, memory safety?
The vibe coded rust port did not get them any closer to full compiler verified memory safety. They still need to go file by file, bit by bit, and make it memory safe, at which point, why not just do that from zig.
pas 8 hours ago [-]
> at which point, why not just do that from zig.
because for the same effort you get a better result if you can and do use a better tool?
QwenGlazer9000 8 hours ago [-]
I didn't say don't use LLMs. In fact, I'm quite certain they could help quite a bit.
But I don't think they did it in a way that actually provided a meaningful gain. My point is that the state of the codebase right after the code is just a worse version of the original zig code with few of the rust advantages. Going from that Frankenstein rust code to actual memory safe rust is a similar leap to going from zig.
Sharlin 10 hours ago [-]
And how long will it take to rewrite the Rust rewrite to idiomatic safe Rust?
stnikolauswagne 14 hours ago [-]
Can only speak for my own project, but can give an example:
After an aquisition earlier this year I got the task of doing an SAP-Integration for the new company, last time I did this 5 years ago it was a 6 month task, but with the experience and skills ive gained since I estimated it would be a 3 month project (with or without AI, most work is just logistics, AI cant help much there).
In those 3 months I was able to not only integrate SAP but also deliver a completely modernised user-facing software for that integration. While I could have written that software myself in a vacuum it would have never been worth it financially, since it would have delayed the launch of the integration by 6+ months. Building the software post-launch of the integration would have easily taken 2.5 years at minimum.
But this is also basically a "spherical cow in a vacuum" scenario, where I was essentially acting as a solo dev, in full operational control of the project, with deep domain knowledge of the topic and an allready fully set up codebase that I knew perfectly while working down ideas I've had in my backlog for 5+ years.
jatins 11 hours ago [-]
> Building the software post-launch of the integration would have easily taken 2.5 years at minimum.
What you have said is correct, it lets you build software much faster. The question however is: is that software making money for the company? (Not talking about what you built but in general)
I think, with AI, companies are saying yes to a lot of things they would have said No to ik say 2020. And as a result realizing “just building it” is not the answer.
Previously your GTM team or Product team would say “If we ship some big project X, we unlock $Y in revenue” but now people are realizing that those projections were really more of a hope. So companies are spending so much more tokens and shipping so many more PRs based on hope but a lot of it just doesn’t turn into meaningful revenue, especially not in short term
stnikolauswagne 11 hours ago [-]
Yeah, thats why I meant I was in a spherical cow-situation, this was a situation where we took over a company in the exact same field and I had 5 years of experience on which tweaks would need to be made to be able to rationalise away positions.
This sort of system only works with internal software and an unusual amount of data. If we were in the business of selling that software we could not have charged a higher price for the new version over the old, the tweak could only be unlocked because we were able to control staff hiring and staff onboarding fully to make use of the new changes.
jacquesm 11 hours ago [-]
And it quite possibly creates a huge risk for the company in the longer term, there is only one person who knows how that all works.
stnikolauswagne 10 hours ago [-]
Honestly not super sure if that is more of a risk than it was pre-llm.
As part of the aquisition I got access to their previous codebase which was some sort of incomprehensible PHP monolith, with the persons who wrote that code long gone. Thanks to LLMs I was actually able to extract the core useful concepts (again, sufficiently deep domain knowledge that I knew exactly what to look for). Without LLMs i would have probably extracted the absolute minimum and let the rest rot.
There is no reason a dev of comparable skill and domain knowledge would not be able to do that for what I built here.
jacquesm 10 hours ago [-]
But that codebase was written by people, so you'd be well positioned to take it over. I wonder if the same would be the case if you were to take over a codebase that was written by AI.
By that point it might no longer matter though, but I suspect that such code would have a lot more exposed edge cases than one where someone actually thought things through before coding.
Sol- 11 hours ago [-]
Definitely something I've come around to as well, even though I am very pro-AI and think it's an amazing technology. But the bottlenecks just hit too hard. I notice every day where my usage of AI is so constrained, by myself and my imagination, by external factors like slow feedback loops (gathering requirements), by the AI still not being good enough for certain details (you need to heavily steer it, give feedback, even in long running agentic sessions).
So yes, many aspects of my job are now 10x as productive, but turns out that improves my overall throughput only very little.
blfr 14 hours ago [-]
First, it's not three years since. The real improvements in programming ability arrived in the last 6-8 months.
Second, the Internet didn't show up much in GDP and similar measures either!
But your point stands. Where are the amazing digital products/stuff? I get that it might take time to arrive as we scale up compute and learn new paradigms. But so much infra already exists (deployment pipeliens, everyone reachable on a smartphone) that we should be seeing something.
ben_w 12 hours ago [-]
> But your point stands. Where are the amazing digital products/stuff? I get that it might take time to arrive as we scale up compute and learn new paradigms. But so much infra already exists (deployment pipeliens, everyone reachable on a smartphone) that we should be seeing something.
I'm definitely seeing indie-sized games that appear to have had significant input from AI, though I'm not sure the balance between AI for coding and AI for assets. My experience attempting this directly suggests that the current level they work at can make very simple games as one-shots, but anything more than trivial will produce outputs only as good as the developer's combined willingness to put in effort tweaking things and taking it all one step at a time, and their taste about what "good" even is.
I'm using spare credits to build and improve an isochrone map renderer, which I otherwise wouldn't have had time for (apart from anything else, I'd have had to become skilled in JS+wasm, somewhat of a pivot from iOS). This also requires taking it all one step at a time, having UX and UI taste.
Having lived through GeoCities since before it was bought by Yahoo!, taste is… well. Most people make things that nobody else actually wants.
KronisLV 12 hours ago [-]
> Where are the amazing digital products/stuff?
I've done amounts of refactoring and fixes and written tooling that just wouldn't have happened before.
I'm not sure what amazing new stuff y'all expect but the amount of technical debt in my projects is actually going down, cause I can finally get good enough test coverage, including E2E/load tests that actually prove whether the software works and scales or doesn't - just last week I diagnosed issues with SeaweedFS failing under concurrent writes when backing Sentry and could swap it out for Garage in a day, caught by a monitoring tool I slopped together that integrates with the Sentry API, no issues since.
The environment around me has gone from drowning in tech/ops debt to sort of swimming and at least holding above water for now (cause nobody will pay for 5x more tokens).
It's also insanely good for prototyping and being able to actually explore various ideas and shoot the bad ones down quickly instead of handwaving and looking at a loaded calendar, alongside being able to address well bounded tasks in parallel, better than human developers can - like I can give 5 GitHub issues to the slop machine and have it fix all of the annoying bugs. Issue with how some data shows up? Just feed it the DB dump and let it find out what's up.
Some projects have gone from around 500 code tests to around 4000, and before anyone says they're meaningless, at least 5% of those have caught real issues and helped a bunch, alongside linters and other tooling (including some tools I wrote myself). I've also written both native utilities and some web platforms for myself, side projects that I never would have gotten around to.
I'm measurably more productive than I've ever been (since I did measure that, looking at my commits over the last 2 years) but also burnt out. Still, it's the kind of burnout that's the consequence of context switching and lots of work, rather than the kind that I had years ago, where I had to manually untangle deeply nested Spring Boot service logic all over the place at like 2 AM cause the made up deadlines were kicking my butt.
In contrast to others, I don't need to move the goalposts - the productivity for me is here and now. Any future models will just make it better, unless we experience model collapse.
Disclaimer: you do need a LOT of code tests and validations, otherwise it all goes to shit. Maybe I'm just extending how much time it will be until it goes to shit for me as well, but go figure. You also have to babysit the models more than anyone would like or should, most of my work usually has 20-60 minutes of planning before dispatching the agent.
Sharlin 10 hours ago [-]
For that you need a management that actually lets you use the token budget for stuff like refactoring, fixes, and test coverage. Unfortunately I'd wager that's an exception and the rule is (still) that to most managers LLMs are "feature machine go BRRRRT", leaving even less time for anything else besides pushing out features at a breakneck speed because you're supposed to be 10x more efficient now.
KronisLV 7 hours ago [-]
Don’t ask for permission then?
Implement features at a viable pace and drown any unrealistic expectations in reports of CVE fixes - usually a version bump of dependencies, but quite often also some migration work within the app; write tools to make that work visible like I did with scanning and reports, and enjoy living without having to complain about how you can’t keep up with releases while letting your software rot until it’s a multi-year migration.
Alongside that, do the same for test coverage as well because it’s the humans that have been sloppy for years and didn’t work enough on the test harnesses or good enough coverage, introduce Testcontainers and load testing tools and anything else that makes sense.
Use their money for tokens to build software that is actually good and make your life easier and ensure that their shortsightedness won’t sink the product on a few years.
Of course you could also get fired if they’re stupid, but then I don’t imagine that working for those that are delusional is exactly great either, unless the goal is explicitly to make slop to get investor money in which case the whole thing will probably need to be rewritten from scratch anyways so who cares.
dandellion 14 hours ago [-]
I remember people moving the goalposts like that since, let me check... when did AI Dungeon go viral? Wow, 2019.
blfr 14 hours ago [-]
What's moving the goalposts? I am very much amazed at what Fable can do. I push its code straight to prod.
But I am just as amazed with how little real life consequence it seems to have! Even software houses were hit more by interest rates than by this magical revolution.
If I couldn't directly observe Fable in action, I wouldn't believe in AI.
RGamma 10 hours ago [-]
I read the lack of real-life consequences as a sign that gains through digital technology as a whole become ever more marginal.
What would produce huge "gains" (depends on who you ask) again is the automation of the myriad non-digital roles/companies, but good luck with that...
well_ackshually 13 hours ago [-]
> What's moving the goalposts? I am very much amazed at what Fable can do. I push its code straight to prod.
>> What's moving the goalposts? I am very much amazed at what Opus 4.8 can do. I push its code straight to prod.
>>> What's moving the goalposts? I am very much amazed at what Opus 4.6 can do. I push its code straight to prod.
>>>> What's moving the goalposts? I am very much amazed at what GPT5 can do. I push its code straight to prod.
>>>>>> What's moving the goalposts? I am very much amazed at what Opus 3.5 can do. I push its code straight to prod.
blfr 12 hours ago [-]
I am definitely not impressed by Opus. Quite the opposite usually.
simoncion 4 hours ago [-]
Yep. I've been hearing «Look, we only got tools that are really capable in the past 6->8 months... everything prior to that was unusable garbage, so anything anyone said about the field prior to that is totally invalid.» for years now.
onion2k 14 hours ago [-]
I think it's more that it shifts the thought process from "If this is going to take 30 years then we won't bother, because the investment can be spent on things that pay off sooner" to "If we can do this in 3 years then we'll make that investment because that's a good bet."
The way it changes the game is by lowering the cost of making radical bets so we end up trying more moonshots.
dgellow 13 hours ago [-]
Do you have examples of such moonshots? Ideally not themselves AI related
ReptileMan 14 hours ago [-]
Right now we are in the golden age where we do the same and take time off. Employers have not yet fully caught up with the workforce. I can't think of people that are not putting less hours this year for the same salaries.
margorczynski 13 hours ago [-]
The questions is what happens when they catch up. They'll cut like 50%+ of the workforce? What happens then to the demand that makes their companies work?
Or an example of MS - their main cost like most software companies are people, especially software devs, which are to be replaced by AI so on the surface they would greatly benefit from it. But their products are centered around helping out people do stuff on the computer. Why would you need that when the AI will do it better and faster directly operating on the data or using e.g. Python?
ReptileMan 13 hours ago [-]
Who knows. What I tell is that the AI productivity boom is here. Just for a change right now it is not shown on businesses balance sheets, because it is captured by the workforce in non monetary ways.
moomoo11 5 hours ago [-]
here's how I think of it.
Maximalists: 10x
Minimalists: were always close to 0x individuals, so opinion rejected 99% of the time
Realists: 30-50% productivity gains
shimman 4 hours ago [-]
50% productivity gains would be seen in real economic data tho, you're talk about potentially $5-15 trillion in real GDP gains; all we are seeing is just more mass consolidation of wealth into the hands of the few, we're even at the stage were we are designating extinction zones to empower these individuals even more.
zer00eyz 9 hours ago [-]
> where's the big deliverable from AI ... AI is simply "very useful" rather than being a historical game changer for humanity.
The last 30 years of tech has been about big, the dot com boom, the next unicorn, the magnificent 7 or 11 or whatever.
The thing is that the missing solutions people want have never been the one sized fits all multi Tennant we need 6 rounds of seed funding kinds of problems.
The "app boom" should have been a clue that smaller, niche services are viable.
IF tech were like construction, then most of HN, most of SF Bay Area, most of "tech" seems to be concerned with building the tallest building, or a massive housing development. Your average office worker is more akin to a home owner, who has a busted outlet, a sticky door, a dripping faucet, they just work around it because the resources to fix it (programers) are going to build the next big thing.
Those workers now have tools to address those problems, themselves.
I have lawyers and writers (far removed from tech) talking to me about cron jobs and scripts they are running that solve real problems for them. AI has allowed them to accomplish this.
These things arent showing up on GitHub. They dont make quarterly reports at a publicly traded company. The mountain(s) of change are happening by stacking grains of sand, not one solid rock.
mrhottakes 9 hours ago [-]
So then shouldn't someone be able to see these huge mountains of grains of sand?
zer00eyz 8 hours ago [-]
Hey boss, I just wrote a script that saves me 3 hours of work a week, you got anything for me to do?
Hey, I just found 3 niche businesses that I can run because of AI and clear a million a year after costs.
Fight club rules apply, you dont talk about your advantages.
You get lawyers talking about their AI use (sometimes) because they are protected by the requirements of the bar. You people in niche businesses, or where it's about personal productivity, talking about it cause you cant replicate what they have done. You get folks in the sciences talking about how they are using LLM's to generate the python & scripts that make their world function with easy (rather than be a bottomless pit of time). These are people who might talk about their AI gains, but often have better things to do and don't run in programing / tech circles.
Unless you seek these people out you're not going to find their gains. It would be pretty hard for someone like anthropic to even begin to quantify time savings from generated code by "non programers" - because they don't have the data.
arational 7 hours ago [-]
[dead]
mschuster91 14 hours ago [-]
> If AI makes us 10x more productive, the 3 years since its popularization were enough for a product that would have taken 30 years to build without AI, for instance.
A product still requires a lot of handholding and human thinking, at least if one does not want everyone even throwing a glance at it to immediately be repulsed by the usual AI slop tells.
> I'm an AI advocate but that question makes me feel that AI is simply "very useful" rather than being a historical game changer for humanity.
It absolutely already is a historical game changer on par with the Industrial Revolution when it comes to the amount of jobs destroyed and economies screwed up - and the impact will be even worse in 10+ years as existing seniors retire but no new seniors rise as AI has destroyed entry level career paths.
afavour 14 hours ago [-]
> It absolutely already is a historical game changer on par with the Industrial Revolution when it comes to the amount of jobs destroyed and economies screwed up
Which economies are already screwed up?
IMO it can’t ever be on par with the Industrial Revolution because AI can only really affect the information economy. Things people do with their hands/bodies have either already been automated or can’t be with current tech. If you’d asked people decades ago they might say no one will ever work in factories by 2026 because they’ll all be automated. It didn’t work out that way. I think AI will go the same way: absolutely game changing to some industries (of which software engineering will be one) but a great many will still survive with less dramatic changes.
If anything it might result in more focus on the human aspects. How many people out there earn their stripes putting together slide decks? In a world where an AI can put together the snazziest presentation you’ve ever seen in a heartbeat it’s going to matter more how you stand at the front of the room and present those slides than it does today.
glimshe 14 hours ago [-]
Not disagreeing with you, but adding to my argument. Despite the human handholding, I feel that 3 years would have been enough for 18 months of thinking about the product plus 18 months where a team could get 3-5 years worth of coding/development done. Yet, we haven't seen anything big yet, like a new Youtube/Instragram, an amazing videogame, a major cure etc. Maybe they are coming, but every day that passes is an indication that the net productivity positive of the technology isn't as big as advertised. In my own personal use of AI, I experienced a 30-50% increase in productivity, but no more than that.
lukeschlather 14 hours ago [-]
It's only been at most the past year where AI has been unambiguously helpful and not a hindrance. 3 years ago it gave the appearance of being helpful but it tended to be more of a hindrance.
nacozarina 14 hours ago [-]
perhaps being 30-50% ‘busier’ is something quite different from being 30-50% more ‘productive’ and ppl are just bad at detecting the difference
glimshe 13 hours ago [-]
Maybe so, but least for me in my personal coding projects, I'm going through my own task list noticeably faster. I had created this list before getting a ChatGPT Pro subscription and the number of bugs found in AI reviews and rate of closing tasks has certainly increased. I'm not saying my anecdote scales to teams or even other people, but I have no doubt about my personal productivity change. I wouldn't be paying for it otherwise.
creshal 12 hours ago [-]
IMO the constraint is that AI is still rather anaemic at sustainable green-field projects: It's good at one-off oneshots, and also at refactoring or fixing bugs or adding features to existing projects, where test suites and significant architectural scaffolding already exists, but the more you move away from that, the more wobbly the results get, and the more the human once again becomes the bottleneck, for all the hard work of coming up with all the conceptual scaffolding in the first place. Typing speed rarely was the bottleneck there anyway.
xscott 12 hours ago [-]
Any argument about regulation in the US which doesn't mention that China and other countries aren't bound by that regulation should be heavily questioned. Exactly who are you stopping from doing what you don't like?!?
Law abiding US citizens won't be able to run them, but you didn't solve anything other than making sure US citizens pay Sam or Dario.
shimman 4 hours ago [-]
As a US citizen I do not care about regulating China, they can figure out themselves. Just as they figured out how to imprison their corrupt CEOs, I'm confident they won't let corporations destroy their society like the US.
I rather let citizens of my country exercise our own authority and agency when deciding what is necessary for good societal outcomes (LLMs aren't necessary).
blitzar 12 hours ago [-]
Any arguments that US laws or regulations should apply to people in other countries should be heavily questioned.
xscott 11 hours ago [-]
That's definitely not an argument I was making.
w10-1 2 hours ago [-]
> by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world. That is totally on us
I don't think it's on them. Proximally, it's not their job to deliver the end-use value; it's their customers' job (if they're doing it, it would be a strong sign of tying). More generally, they operate as a commercial SaaS entity (non-profit or not), and net benefit to the world is not what any company is structured for (even public benefit corporations have mainly indirect effects).
So I think that "criticism" is just a back-handed way of promoting themselves in messianic terms.
IMHO Anthropic, OpenAI et al continue the practices that make people resent companies (US companies): You have to be either connected or nominal (show of diversity) to actually participate in the opportunities or protect yourself from their predations. That's just the nature of knowledge work: you need a ladder of opportunities that start very early to participate, and you'd need to understand subtleties of engagement to see how you're being fleeced as a customer. The divide of rich to poor is not about money but both knowledge and access; it's felt even by long-time experts in the field who are left out or ousted.
jbs789 9 hours ago [-]
I am skeptical of David. But he is spot on with this comment:
“Yet Dario blames public negativity on a long-standing loss of trust in institutions rather than his own messaging.“
Dario’s arrogance and blind spots are really something to behold.
cucumber3732842 8 hours ago [-]
I think he's on to something.
Nobody is worried about any AI company in the abstract the way people are worried about other companies in the abstract (e.g. an oil company might ruin the environment just simply as a byproduct of engaging in their core business).
Everybody worried about their government, their utility companies, their employer, their landlord, all the faceless BigCos they do business with, etc, etc. getting their grubby mitts on AI and then using it to screw them even more than they're already being screwed.
The whole system is set up to screw people right now and it does it as fast and as hard as it can. AI is see as bad to the extent that it lets the screwing happen at a higher tempo.
ben8bit 10 hours ago [-]
I really liked their brand position a year ago. But having Boris talk about "Claude's brain" on stage & their other founder pretending AI is sentient in front of the pope - they're coming off more as a religion than a tech company (and this is aside from the obvious model issues they're having).
seydor 7 hours ago [-]
That s the thing with all US companies (chinese cos don't seem to go metaphysical)
kingstnap 14 hours ago [-]
> Overall my view is that AI is structurally a technology that tends to concentrate power
> Open-weights do help some with this but are nowhere near a sufficient solution
Which is exactly why Anthropic contributes nothing to, and actively pushes for roadblocks and regulations for open weight models. Can't allow any hope to the masses.
Only people with $$$$$ are allowed to touch Fable. Which of course we have have aggressive guardrails for in case you even try to use it for something dangerous like AI model developement. Plus we are going to retain all data submitted to it just in case someone is trying to be sneaky.
quikoa 8 hours ago [-]
> something dangerous like AI model developement
It's very dangerous to Anthropic's bottomline!
simoncion 4 hours ago [-]
> Only people with $$$$$ are allowed to touch Fable.
This was primarily a clever way to try to shore up their revenue numbers. By preventing folks who don't have seven-to-twelve-figure bank account balances from using the tools that were most expensive to build, they might actually stand to make money from those tools... once they're forced to do accounting that includes the cost to make those tools.
As a bonus, R&D cost for serving the fixed-price-subscription rabble can be massively reduced by providing them with older tools that have had the goop around them tweaked just enough to make the tool's output have a different "flavor". That class of customer will never know that they're being conned.
Depurator 14 hours ago [-]
It will be interesting to see what kind of progress in biology and medicine they will preview during the fall. He does make a good point that most of the progress so far have not been material in the sense of providing real positive outcomes for ordinary people.
Or even for the HN crowd, when will we see e.g. "Mythos aided research discovers 10 new viable battery technologies"
QwenGlazer9000 10 hours ago [-]
> He does make a good point that most of the progress so far have not been material in the sense of providing real positive outcomes for ordinary people.
Even worse, it has made a significant number of people's lives worse. Some major, like layoffs, some less major, like PC prices, some are just annoyed by AI slop.
And then Dario himself wouldn't stop yapping about white collar work being gone in X years...
rimliu 13 hours ago [-]
None?
Depurator 12 hours ago [-]
Fair enough, help navigating complex and bureaucratic processes, personal validation and a more tailored search engine do provide some value
polytely 2 hours ago [-]
What is the lesswrong-crowd answer to the non-existential risk to working people?
I'm way more worried about these guys breaking the economy in such a way that enriches them and makes me poorer, than I'm about any existential risk these guys cook up in their thought experiments.
Like if you are making more money than god in silicon valley I guess you don't really fear losing your job or house, but for most people that would be quite bad.
I never hear these guys articulate what their concrete vision for the economy if they make ASI / AGI happen (one of those is the correct shibboleth and I don't remember which).
jacquesm 2 hours ago [-]
This comment exactly nails it. That's the feeling that this all gives me. Sorry for the inconvenience, but we just had to bankrupt all of you to save humanity...
AnodicElegy 9 hours ago [-]
"I wrote Machines of Loving Grace because I didn’t feel the AI industry was painting an inspiring enough picture of how the technology could radically transform the world for the better. The bulk of the essay is devoted to refuting skepticism of AI’s potential in health and biology, and showing why I think it will actually be possible to cure most human disease in ~5-10 years, as crazy as it may sound to ordinary people and frankly to biologists as well (I used to be one!). And, if you read my most recent essay (Policy on the AI Exponential), I discuss concrete proposals for how to streamline the FDA process to make sure the deluge of AI-accelerated drugs isn’t slowed down by the regulatory process."
This is fantasy, and I believe the vast majority of those actually working in pharmaceutical R&D would agree. Contrast Amodei's opinion here to the Derek Lowe post I shared recently: https://news.ycombinator.com/item?id=49313367 . Derek Lowe's opinion is closest to my own experience: AI, whether it be traditional ML or LLMs, can help here and there -- the former to help you to triage paths to explore in a way that's a little better than intuition in some cases, the latter mostly to generate code faster in the code-dependent aspects of pharma research -- but neither of these things are significantly widening the main bottlenecks. I don't think the data exists to do so, especially since so much of pharma R&D is looking for higher and higher hanging fruit (i.e. exploring avenues for which a trove of relevant training data does not already exist).
seydor 7 hours ago [-]
We need proof that AI can climb the vertical axis and is not just horizontally expanding on what we already know. ANNs (including LLMs) are approximators of things that already exist , but science occasionally needs paradigm shifts and we have not yet seen any of it with AI.
mbgerring 3 hours ago [-]
I don’t think it’s a messaging problem or a trust problem. I think it’s that the world AI boosters want to create is one that most people don’t want to live in. “You will be fundamentally without agency or purpose, and people you don’t like will shape the world you live in forever according to their whims” sounds bad to most people. Having cures for disease so that I can live longer in a world that I’d rather not live in at all doesn’t sound like a good trade.
jacquesm 2 hours ago [-]
They might want to live in it but they won't be able to afford it.
__MatrixMan__ 8 hours ago [-]
> I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer.
The way to do this is with large automated wetlabs running experiments and generating novel training data on the behavior of cells.
It's with partnerships with governments and universities that educate the masses to the point where many of us are better able to contribute towards the project of growing the available knowledge in a useful way.
It's with more NIH funding and fewer anti-science propagandists attacking that funding so they can sell their snake oil without pesky facts getting in the way.
What's actually happening is our data centers are spewing carcinogens into places where humans will breathe them and we're lying to ourselves about how if we just keep going this way we'll cure cancer.
Stop making the world less habitable by trying to squeeze another drop of model utility out of the existing knowledge, and start helping with the creation of new knowledge.
It's going to take decades to ramp up. Let's turn the page and get started.
seydor 12 hours ago [-]
Is this cure of cancer in the room with us?
danqqqq 12 hours ago [-]
[dead]
isoprophlex 14 hours ago [-]
> "I know that there’s a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I’ve always found this to be an overly simplified picture of the world."
> wall of text follows
> doesnt proceed to clearly tell us what the actual picture of the world is, then
am i correct in summarizing that the line of reasoning is
- frontier llm access means you are at an economic advantage
- a big risk of this is ongoing wealth concentration
- the "open weights" approach can't solve the problem of wealth and llm access being linked; you need compute too, and compute is expensive, thus "open weights" still favors the wealthy
- instead we need "objective and fair institutional processes", as this will allow small labs cook up their stuff while frontier labs get regulated
why would i care about what these smaller players do, if economic advantage = frontier model access? also, the reasoning around "why bother with open weights because compute is expensive too" seems like the kind of logic a motivated 12 year old could work their way around in 30 seconds. things are not either/or dario, you said so yourself.
seems like a 400 word corpo misdirection essay. par for the course.
knollimar 14 hours ago [-]
He seems to be grasping for a point but not providing argument for what it is exactly. And certainly not support that is logically coherent
coldtea 14 hours ago [-]
> I know that there’s a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I’ve always found this to be an overly simplified picture of the world.
To paraphrase Sinclair "It's difficult to make a man realize the issues with regulatory capture, if they're to be the benefactor of regulatory capture".
> A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice.
Not so sure. After all lack of the latter did help establish the current tech overlord rule.
meindnoch 11 hours ago [-]
> A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice.
Especially when said individuals are rich.
alansaber 9 hours ago [-]
Vast majority of AI headlines are 1. bad or 2. gimmicks or 3. remarkably charitable forward thinking- this is self inflicted
maeln 14 hours ago [-]
> This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while advantaging smaller competitors.
> ...
> completely exempt any company below a certain amount of revenue or model training costs from being covered at all
One could argue that "Frontier AI" company know they have nothing to fear from company with less than XM$ revenue, and so their support for this type of regulation is still a way to force regulatory capture. In any case, whatever regulation you support, it's a regulation that you didn't have to handle when you were growing, but that incumbent will have to deal with.
Whatever it is Dario's intention or not does not matter. Capitalism push to consolidation and the eventual regulation that will need to be applied to mitigate the externality from a new industry, will mean that their will only be a handful of "very big" winner. Same story since the beginning of the industrial era. Even if that's not what Dario personally want, it's in the best interest of the shareholders, which will force Anthropic to do everything it can to be one of the big one.
> Overall my view is that AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws).
The same can be said about a lot of other industry (aviation, oil, chip manufacturing, ...). Every country / union big enough will finance their own champion to try to keep a foot in the industry even if they are not the best.
axegon_ 14 hours ago [-]
Called it! He was bound to squeal after the release of QWEN 3.8 in one way or another and here it is. I wasn't even a teenager but I'm getting so much Jobs/Ballmer flashbacks with internet explorer and microsoft office.
Edit: Does anyone else notice the switching between dashes and em-dashes between paragraphs? Tells you a lot about the man, doesn't it.
zxexz 14 hours ago [-]
> Edit: Does anyone else notice the switching between dashes and em-dashes between paragraphs? Tells you a lot about the man, doesn't it.
That he doesn’t know how to use basic word processing, or even agent SKILLS.md or whatever it’s called now. Or maybe it’s just the next generation of vagueposting.
onion2k 14 hours ago [-]
Tells you a lot about the man, doesn't it.
The CEO of Anthropic not using AI extensively would be news.
kilroy123 11 hours ago [-]
Putting on my tin-foil hat here. I suspect we'll see major RAM shortages well into the 2030s to stop the commoners from having enough RAM to run powerful local models.
jacquesm 10 hours ago [-]
And open source in general. Also: keep in mind that without open source to feed it there would have been no agentic coding.
raldi 12 hours ago [-]
I don’t understand your edit; can you quote an example switch?
toasty228 12 hours ago [-]
The tech industry cooking up some new way to screw people over for 25 years
The tech execs waking up on a random monday:
> I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.
No shit Dario, no shit...
Havoc 12 hours ago [-]
Yeah and we’re not even done with dealing with the societal consequences of social media and dopamine algorithm driven everything
knollimar 10 hours ago [-]
That sounds like a "dismissal attack" and we're giving you less now to combat it
blitzar 5 hours ago [-]
Never meet your heroes.
Also ... Never read the tweets of your heroes.
onion2k 14 hours ago [-]
When people talk about Qwen 3.8 being on a par with Fable, they're really talking about Qwen 3.8 Max aka Qwen3.8-2.4T-A95B. That's a 2.4 trillion parameter Mixture of Experts model with 95B active parameters. You need about 400GB of RAM to run it. No one is running that locally.
The distillations of Qwen 3.8 down to a 27B model are good, but they're not on a par with frontier models.
When Dario talks about open weights not being a solution this is what he means - if you don't have 400GB of VRAM lying around the fact that there's an open model like Qwen3.8-2.4T-A95B doesn't really help much. If we're not regulating how models are available, or making sure access is open, then RAM prices will mean everything concentrates on a few very rich companies.
xscott 12 hours ago [-]
I don't really understand the argument you're making, but just to add a data point:
DeepSeek V4 Flash 0731 is 167 gigabytes from the developer and as a GGUF with no additional quantization. It limps along on my 192GB M2 Mac from several years ago [0]. This model tests better[1] than Claude Opus 4.6 released in February. That's six months ago - what will be available 6 months from now?
So yeah, enthusiasts aren't going to run frontier models on their gaming machines, but a small office could easily justify the $30k - $100k cost to run something like this at high speed. The small company I worked for routinely spent that kind of money on Dec Alphas twenty five years ago, and that's not accounting for inflation adjustment.
And this is completely discounting the advances smaller models are making. You're right that Qwen 3.8 comes in different sizes. However, Qwen 3.8 27B and Qwen 3.6 27B do run on gaming cards, and they're better than the frontier models from twelve months ago.
I have no idea what will happen in the future, but I wouldn't base my guesses solely on the largest open weight models.
[0] Yes, it's unpleasantly slow (5-8 tok/sec)
[1] Yes, benchmarks should be taken with a lot of salt.
woadwarrior01 14 hours ago [-]
> The distillations of Qwen 3.8 down to a 27B model are good, but they're not on a par with frontier models.
Does it have to be? There are plenty of coding tasks, where it's good enough.
onion2k 13 hours ago [-]
Practically, no, the distill is great. It's fine to use it.
However, if you're having a discussion about access to frontier models, and using Qwen 3.8 as an example of how open weights is a solution, then you should be honest and accurate about what you're talking about. Making an argument like "People can run Qwen 3.8 at home. That shows open weights are great." is a bit disingenuous if you're not also making it clear that you're not talking about Qwen 3.8 Max (or that you have a beast of a PC at home :D ).
woadwarrior01 13 hours ago [-]
I agree 100%. IMO, it all started with ollama misrepresenting the Deepseek R1 distills as Deepseek R1, all for hype and marketing. I've had so many ostensibly technical people telling me: "I tried DeepSeek R1 and it was terrible", and every time when I probe further they'd tried the tiny 1.5B Qwen2.5 distill model that was further brain damaged by ollama's naive RTN quantization[1]. DeepSeek themselves were very forthright about it by naming it DeepSeek-R1-Distill-Qwen-1.5B[2].
I suppose the road to technical hell is paved with marketers and grifters. :)
Exactly. There are common coding tasks that these models can adequately do. They are absolute trash for anything that isn't coding. And even with coding, they are so, so far behind frontier models.
dgellow 13 hours ago [-]
No one is running that locally because of the AI bubble consuming all the hardware in the industry. That won’t be the case long term though
dude250711 14 hours ago [-]
So, what does a for-profit CEO, beholden to profit-seeking investors, have to say?
ed_mercer 12 hours ago [-]
Yes, he has financial incentives. That doesn't automatically make what he says false or dishonest.
gbrindisi 10 hours ago [-]
it kinda does unless he is straightforward about his financial incentives
DangitBobby 8 hours ago [-]
Lol. If he included everything you'd claim he needs to sound honest you'd criticize him for being long winded instead.
simoncion 3 hours ago [-]
That's what either a disclosures section or a series of footnotes at the end is for.
For folks who care about quality discussion and writing, it's a tragedy that people are using microblogging tools to publish essays. The length restrictions enforced by and inferior formatting tools provided by these things make it difficult to have serious talks about complex topics.
Of course -for a subset of the population- those limitations are a feature, rather than a bug.
webdood90 7 hours ago [-]
What in the world could possibly compel you to shill for this man?
saidnooneever 11 hours ago [-]
he notes how to critisze frontier labs for failing to deliver, but says also not to blame their marketing. That is next level dim, because the marketing sets expectations for what they will deliver.
this guy need to stop vibeposting nonsense.
cloudie78 11 hours ago [-]
How’s that moat coming along, Dario?
Or are we still doing the lawfare strat since you can’t get any meaningful lead on Chinese models?
How about the involvement in the Iran bombing that killed 160, out of which 120 were kids?
Forgive me if Anthropic doesn’t really rhyme with trust over here.
7 hours ago [-]
mannanj 4 hours ago [-]
Isn't trust fundamentally about honesty, reputation and good will?
How can you restore trust by ignoring the past deception, bad will and abuse - Is Dario saying you simply ignore that, and deliver on an outcome later?
If I break my agreements of how to treat people and they do not trust me, it's on me to restore that trust through acting different and showing them I've changed. It takes ownership of my past actions too, not just shoving them under a rug.
nnmg 13 hours ago [-]
From anthropic we now have Science as a PR strategy. God I can't wait until this bubble pops.
livinglist 12 hours ago [-]
> Overall my view is that AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws).
If only money thirsty capitals didn’t drain the HBM/DRAM/NAND capacity to rush to build out all the data centers so that they can make money off of inference, driving up memory prices like mad man leaving consumers with not only no chance of local inference setup, but also higher price of consumer electronics. Of course it has nothing to do with regulation, more to do with the greed.
BrenBarn 13 hours ago [-]
> I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.
It is so tragicomic to see people whose lives are built around companies coming so close to realizing that everything they do is bad, and then at the last minute swerving aside to convince themselves that no, if they just do more of it and somehow do it "better", then it will all be okay. The reason people don't trust companies and believe they are cooking up some new way to screw them over is because that is what they are doing. If Dario or anyone else really wanted to dispel that perception there's an easy way: do a total 180 and start fighting against everything you've been pushing. But none of them will do that because they still fundamentally believe that what they are doing is good, and are unable to see that fundamentally it's bad.
game_the0ry 6 hours ago [-]
There were may companies that announced layoffs and cited ai as one of the factors: Amazon, Oracle, Meta, Microsoft, Block, Salesforce, Cisco, Cloudflare, Atlassian, Coinbase, Snap, GitLab, Monday.com.
Both founders of Anthropic and OpenAI have talked about how AI will replace many jobs, and did so with a sense of doom and urgency.
And yet...they are (pikachu face) surprised that people do not want data centers (which will increase utility costs for consumers) and graduates boo tech leaders at commencement ceremonies?
Tech leaders are clueless.
doubtfuluser 11 hours ago [-]
> „… it will actually be possible to cure most human disease in ~5-10 years, as crazy as it may sound to ordinary people …“
Nothing beats the hubris of the Silicon Valley CEOs and self proclaimed “elite”. I’m missing people calling out “we are better than that”… NOT
andrewstuart 10 hours ago [-]
Is he worried?
Dario is always deeply worried.
I read somewhere he is the Woody Allen of AI.
brador 12 hours ago [-]
For $20 a month you can experience what it’s like to have a 130+ iq for a few minutes a day.
Why does this need a master?
Do we regulate bio geniuses for the danger they pose? No, we do not.
Do we regulate other tools like hammers and drills? No. Books? No.
12 hours ago [-]
3dsnano 10 hours ago [-]
i’m all for ambition and tenacity, but this is psychopath level communication… blissfully unaware, simply incapable of reading the room
if claude still can’t yet write and refactor coherent code (on its own, replacing software engs, just like he said last year), why would we expect it to… fucking cure cancer?
cindyllm 9 hours ago [-]
[dead]
stego-tech 8 hours ago [-]
That is an incredible amount of words to expend upon a position tantamount to “neener neener we won’t change a thing and you’re in the wrong for doubting us.”
I’d really appreciate it if folks stopped paying more attention to the bloviating or whinging of AI executives and instead focused exclusively on their actions, for therein lies the true nature of their goals or vision.
lenerdenator 9 hours ago [-]
Cut your prices and stop expecting everyone to sacrifice their jobs and their wallets for your vision and desire to be a trillionaire.
Ta-da. Trust crisis solved.
petesergeant 14 hours ago [-]
I think that Dario is intelligent, incisive, has good judgement, and is well-intentioned. I hope he continues to wield influence. I think Sam is also most of those things, but I think power has a One Ring-like effect on him. My most contrarian view is that I think Elon’s problems are primarily appalling judgement when it comes to areas outside of tech, and has the emotional regulation of a child, although somewhat incredibly I believe he is fundamentally well intentioned. I don’t think any of those three really want tech feudalism, although I think the current US administration would press a “turn us into Russia” button the second they caught sight of it, which is to say, I think they have the worst of intentions.
mschuster91 14 hours ago [-]
> I think Sam is also most of those things, but I think power has a One Ring-like effect on him.
That's putting it mildly, very mildly. The guy all but wrecked the entire world's DRAM supply chain by using negotiation tactics that, if there were any semblance of regulatory authority left in the US, would lead to criminal charges for market manipulation.
Arrest Sam Altman, he is one of the most dangerous people on the planet. Not a single shred of respect for the 99% and the consequences his actions have on them. (And similar things apply to the other controversial figure you mentioned, Elon Musk actively enjoys hurting people by the looks of it.)
jLaForest 11 hours ago [-]
This guy's wife was begging Epstein for money for her porn business after Epstein was a convicted sex offender, but we are supposed to trust his judgement?
"Cami Clark, the low-profile wife of Anthropic CEO Dario Amodei, tried to court Jeffrey Epstein as an investor for her “luxury porn company” and women’s health startup, the Wall Street Journal and emails in the Epstein files reveal, as the Journal’s reporting raises new questions about Clark’s influence over her powerful husband and his company."
What a slimy couple.
jacquesm 10 hours ago [-]
Details...
meindnoch 1 hours ago [-]
pardon?
jacquesm 38 minutes ago [-]
That these aren't details. And for some reason there is always an Epstein connection.
arm32 6 hours ago [-]
This makes him immediately lose all credibility related to common judgement, safety or alignment IMHO.
thiago_fm 14 hours ago [-]
Dario's messaging has been extremely confusing and HE'S RESPONSIBLE for further increasing the negative sentiment of the public towards AI.
He's clearly focusing on being on the news to provoke emotions on people, preparing for the Anthropic big blockbuster IPO.
Dario, Sam Altman and others should instead be praying every night that this will take us to the Singularity VERY SOON, because if it doesn't, the entire country will blame them for absolutely destroying their retirement and the US' economy once this AI Datacenter bubble pops.
eru 14 hours ago [-]
There are other countries.
dogleash 8 hours ago [-]
>or the tech industry and always suspect that we are cooking up some new way to screw them over
Looks like Amodi is laying the groundwork so he can re-run Eric Schmidt's "but it happened" playbook. Do whatever you can to make money, wait for the dust to settle and walk off into the sunset after one final "whoopie daisy, didn't mean to do that to you. sucks to suck."
greatgib 10 hours ago [-]
I think that this guy and Anthropic are a clear example of the "power corrupt" rule.
Basically he is probably a nice guy and well meaning from his point of view.
But, now that they hold a huge power to impact "on demand" AI usage and regulation, and they are kind of the tip of a monopoly on advanced AI, they will see themselves as more important than anyone else, and with the moral duty to impose their view on everyone that hasn't the worth as them.
Just imagine if the company was still a small outsider, their official point of view would probably be a lot different. Especially with something like "regulation is a barrier to entry"...
jacquesm 20 hours ago [-]
* * *
darkwater 14 hours ago [-]
Why the comment submitted by jacquesm, who posted the link and it's not exactly an anonymous poster, that said "Apologies for linking to X but this is worth reading." has been flagged to death??
coldtea 14 hours ago [-]
Maybe because "apologies for linking to X" is needless gaudy virtue signalling.
Make it "Apologies for linking to Amodei" and we're talking.
dyauspitr 12 hours ago [-]
Just seems like virtue to me.
11 hours ago [-]
defrost 14 hours ago [-]
Vouched. (A few more might resurrect it)
eru 14 hours ago [-]
You can 'vouch' for a comment, if you think it shouldn't be flagged.
darkwater 14 hours ago [-]
I'm more interested in why somebody would suddenly flag that post. It's not spam, bot, offensive or anything else. Maybe you could downvote it if you don't like the take on X - it still would be childish but at least would be proportionate.
sscaryterry 12 hours ago [-]
Why? I'm sure there are plenty of Anthropic employees on here...
isoprophlex 14 hours ago [-]
i was about to post the same thing. i vouched for the comment, but it's too dead to recover from my single vouching action.
raverbashing 14 hours ago [-]
This site's audience has been less of the technical curious kind and more of the pathological misalignment one
WorseIsBetter 6 hours ago [-]
[dead]
MMTlover 6 hours ago [-]
[dead]
locitra 12 hours ago [-]
[flagged]
hn5nw0vcqg 9 hours ago [-]
[dead]
agnosticmantis 10 hours ago [-]
> Anthropic is ramping up its efforts very quickly in biology and medicine
I hope they have better sandboxes in their biology lab than they did in their cyber security lab.
More seriously, these sociopaths will soon release lab-made viruses into the world to prove AI's capability and to force governments into regulation of open models. It should be obvious by now that with their savior complex, the goals (cure cancer, AI safety, save humanity) justify any means.
VCFundedGenYer 5 hours ago [-]
The less he says the better. This was saying way too much in a far too pretentious way, without addressing the core problems.
It's not quite Sam Altman levels of delusional, but it's up there.
This is not how to swing the pendulum back in a positive direction.
Want to make a difference? Make AI use little to no natural resources, stop eating up RAM, stop making everything expensive, etc. Otherwise, don't talk.
fabsalvadori 11 hours ago [-]
I see little conversation about AI powering robots' core and the fact that we are going to have like a billion robots by the end of this decade. Yes, sorry if trust is low.
mcntsh 12 hours ago [-]
So the solution to convince the public that these companies aren’t “looking for new ways to screw them over” is to try to go into biomedical research.
I guess it’ll be great for Anthropic to have the cure for cancer, but what’s that gonna mean for people with cancer? Funny how he doesn’t talk about that part.
miyoji 10 hours ago [-]
> I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.
Oh gosh, I can't imagine why that might be, could it be the history of screwing us over?
If you want to build trust, I have a simple suggestion that Anthropic or OpenAI could implement today, if they wanted, and it would make me trust them probably 1000x more than I do today. That change is quite simple: stop selling to governments and enterprise. If you really believe in this product, you can earn enough money from regular people, you don't need government money and you don't need to sell to those corporations we don't trust. Just sell it only to individuals directly and add in your TOS that no company can force employees to use your product, or you won't sell it to anyone who works there anymore. This would strongly signal that you don't want AI to concentrate power, since you won't sell it to concentrations of power. It would build a lot of trust.
Oh, you want to "cure cancer" instead? Of course you do. Why am I backing away slowly? Maybe because I'm pretty sure you're cooking up a way to screw me.
[...]
> I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer. I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world. That is totally on us, and I think it’s the criticism you should be making, instead of all this stuff about messaging and marketing.
> We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that.
Turns out "winning back trust" doesn't have anything to do with any actual concerns people may have re: employment, electricity prices, stock market bubble, intellectual property, scams, cybersecurity, environmental issues etc. Rather we'll just do all of that even harder and the miracles ("curing cancer", lol) we've so far failed to deliver are bound to arrive in short order!
This is honestly hilarious. Dario must think people are stupid.
First, cancer research charities are some of the most well funded on this planet. They all obtain donations on the basis of "together, we will cure cancer" messaging. They all fund the best science they can.
Second, the human body is a complicated thing. You can feed your fancy LLM as many textbooks and academic papers as you like, but the reality on the hospital ward will always be different. Why do you think student doctors have to spend so many years "doing the rounds" Dario ? They are all academically smart, they are all capable of memorizing text books ... but there is no substitute for seeing and doing the reality.
I barely trust Claude to write code, let alone find a cure to cancer.
It's like saying "we'll cure infectious diseases"
Don't worry, I'm sure that's his next blog post. Right after Claude has finished solving famine and poverty. ;)
We have more than enough food on the planet for everyone, and we have the ability to deliver bombs or soldiers or amazon packages to anywhere in the world in days or hours. Combine excess food with amazon or army level logistics, and famine is gone.
What people mean when they say “solve famine” is can we make people not starve without actually doing or changing anything - which is of course impossible.
We do this. Have you ever heard of UN World Food Programme? $14B funding in 2022 (since then dropped drastically, guess why).
Has it solved famine? No. Would it solve famine if it got the funding it needs? Unlikely.
Because you cannot just spend billions of dollars to send amazin food parcels across the world and expect that to fix climate change, draught, poverty, colonialism, racism, illiteracy, lack of education, and propably 100 other issues all contributing to the fact that we still have not end hunger. And no, AI won’t solve it either.
So one would expect $14b/year would solve about 14/93 = 15% of world hunger.
> The World Food Programme supports over 152 million people annually and delivers life-saving food assistance where needed most. (https://en.wikipedia.org/wiki/World_Food_Programme)
An estimated 757 million people are food insecure (https://www.who.int/news/item/24-07-2024-hunger-numbers-stub...)
152/757 = the world food program helps ~20% of people who are food insecure. That's well in the ballpark of the estimated 15% spend.
If we spend that extra $93b, we'd be well within the range to actually stop world hunger.
Yes, "climate change, draught, poverty, colonialism, racism, illiteracy, lack of education" are still issues. No, you don't have to solve those problems to feed people. Don't let perfect be the enemy of good.
Almost all recent famines have been caused (either deliberately or incidentally) by government policies, not lack of money. The UN spending 93 BB may (somewhat) improve the situation, but is unlikely to end food insecurity.
Eliminating corrupt leaders (i.e. Kim Jong Un, Vladimir Putin, etc.), and poor regulation (e.g. Bombay Fodder and Grain Control Act, 1939) would be much more effective.
https://en.wikipedia.org/wiki/Famine#20th_century
> Has it solved famine? No.
Hoist by your own petard
You know that saving millions of lives every year is better than NOT saving millions of lives, right? Just because something doesn't solve the problem 100% doesn't mean it's not VERY VERY good.
How weird that all those humans seem not to trust the logic of their betters ;P
the ultimate utopia
It is possible to prevent most viruses by recoding the human genome (requires embryo engineering or pre-embryo engineering) to use alternate codons that existing natty viruses don't use. This wouldn't prevent synthetic viruses from being developed that use the alternate codons, of course. See (1) https://diyhpl.us/wiki/transcripts/hgp-write/2016-05-10/ultr... and (2) https://diyhpl.us/wiki/transcripts/hgp-write/2017-05-09/ultr... and (3) https://x.com/RokoMijic/status/1720875112541364535 for some background.
As for "curing cancer", I suspect that Anthropic or OpenAI will use their high status/reputation to launder the existing solutions (germline, p53, enhanced DNA damage repair pathways, higher tumor-suppressor-to-oncogene copy number, more robust tumor-suppressor networks, more self-anti-cancer apoptosis, CD47 suppression, etc) that society has refused to accept. They might have enough reputation to get people to look at these ideas. It won't cure people with stage 5 turbo-cancer a.k.a the walking dead, but hopefully we won't be moving the goal posts _that_ much on them.
"Synthetic genomes unveil the effects of synonymous recoding" https://www.biorxiv.org/content/10.1101/2024.06.16.599206v1
"Probing the limits of genetic recoding using multi-omics-guided evolution" https://www.nature.com/articles/s41467-026-74300-9
"Escherichia coli with a 57-codon genetic code" https://www.science.org/doi/abs/10.1126/science.ady4368
Is it a product available to humans at all?
A more sophisticated view is that cancer (one idea) is an problem of rates (like most things in biology) and there is a tumor burden per unit time from whatever causes vs. the immune system's capacity to detect and destroy them per unit time. As humans age, the balance tips in one direction. If the balance is too far off for too long, tumors accumulate, doctors take notice, and assign whatever labels to the condition.
Looking into particular causes of tumors (and calling each cause its own cancer) can create a vast research program that employs a lot of people, since there are so many mutations that lead to tumors. But interrupting the development of a particular mutation is not going to have the same society-changing effect that solving the fundamental issue of rates would.
Also, not even all cancers have tumors. You have solid tumor oncologists and then oncologists who specialize in blood and lymphatic cancers, “liquid cancers”.
In addition, cancer does not arise from one mutation. It’s a set of mutations that allow cancer to occur. You need mutations for growth, mutations for preventing cellular death, mutations for stealing energy, mutations for hiding from the immune system. Your body, right now, has probably millions of cells that could be candidates for cancer. They either kill themselves or your immune system nukes them. You really need the stars to align on this stuff to get actual cancer.
And then, it’s not like the cancer stops mutating. Cancer becomes resistant to chemotherapy through mutations. Cancer is “smart”, it will even temporary slow its own growth (metabolism) to intake less chemotherapy through treatment.
Here's another way of framing things that might encourage better discussion: Suppose in a few years, it was announced that an LLM did discover a cure for a cancer. What approach would you expect it to have taken?
1. Developing an intervention for every possible mutation that leads to loss of cellular identity, and uncontrolled growth.
2. Preserving immune function later into life.
3. Preserving genetic and epigenetic accuracy later into life.
Maybe it will be 1. But I would bet on 2 or 3. Which both assume that by reducing the difference between rates, the phenomenon of cancer could be reduced or eliminated.
Also: https://pivot-to-ai.com/2026/07/30/google-kills-alphafold-so...
Literally first phrase on Cancer Wikipedia entry.... [1]
"Cancer is a group of diseases involving uncontrolled cell growth typically resulting in tumors with the potential to invade or spread to other parts of the body...."
[1] - https://en.wikipedia.org/wiki/Cancer
I'm inclined to believe that the thing - that everyone intelligent says is complicated - is actually complicated. I would be surprised if the thing - that everyone intelligent says is complicated - is actually simple.
This is almost like an application of the conspiracy theory gambit where you try to find out how many people would have to be "in on it" in order to disprove the theory's likelihood. Like flat earth.
All of those guys think people are stupid. That theme has been repeating over and over again.
Nixon was 24% when he resigned, which is amusingly not actually the lowest, as Truman seems to have twice achieved a spot rating of 22, though Nixon remains the lowest rating for a full year. I wonder what Truman did to be less liked than Nixon. Those were both well after dropping bombs on Japan and before the first H-bomb test.
Or do you believe that you are smart in every way there is to be smart?
Maybe I'm overly cynical, but I think charities exist to exist. They don't have a strong incentive to actually deliver on their mission. Everyone at the org may be fully bought it and obviously want to cure cancer, but as an organization it would fail to deliver on the promise. Like most organisms, their actual purpose is to continue to be influential, grow and continue to exist.
I think cancer will be solved by some group that could make a lot of money solving cancer. The probability of any one path working is very low so the payout would have to be very large for anyone to be willing to pursue.
If a big public charity is able to cure a cancer type (eg. lung cancer), then it's next aim would be to try cure another cancer type. Let's just say they are somehow able to cure all the cancers, and therefore they cured cancer, then there's distribution of the cure, with a focus on 1st world but then it would have to go to developing nations too. Along the way, there may be issues that arise in recovery from the cure, so money towards helping survivors there. Shift goal, shift goal, shift distribution/priority of donations, on and on and on. For the proper charities that do want to meet their goals, something like this is what they'd do.
In Canada, measles cases suddenly exploded in 2025 as well, and they don't have RFK Jr. there: https://health-infobase.canada.ca/measles-rubella/
The policies around measles vaccines have shifted a little since RFK became HHS Secretary, but not in a material way. Generally the messaging has focused on putting measles, mumps, and rubella in individual vaccines rather than combined, but at no point has the federal government recommended not to get vaccinated. That, combined with the Canadian data, implies that RFK or other changes to US health policy are not likely to be the cause of the measles outbreaks.
If you can find and patent a cure to cancer, you can charge whatever you want for it, at least in the US. I don’t think profit would be an issue.
And even if ending chemotherapy would discourage the incumbents, smaller companies would love the opportunity to kill them.
For example, Polaroid didn’t want to embrace digital cameras since they made tons of money selling physical film. But all the smaller camera companies didn’t care, since they weren’t benefiting from that and wanted to just take Polaroid’s market share. And that’s why digital cameras took off regardless.
If you want a simpler example that doesn't involve biotech research, I would just look at the diversity of animal welfare organizations, and their relative quality. Some of the best animal welfare organizations in terms of impact are local rescues run by (in my experience) spouses of wealthy career-people (executives, lawyers, etc.) who have a ton of free time and love animals. But then there are also plenty of animal welfare organizations with questionable motives, and questionable spending patterns - bordering on fraud and embezzlement, if not outright fraud and embezzlement.
I mean, yes, "I haven't looked into the specifics, but working from the first principles I came up with in 3 seconds, cancer research charities are in fact cancer on society" does come off a bit reductively cynical.
And they’ll be using detailed medical records. Personalized medicine and the use of ML in medicine has been around longer than LLMs (and will continue regardless). There is a lot more data available than just studies and textbooks.
https://www.cancerresearchuk.org/about-cancer/treatment/pers...
"If you or a loved one have taken a class of drugs known as GLP-1, you may be entitled to compensation..."
What he is saying makes sense to me.
Is it that you don't trust a single word he says and that he lies for financial gain?
Or do you think he is wrong and that applying AI to this kind of research cannot possibly produce a breakthrough and lead to a cure?
> Do you not think that Amodei, who worked on research about cancer biomarkers at Stanford, knows a thing or two about how to apply AI to the problem?
It certainly shows he is ambitious.
> Is it that you don't trust a single word he says and that he lies for financial gain?
Yes.
> Or do you think he is wrong and that applying AI to this kind of research cannot possibly produce a breakthrough and lead to a cure?
It is possible, but both could be true, that's not an exclusive or situation.
I am not familiar with Amodei's personal history, but I generally do not assume the worst of people just because they are extremely successful, like many others here seem to do.
Between Musk, Altman, Schmidt, Amodei, Gates and a bunch of others I have a really hard time figuring out who is the least ethical. Probably Musk. But it may well be that the long term damage done by Amodei and Altman is larger. Ask me again in 10 years, if I'm still around.
I see the conflict of interest of course, but so far I have not seen anything that convinced me Amodei works towards any other goal than benefit for humanity as a whole, like he claims.
if many other see it as substantial ground, and you just say "I don't see anything", how do you think this discussion can go forward?
It's amazing that I even have to ask for it, as if I am just supposed to believe so.
Alternatively, one could also engage with my argument that Amodei's position seems consistent with his beliefs, and point out flaws with it.
As for "bad person" is concept you brought to discussion, it could mean many things, but you were provided with enough ground from opposite point of view.
> It's amazing that I even have to ask for it, as if I am just supposed to believe so.
toxicity only adds to your character of bias and denial.
The sbf vibes are on the nose. Remember that it was a big deal that sbf bought into Anthropic, and many if not most Anthropic employees are EA, the tribe of people I'd rather call "rationalizists". The main thing they rationalize, is of course themselves taking on power and what sbf made famous as uh the linear approach to risk
Between Altman and Amodei, I'd say that Amodei is the more dangerous just because he seems even more "reasonable" to nonjerks. You should ask the mods which one of the two they think are more reasonable. It's clear to lobbyists and politicians --- people familiar with the human jungle that is DC--- that Amodei is weirder, though.
I think I have a good metric for who is least ethical. Which one on your list is most unlikely to write good satire (or most likely to write unintentional self-satire?)
I'd say its Schmidt, followed by Amodei, and then Musk. See Musk on SNL. I think Gates had some TV appearances where he was reasonably funny, almost self-deprecating.
If you ask me, SBF ran an obvious scam in a sector that is as a whole a scam.
With Anthropic's product, I experience a real benefit today.
This reminds me of the myth of Prometheus. I think most Greek thinkers would agree that Prometheus was the hero, and Hephaestus a complete jerk, even though one was the master of fire and the other merely stole it. Of course they had to make up a punishment for Prometheus so that he could earn his status the meritocratic way..
Probably much against most people here, I separate the product from the person creating it. As another Anthropic example, there was the math guy at Anthropic who disproved a long running conjecture.. it doesn't look like he's getting Fable and Anthropic the respect he thinks they deserve. The process matters.
People are sensitive to disrespect hardcoded in the process, especially against "tried and true" norms, and in most cases they are right. The ability to write satire could be a kind of razor to tell apart the influencers who are likely to be righter than their detractors
The Fable quota on the $200 personal plan could be higher, but the difference to Opus 5 or 5.6 Sol is not large enough to gain any real benefit from unlimited access to Fable, for my use.
From what I heard their internal 'Model 2' is hardly much better, but hopefully they will have something newer soon.
You don't become a CEO of a company like Anthropic (or any sufficiently large company) without lying for financial gain
It will become victim of the conflict between its core search for truth, the current administration mandate to spy on its own prompts for approval, and secret orders to hide the mission true purpose, like that other computer we have heard about.
Can you find any explicitly EA organization that is at all aligned in perspective with anything you suggested?
It's anti-human and evil. What a shame that it's illegal for me to say what ought to happen to them.
BTW, not sure what gets more anti-human and evil than wishing violence on whole groups of people!
Just like the utilitarians in the thought experiment tossing everything positive in their life into the bottomless pit of suffering, the effective altruist AI thought leaders are tossing our entire economy to the gambit that the singularity will bring about the end of human suffering.
Capitalism has always been a tension between capitalists, who want maximum return on capital and workers, who want pesky things like a living wage, sick leave or safe work conditions.
For the longest time, the only power labor has had to get those things was the power of collective bargaining. No agreement with your workers meant no production happened.
Now, there's finally a chance at salvation. The capitalist Messiah is AGI and it will finally deliver them from those annoying laborers.
This ideological bent is why they're putting everything they have into AI. It's why VCs and their fellow capitalists are going so crazy.
They see it as a way to finally solve the contradictions of their ideology, but in reality it'd only create a new one: If everyone's out of a job, who will buy their products?
Can't help wondering if they've really thought this through.
AI will make human thought economically useless. In the new economy, the one economically useful thing we will be able to do is legal property ownership.
Permanent underclass is ideological goal of the above people. AI is an impersonal technology with no goals or wishes.
There's an inevitable effect that it has, and the only lever you have as an individual is being on the right side of that.
Even if the US federal government refuses to act, smaller government entities can deploy their own compute clusters running strong open-weights models like Kimi K3, and provide their own citizens with free access to strong AI models. There may come a time when even Chinese providers no longer release strong model weights, but then the previous generation of strong open-weights models can be used to accelerate the development of data pipelines and training setups to build the next generation. California or Australia (or many others) could fund university-led R&D efforts to build completely open models.
Despite the angry noises that leading providers make about their model outputs being used to train competing models, that's not illegal. States can use Common Crawl, Anna's Archive, Books3, commercial model outputs, and whatever other data sources that are useful for building models. States have sovereign immunity to ignore copyright complaints, if they decide to pursue model development.
The big AI companies don't have much of a moat. Let the big companies spend a fortune figuring out what works best, then openly build imitations having 95% of the capability at 5% of the cost.
AI is the least of the problem.
This is correct. But we don't need to ban AI to avoid the permanent underclass nightmare.
It's time for the means of production to be owned communally.
It's either a cybersocialist utopia or a neofeudalist dystopia.
Rosa Luxemburg formulated it pretty well more than a hundred years ago: Socialism or Barbarism!.
What does this mean in practice? That an individual cannot own a machine (which is a means of production)?
As a first intuition, I feel like its mostly server farms and data centers that are the concern, large scale means of production.
It feels right that those would be communally owned, be that via democratic government or maybe some sort of worker owned coop?
Today, people have leverage because removing them has real economic consequences. With AI, we need to be on the useful side of the divide to have any leverage.
And then we'd better hope AI has respect for property rights.
It's probably a lot easier to convince the military AIs we're building to do it.
I don't particularly like the situation, but I plan to do what I can to be on the winning side. The world is going to be what we're all in on building, and for people on the winning side it may be a utopia. Either way, I know which side of the gun I want to be standing behind.
Also note that an AI well aligned with the people who initially trained it wouldn't need to actually kill anyone that didn't take action against the owners, it would just need to outcompete people for resources, and let them fend for themselves in extreme poverty. There's no need to act one way or the other beyond that.
I'm talking about morality here, not practical ability. The ultrarich have their circle of friends and family, who themselves have their own social circle, on and on until suddenly you're not looking at wealthy people anymore. Six degrees of separation and all that. How many people in the social graph can know about this extermination plan before the whole thing collapses?
It's just disenfranchisement. Only the people who try to use violence to replace the political leverage they lose when they lose economic leverage would need any action taken against them, and that's easy to justify as self defense.
I think it's inevitable.
When AGI becomes reality, that reasoning disappears. When everyone's worthless, no one is. Thus the scenario you speculate about would need everyone in the upper echelons of society to be consciously, openly sadistic power-hungry people, which is not the case even when taking into account the higher dark triad rates among them.
Why did you think there's a massive uptick in surveillance and policing technology and automation? It's to make the French solution unfeasible.
That was the exact line of thought, coupled with resources are limited (marketed as climate change), that led to the 1 child policy in China and many other decisions that shall not be spoken of.
It doesn't matter that I can't do it. They'll pay me lots of money if they think I can.
The underlying purpose of AI is to allow wealth to access skill while removing from the skilled the ability to access wealth. (read on HN, not my words)
Curing cancer sounds insane, but it's also a research problem, not a societal level coordination problem. And one AI has already proved to help with breakthroughs (alphafold). IMO it makes sense for them to shoot for something like that as proof of AI's beneficial sides.
This is not true. There are two companies at the center of AI direction: Anthropic and OpenAI. If there were anyone on the plant who has the ability to influence our direction then it would be Dario Amodei.
Acknowledging the grievances is a good step, but it's not enough. There needs to be a clear explanation of actions to address them, a plan to enact those actions, and commitments with consequences in failure of those actions. Tell people how you're going to make them more employable and effective and needed. Tell people how your datacenters will be carbon neutral. Tell people how financial actions resulting in a frothy market will be coming to an end. He and Sam Altman alone have this power and their inaction says everything we need to know about their intent.
Hoping that they solve those huge problems themselves is basically hoping that they will have more power.
It would be like expecting Craigslist to rescue journalism. The amount of power needed to disrupt an industry is nowhere near what's needed to fix it.
Possibly, but not necessarily in a better direction. And that's the whole problem here. They should all go watch Phantasia.
If you don't tackle the societal level problems, nobody will care about research breakthroughs.
I don't think anyone will care about the largely made up issues about electricity prices or water use in case AI literally cures cancer.
His body was healthy, aside from the bullet in his skull.
I just don't buy the AI labs approach to this stuff. Like, unless we can basically simulate the entirety of human biology, I don't really see how LLMs can make progress here. Maths is different as it doesn't require a real-world interface, and programming already (by definition) can be simulated on a computer.
Without that, I can't see much (if any) progress being made on domains like biology.
Drug discovery is similar AFAIK. The space of possibilities is even larger than protein folding, but it's structurally similar enough that I think AI will help to make progress on the discovery side. Actually getting the drug tested and approved is another matter though for sure.
For one, the question for Anthropic is whether LLMs, specifically, not AI techniques more generally, can help significantly with cancer research. And here, all experience so far is that LLMs only really work when they can easily automatically verify their own outputs and self correct - such as in math (using automatic proof verifiers) or programming (using compilers and unit tests).
The second problem is that biological research speed is highly dependent on slow biological processes, such as cultures and long term studies. In programming, if an LLM could provide excellent insights and research suggestions 100x faster than a human, it would speed up the work roughly 100x. But in biology, it would only speed up the total work by a small amount - as any insight, even if absolutely brilliant and spot on, would still require months and years of actual experimentation.
If I was Dario, and I was after a cynical sales grab to go with the "actually curing cancer!" spiel, I'd probably aim for a drug repurposing strategy. I'd use the above approach to find existing drugs that might target known pathways that drive incurable cancers. I'd only screen compounds with extensive safety data and easy delivery mechanisms, and from the in silico hits, I'd throw a tonne of money at rapidly experimentally screening all those candidates in parallel, and then rapidly push those that worked into clinical trials. I'd assume that, with a small but non-negligible prior, and the money to push through hundreds and hundreds of candidate compounds through at once, I'd have a reasonable chance of getting one drug through to a "Claude Cured Cancer!" show-stopper headline.
But I think even then, with all of Anthropic's wealth, you'd need 2 years minimum to move from initial screening targets to a Phase 3 trial. And it would be an almost criminal waste of research funding to get there -- the dollars for discoveries ratio would be appallingly bad.
(If I was Dario, and I actually wanted to cure some forms of cancer? I'd just use my obscene profits to fund actual cancer research, step back, and let the researchers get on with it.)
I do agree that this kind of targeted approach makes sense.
However, discovery is not really the issue here. Running the clinical trials (1/2/3) is much much more difficult, and consumes basically all of the time in drug development, so even if LLMs perfectly automate this, the speedup will not be particularly large.
But something tells me either they can't or they won't, so no trust will be built.
Curing cancer is a societal problem. We have lots of cures for common diseases but resource allocation means people don't actually receive the treatment they need. For example, prohibitively expensive gene therapies, or HIV treatments in developing countries. A disease may have a "cure" but if people who need it don't receive it then from their perspective it may as well not exist.
Anthropic is literally creating the bubble. It is not beyond their scope of influence, it is literally what they are consciously achieving.
As for peoples jobs, same actually applies. Anthropic is selling itself on dream of replacing jobs, even or especially where they are well aware AI does not perform that well. They are actively trying to replace people quickly before management notices it does not work well.
And also, they can influence how much their data centers contribute to global warming.
I think typical bubble behavior the leaders have set up the whole promise to fail. Everyone is expecting some faux super intelligence to come and find a cancer solution everyone else missed. However, it is just as likely that vanilla current LLM's will create enough of a productivity boost for back office automations in research heavy hospitals to create the space for regular humans to create these breakthroughs, but LLM's won't be able to claim that for themselves and inevitably “fail”.
P.S: I am not saying applications will go away but LLM's are clearly massively effective here at the rote parts of it all.
It's hard to keep track of the frontier on bio ML, but it seems that we're going slower than what Demis Hassabis said in 2024 with 5 years to full cell molecular simulation. We still aren't able to reliably model a tiny surface of the cell membrane.
And of course there's Derek Lowe's takes on the drug discovery pipeline waiting for the proof in the pudding.
To me the only reasonable bullish position is that there is a very non-linear AGI threshold for accelerating progress that we haven't hit yet.
For me personally, I'm looking at other more tractable fields as a proxy to measure this kind of progress. The best modest evidence is from the agentic coding area, (modest because these kinds of gains may not translate to bio progress). Other soft-ish fields to like legal/law/tax are also interesting to watch, as a small amount of people are now trusting AI for these areas that were considered totally unusable a year ago. Another proxy is being able to generate generally entertaining media.
Turns out that “give a reasonable probability of being close enough such that you can bootstrap a solution out of experimental data” gives a very high utility and effectively obsoleted several experimental techniques overnight; pretty much “Molecular replacement” is about the only technique for phasing resolution anyone bothers with any more.
But again, “Bio” is an _extremely_ broad term; for every part of the field Alphafold had a big effect on there are a thousand different parts of the field that it did nothing for.
They made huge progress, but I would say that the vast majority of work on this problem was designing the harness for the model. That's a lot of work for each and every domain.
Isn't that so far only static folding?
[0] https://www.science.org/content/blog-post/so-how-ai-drug-dis...
At the same time, the best Anthro models refuse to answer biology questions, keeping the public in the dark not only about new AI-assisted results but also about conventional and health-protective research.
D.A. >> we hope to have incredible results in the coming years and some early glimmers in the coming months.
So, what changed? He went from a promise of AI bonanza any minute now to a promise to maybe cure cancer in the coming years, that's right, an unspecified number of years. Until that time, we're supposed to eat glimmers of hope because the AI-caused inflation will make food unaffordable.
D.A. >> When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that.
And don't forget to stock on ear protection, because when something happens, it's going to be "as laud as possible", you've got Amo's word too - to marvel at in the meantime.
No snark here from my side, just curious about your vision - which concrete actions by Anthropic could improve trust for the concerns you listed?
Considering the vast amount of money, world-class talent and political and industry vonnections Anthropic has access to, it feels a bit unfair to ask me to solve this for them.
It signals to people who have money and/or influence that the speaker is "committed to doing the right thing" and "eager to find a workable path forward" that "meaningfully addresses the 'very real' concerns and misunderstandings that non-experts have about what the company the speaker is associated with are doing".
Once the speaker has signaled that, what more could they possibly do? They've signaled their willingness to open the channels of dialog with the rabble who are non-experts and -thus- cannot possibly be engaged with as equals. By doing this, the speaker tried their best, didn't they?
Or, to put it another way, it's PR and damage control... soothing theater for the folks who can actually affect them.
So Anthropic are going to spin up a medicinal chemistry lab and start mouse experiments?
I mean, it would be nice, but I don't think that the guys at Anthropic know what they're talking about here, or what they might be getting themselves into. (If indeed this is more than just PR.)
https://www.writingruxandrabio.com/p/intelligence-is-not-the...
Didn't and don't mean to disparage anyone's medical struggles, but "a cure for cancer" is a well-worn strawman. One that's been achieved for the low hanging fruit, the higher ones are seeing steady progress (already before LLM chatbots, even!), the bottleneck isn't "intelligence" and to the extent there are socioeconomic (access to screening, treatment) or environmental/lifestyle factors involved the AI boom is likely just making things worse!
I'm sure Dario knows this, and it's anyway too pedestrian compared to the usual list of fruits of ASI. The text probably originally read "nanobots eating you alive and uploading to the cloud" or something, but they figured that wouldn't go over with the intended audience. "What do the peasants care about? Oh I know! Curing cancer!"
How else are you going to work towards a dream? It's how Elon Musk managed to get reusable rockets when everyone said it's unfeasible.
Granted, most ideas don't work, this is why you need testing, but I am a bit surprised to see this attitude on hacker news.
But they said that they want to cure cancer. That's far outside the core competencies of any LLM, and it requires a lot of real-world wetwork with liquids, chemicals, cell line experiments, animal experiments, etc. They can't merely analyze existing data -- they'd need to generate vast amounts of new data, which isn't really the case in physics. And then regulatory approvals and so forth.
"We're working on curing cancer" sounds more like a poor PR attempt than an actual effort, though I'd love to be wrong.
As if we didn't know the challenges in translating research into clinical treatments.
I wonder if many of the commenters here read the post, where Amodei specifically references his essay with suggestions for streamlining the FDA approval processes: https://darioamodei.com/post/policy-on-the-ai-exponential
Writing a paper on streamlining the FDA approval process is nice, and I agree with all of it, but it's one thing to do that (about 4 hours of work) and quite another thing to cure cancer.
My point was more that LLMs are exceptionally weak at developing drugs. They're strong at every math/logic/language task, but curing cancer is something of an entirely different nature. Again, if he wanted to "solve physics," or "develop a universal real-time language translator," or even "solve global warming," I wouldn't be nearly so skeptical.
...The first is something that can be credibly attempted with a next-gen super-LLM and a ton of compute. The second is something that LLMs are exceptionally good at, and a dedicated engineering effort might get you there. The third is a modeling/simulation problem with solutions that are, at least in principle, known but haven't yet been thoroughly weighted. (i.e., how bad really is ocean seeding or launching SO2 into the atmosphere?)
But curing cancer? It's not even clear how to approach the problem. Cancer is also heterogeneous... so when I say that LLMs are weak at developing drugs, the problem is compounded, as curing cancer might require dozens or hundreds of distinct new drugs. It's like Dario picked the least likely, most difficult problem to solve. So it's weird.
Did you know that Amodei was working on research about cancer biomarkers at Stanford?
Given his background in biology, it is hardly like he is a random tech guy with no knowledge on the topic, as many here seem to imply.
Amodei mentioned "actually curing cancer" as an example of undeniable public benefit, not as an exclusive research focus or anything like that. He is clearly talking about generally intensifying biomedical efforts, not promising a cure for cancer specifically.
Where do I find that in the Act (or code of practice etc.)? IANAL, but Article 50 reads different to me but if there is a comment or guide how to read it - also fair enough.
“The detection solution may be made available in the Union as one or more of the following: (i) a public, ideally standardised, specification allowing any third party to implement a detection mechanism; (ii) a piece of software (e.g., a standalone executable or library); (iii) a cloud-based service accessible to users in the Union through an API.”
Option 3: "a cloud-based service".
And accessible "to users". NOT to the public.
Note: this is not the actual law, it is the code of practice, ie. guidance for model providers. The start of that document clearly states that you can comply with the law in other ways if you want, you'll just have to justify yourself. So you have a choice to not even do this.
The actual law is here:
https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-5...
As to who decides the law clearly states who decides if someone is legal, like in most EU legislation. It's not the courts, it's "National market surveillance authorities designated by each EU Member State", and there is an EU office as well (and it is explicitly stated that they are not allowed to override each other). So every EU country has the right to provide exceptions to the law, just like they do for the GPDR. You do not have any rights under this legislation as an individual. Only these "National market surveillance authorities" get rights under this legislation.
Secondary: article 7 additionally gives the EU commission the power to declare any code of conduct they want that declares what compliance with the AI act actually means.
(and, of course, this is yet another attempt at declaring math illegal. The only way to actually enforce this legislation is for all models to comply with this, all over the internet. Obviously the EU does not remotely have the power to make that happen)
> puts in it's regulations that you're not allowed to use AI to communicate with them (we all know this is coming),
.. do we?
> You get a mail from the government about taxes. You want to check if that text is generated by an AI
No, actually, I want to know whether it's correct and whether it's legally binding. Using AI makes it less likely to be correct, sure, but ultimately whether it's human, spreadsheet, or AI the important thing is the legal right to correct process.
Which is mostly a good thing because most of those processes were cooked up to be exclusionary but with plausible deniability
God forbid some upstart get a big contract they "shouldn't" have gotten because AI lets them create a submission that's on par with a big guys.
God forbid Joe Schmo be able to prepare the paperwork and dot his I's and cross his T's and be able to take advantage of some stupid zoning exemption that you spec'd out to essentially be only available to big moneyed interests.
The ability to suffer one's way through beurocratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI. But people don't frame it like for obvious reasons.
These are adversarial processes. You might get a win in the short term, but everyone else is playing the same game with access to the same tools, and developing a game-breaking strategy in a PVP multiplayer game is going to both escalate and get patched out.
A colleague of mine gave the example of using AI to snipe the UK driving test queuing system, which currently has a massive backlog. Which is great .. for them. Someone else gets pushed down by automated sniping. Then everyone else will have to start sniping. Then the website falls over because it wasn't designed to deal with 100m requests from the few hundred thousand people who are trying to bot their way to victory.
(I can see one likely patch out of AI spam being "you must submit requests in person 9-5, online system is closed down"!)
> I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.
Well, yeah Dario. You keep PUBLICLY TALKING about ways you want to fuck over people and everyone has noticed. I'm sorry, but when a lot of people are living paycheck-to-paycheck and watching layoffs everywhere, being like "well it'll take all the jobs and maybe kill humanity" is going to set off a LOT of alarm bells, despite your attempts to make concerned looking faces. You're trying to solve hypothetical problems in unproven ways, while worsening real problems people actually face. These people are SO disconnected from the society they live in.
I know people in positions of power often see lots filtered information that skews their perceptions of reality, but it seems like the level of pure hype Dario's pushing exceeds even the typical sycophantic filter bubble. Cure cancer indeed.
But instead of doing that, I'm going to drop everything and become a linux kernel maintainer. Learning about the internals of how RCU concurrency is implemented across subsystems is surely what I need to do right now!
(/j before I get crucified)
Anything big enough always is, no? The next scale of order in the hierarchy starts concerned itself with what's good for it, at the expense of the smaller parts.
The fact that government is like a giant organization that everyone lives in, the fact that it mostly doesn't fuck us all over in terrible ways, that's a testament to the amazing alignment of regulation and law imho
People have a problem with the efforts, not that they haven’t reached utopia yet. Doubling the efforts will not solve the problem, it will only deepen it.
Surely it will be quite dystopian future - AI will dictate how the money allocated, where there is only one private company left - Anthropic (as per Dario says)
The way he's coming off here though gives me SBF vibes.
The bio/cyber/alignment risk rhetoric is nonsense. Somewhat counterintuitively, all of those risks are lower/non-issues if frontier labs release their weights and do all of their research in the open, as explained in a past comment: https://news.ycombinator.com/item?id=49313739
Again: if we were to regulate anything, it would be to force frontier companies to release their model weights and do all of their research in the open. This is a necessary yet insufficient condition for the intelligence explosion to benefit the vast majority of people.
1. He is giving his word that if he cures cancer, he will brag about it as loudly as possible. Admirable dude! I actually trust him on this 100%
2. “We hope to have incredible results” -> doesn’t everybody hope to have incredible results?
3. “Some early glimmers in the coming months” -> is that planned for a few days before the IPO?
This guy says less than politicians do.
He hasn't learned the correct speak yet. He should instead say that he is _very confident_ that they will cure cancer until the end of the year.
Really, guys, learn from Musk, he is the master of promising everything, but not actually committing to anything.
Which is apt. His LLMs are slated to replace the work force, and he will replace the politicians.
I would imagine "hope" in this context is more expectation for him than the average person who might "hope" for something incredible but rarely do they base their entire career on it. Or maybe I'm just not too acquainted with silicon valley types. I always thought the whole delusional optimism was just a show for investors.
This is a good thing cause all politicians lie all the time :)
Most people start lying long before they reach any position of power and in many cases, lying is exaclyt how they got there. That's what makes it hard to fix: by the time someone is in power, the behavior has already been rewarded and is already doing a snowball effect. I'm not defending it (especially in Politics, it shouldn't be legal), just acknowledging reality that people lie about tiny things, so of course they lie about big things. It's more of a people problem, not only a people-in-power problem.
In practice, I doubt that they have a way out of all the lie-parade as well while being able to maintain their position, imagine if today any politician/CEO suddenly acknowledge "oh sorry guys, all that time we were just lying" => Career death (with snowball effect)
They call it The Republic.
Anthropic is the fastest revenue growing company in the history of Capitalism. I don't think they need any more marketing for investors.
Hence the "We're curing cancer, no, really" pitch - because within 6-12 months everyone else is going to be doing code and text cheaper and better.
What’s the source for this claim? Is this true in real dollars or just nominal? What about as a percentage of total GDP?
https://www.axios.com/2026/04/13/anthropic-revenue-growth-ai
and people say advertising doesn't work
The future is being held back by profit motives.
tl;dr - there is no conspiracy. It's just just hard.
PS: correction - not generics, but variants of Ozempic. But the point still stands.
Yes, I know someone patented a "cure" for hemophilia which repairs the DNA of said patient. But that's just because no other pharmaceutical thought it worth their while to fight the patent since the number of patients is small.
But I can guarantee you that a cancer cure patent that repairs the DNA will be fought in Court.
You can't patent "repairing a road" but you can absolutely patent a new mechanism to do so.
Most patents are of this form - there are far more patents for improved ways of doing things than initial ways of doing things.
Anthropic in particular has developed this almost Orwellian like veil of condescending rhetoric that on the surface suggests they're looking out for you while underneath they're taking actions that suggest they do not trust you, Mr/Mrs Ordinary Person. All the safety rhetoric, never supporting open weight models, the lockdowns on harnesses outside claude code, etc.
Anthropic if you care about public good, do something to empower people. Release an OSS model. Open source Claude Code. Open source some inference tooling or something. Just give people anything except your words.
Is this what kids call rage bait these days?
Because the reasons outlined in the article, plus many others. Concentration of power, unemployment, rising energy/living costs, stealing of digital artifacts owned by people.
I myself acknowledge that AI is useful. But due to the reasons above, I will be anti AI for the most part.
Yes, it doesn't matter if AI manages to cure cancer. Due to the reasons above, I will be anti AI. I suspect a lot of other people feel the same way.
"Please run N tasks doing X for Y duration, I want to test something"
"Hmm but that would waste tokens, we shouldn't do this test"
What the hell type of product is that? I wasn't asking, I was ordering it to do something, if Anthropic can't get their model to do as I say then I'll just go use someone else's.
If an ASI model is 100% aligned to user intent then you only need one person on earth to prompt "kill everyone" for an extinction-level invent.
There's no logical way around this. The model either has to ignore the person at the helm or we have to do a multi-national abort before RSI. Anthropic is trying #1.
― Terry Pratchett, Thief of Time
The AI industry and all of its sycophants in the Trump administration are working very hard to make you believe this.
https://www.forbes.com/sites/antoniopequenoiv/2026/06/10/ant...
https://www.washingtonpost.com/technology/2026/03/04/anthrop...
If MS Word had some kind of systematic bias that made it likely to corrupt the notes taken in it such that they tend to be more bloody battle plans, then that would indeed lead to some responsibility on the MS engineer's part for any mistakes that result from this bug. Similarly for Ford engineers if their vehicles had some bug that made them more likely to lead soldiers riding in them to the wrong destination.
If your system is designed to help choose targets, and it chooses the wrong target, especially with such tragic consequences, then your system is likely not fit for purpose and you should be held responsible for selling it to the army. And that is assuming it didn't actually have some bias + misalignment problem where it intentionally tried to kill children, and selected this school specifically because it gave some plausible deniability. For all we know, it's even possible that some bias like this was known about inside Amthropic and hidden from the public - unless an external investigation is made, that can't be ruled out.
If you provide tools that help an organization kill easier, and possibly lazily rely on your inaccurate tools' judgment, and you do not care about the outcomes of these tools' uses, that's on your soul, too.
Yes, causality and attribution are hard sometimes, but it may have been that without it, hundreds of humans would be alive right now, and he doesn't even know. Maybe he should talk to the kids' parents about what alignment means.
And we’ve seen unprofitability not being an issue for a lot of companies. Markets function in funny days nowadays.
I feel that everyone in this small AI bubble (compared to the "normies") is busy trying to take advantage of AI to surpass everyone else, while fearing being surpassed by everyone else.
Most people assume LLMs are just a tool for making money, an arms race over wealth and social rank. But those like Dario seem to be worried that LLMs could potentially be used as automatic rifles by some dedicated psychopaths. Whether it could be that dangerous, or just an exaggeration, or we should have everyone armed, require licenses to own one, or outright ban them, is debatable. But I can understand his point.
All I can say at the moment is that this is more subtle than the superficial conspiracy/black-and-white narrative.
This line of reasoning is really very silly. I can be looking out for my young child by choosing not to put them behind the driving wheel of my car.
There are both stupid and malicious people in the world. If you have a tool that you believe may be genuinely dangerous in the hands of the wrong person, it is not the smart thing to do to give the public unrestricted access to that tool. Do you think it would be sensible to give every individual on Earth a nuclear weapon to do whatever they want with?
This is actually a good point. Open models are getting better and better, some of them might even be useful in consumer hardware now. But if AI performance is still correlated with compute power, then no doubt power will remain with the people owning the chips.
The point "AI is structurally a technology that tends to concentrate power" is not that correct. They need this statement to be true, otherwise no way to justify the trillion evaluations.
Iran has hit the Amazon data center in Bahrain. The Ukranian deep drone bombing campaign has hit refineries, but also Wildberries, the "Russian Amazon" warehouses. The US campaign against Iran now, Iraq and Serbia previously, targeted power infrastructure. It will obviously be a target in the next war, and AI goes on that list too.
Every time the US completes an AI data center, someone in the Chinese nuclear command updates their target priority list. And vice versa.
You might have stumbled on the way to convince people that more data centers are a good thing
Electricity needs transformation to be useful. AI is the transformation layer, a sufficiently advanced AI system could secure an advantage in practically any economically relevant domain. If you have the best AI in world where embodied robotics are sufficiently advanced, you also have the best doctors, lawyers, engineers, logistics network, PR, weaponry, tactical leverage, etc.
How is it not true?
The biggest companies in the world are, quite obviously (just look at the numbers) going to be AI companies. The most powerful governments in the world are quite clearly going to be those that are close to (or in control of) AI companies. People have a very hard time seeing the second order effects of the control of intelligence - we need to fix that.
The outputs of models in AI datacenters are the work.
The electricity company does not get the entirety of my useful input to the work as a result of me using electricity, but an AI company does.
Are you self-hosting Google or Bing? No, but we have quite a huge ecosystem of full-text search tools with PageRank, with options to scale to almost Google scale (if you have the money). After all LLM training starts with the same crawl mechanism.
As long as barriers to entry is not too high (ie. it makes sense to take the risk to start a business that provides something similar - usually for a niche) market forces work.
We have the classic empirical chart reproducing microeconomics.
https://www.fda.gov/about-fda/center-drug-evaluation-and-res...
And setting up a pharma plant is also very capital intensive.
Here the obvious barrier to entry is completely artificial. (Which provides an incentive to spend a lot of money on R&D -- though it naturally raises the question of Pareto efficiency.)
1) in things like tax law, registering with city hall, dealings with the DMV, your phone subscription, insurance contract, ... you will find that one of the new fine prints in the contract will be that you're not allowed to use AI to communicate with them.
2) because of how SynthID works (you need the SynthID keys to verify, which are secret. So the only way to find if text is ChatGPT/Google/Anthropic watermarked is to ask ChatGPT/Google/Anthropic), government and large companies can enforce this against you. That is what the watermark is for. To end any insurance claim written by AI with "you're not allowed to submit AI written insurance claims" and refuse it outright there and then.
"Sorry your request was AI watermarked and pursuant to law 234 of 2025/03/11 chapter 3258 paragraph 33 decile 1299 we hereby close it without response"
3) when they reply, however, they use a custom model that also has custom SynthID keys. You will not even be able to tell their responses are AI written, or at least, you won't be able to prove it. You won't be able to enforce any AI-related rights (ie. the right to talk to a human) you have under the law against large companies.
In other words: this is to make sure that all the advantages AI provides are available to deny your unemployment claim, and to Verizon to charge you more, but completely inaccessible TO YOU when you want to change to a cheaper subscription. They can inundate YOU with AI-written requests BUT YOU CAN'T.
Self-hosting helps because it prevents them from verifying if your responses are AI written, because you can generate non-watermarked AI text and so there is a level playing field.
2) What actual law/regulation would that currently be that would be used for such an outright refusal?
3) Would that comply with current regulation?
It'll be spec'd out so that it's cheaper to bend over and take it than take it to court and prove them wrong.
Toll road contractors, military base housing, municipal utility providers, private prisons, auto towing/impound, private rail transit operators, utilities, etc.
Basically every company the government grants a captive market to abuses everyone in that captive market as much as they can get away with.
For government, because it's in law or regulations (ministerial decisions in Europe). For large companies "You agreed to it" (you know, like you agreed to allow Verizon to sell your location data to Palantir)
The other 2 questions I don't understand. My point is that the EU AI directive makes this possible. Makes it possible in ONE direction, while prohibiting the other. AI can be used by government and large companies to spam you and deal with you, and can't be used by you without being 100% up front about that to them (ie. enabling refusal)
Just agreeing to it isn't necessarily enough at least in some countries.
2) What laws and sections specifically makes that possible? Are there examples of that happening?
3) Where can I find that interpretation of article 50(?)? Some other article? (To the extent that things would need to be labelled/watermarked etc.)
No. Here is the list of organizations that have the power to make laws in the EU (and JUST the across-the-EU part of that list, within countries, within states, within provinces, within towns there's another list). This is referred to in legal tradition as the "Hierarchy of norms", because there is also a clear order defined.
https://eur-lex.europa.eu/EN/legal-content/glossary/eu-hiera...
however, AI doesn't really influence this. already there's a lot of problem with things like Ticketmaster, Apple's walled garden, abuses of IP law (patent trolls, DMCA trolls), etc.
the insurance industry is a prime example of this. the suffering caused by power imbalance is incomprehensible, and yet there's not enough political will to address this.
sure, it's easily possible that some important aspects of our everyday lives will be worsened by bad AI regulation. but IMHO this is wholly an upstream problem, it's a symptom of bad politics. (a byproduct of the Zip2 to Tesla to "democracy with roman salute characteristics" pipeline.)
that said, obviously the foundation to have any chance of a nonpatological market to exist is that self-hosting has to be legal.
Anthropic clearly are not though, and to call their operations wasteful is an understatement.
The #1 post on HN right now[1] is full of people jubilating about how they can run Qwen 3.8 27B on their > 5 year old GPUs. If that isn't democratization of AI, I don't know what is.
I'm sure he's smart enough to instantaneously realize this too, but as the famous Upton Sinclair quote goes, he won't mention it even if he does.
[1]: https://news.ycombinator.com/item?id=49324985
I think access to compute will matter just as much, if not more, as access to models.
> Even if we assume that models will no longer improve and we reach a point where everyone can run Fable in their laptop, surely running 1000x Fable agents would give you an advantage.
IMO, asymptotic advantages are marginal. At least for coding, we got a glimpse into how much of an advantage it gives (or doesn't) when the Claude Code codebase leaked[1] ~4 months ago. :)
[1]: https://news.ycombinator.com/item?id=47586778
If you want to leave it running with the fans going crazy for 40 mins or overnight or something, fair enough, but otherwise it doesn't seem worth it to me. It's certainly not "interactive", even taking into account the over-thinking it does by default.
The 3.6 (maybe they'll release a 3.8?) MoE model is much more usable (but obviously not as good) on this machine spec.
I'm not sure whether that's low or high, or how it compares to a general audience.
Making "tech" a career and a societal goal onto itself, without the adjoining understanding of and deep commitment to ethics and the responsible use of power, is why we're sliding into authoritarian rule by a small circle of techno-oligarchs.
We need less "move fast and break things" and more "plant trees you will not live to see bear fruit."
The AI labs don't matter. It's the hardware that matters. Everyone should have access to local AI capable hardware, not just these trillion dollar corporations. We just need to ensure consumers continue to get powerful hardware somehow.
I propose taxing the hell out of these hyperscalers and giving subsidies to individuals and small business to foment local and decentralized AI capabilities. Also ban these "we bought the entire chip production all the way through 2030" deals.
Anyway, it’s fun to watch, but remember to stay sane guys, be aware of the pendulum dynamics.
You sometimes see people on reddit amazed that "reddit's opinion seems to have changed". Nothing changed, just a different subgroup. You'll see the other opinion again if you wait for another similar article.
For example, the "did bother reading the article" crowd is attracted to certain headlines, and this probably isn't one of those.
It seems like a couple of years ago the HN and LW commentariat had a lot in common, but they’ve drifted so far in opposite directions that they look more like opposing political parties today.
https://www.lesswrong.com/posts/9RL9MuGZjzm4q3gKG/what-just-...
FWIW I think this is an artifact of site design more than anything else. A commentary site like HN in 2026 encourages hot takes and karma farming. All the "social media" with higher quality commentary like Substack and LessWrong center a large text blob of thought at the top and encourage reading more than writing. This puts up just enough of a barrier to write a hot take that you need to read the whole article and puts enough UI friction from leaving a throwaway angry comment that those folks will scroll away.
HN on the other hand all but encourages shallow posting. The article isn't posted, commentary is more prominent than the article itself. Leaving a comment takes 2-3 clicks and the comment editing flow excludes every other comment on the site, encouraging a write-first mentality. Ranking commentary based on upvotes offers a sweet dopamine hit over collecting karma. In 2026 these all encourage low quality engagement.
HN's UI has stayed relatively the same over the years while the Internet around it hasn't and I think its UI now acts as a low quality commentary attractor. LW changed its site design to stem the tide of low quality comments that show up here. LW comments certainly have their problems, but then, we humans all have problems :) On balance the community is much sharper and much more capable of having intellectually heavyweight, sober discussions on controversial topics. Of course nothing seems to trump my own friend network.
My explanation is - the technology is amazing and engineers admired it, but CEOs kept talking about an exponential curve while engineers on the ground started to notice signs of Gartner hype cycle curve.
We saw this play out with the social media era. Incredible technology but it’s now ossified into deteriorating closed platforms mostly filled with algorithmic, ragebait slop surrounded by ads. Fool me once.
Layoff.
I agree about the ethos and ethics of early hackers. But, I note that -by volume- most of the folks who do programming and nearly all of the people who are software-industry managers [0] are not hackers.
[0] I'm including managers at all levels, including the executive levels.
Mine is based on two things:
1. The fact that i grew up in poverty and never want anyone else to suffer that fate.
2. I was raised in a religious tradition that can be accurately described as socialist.
1) Those with cautious, but enthusiastic, optimism for the technology and an understanding that things never work ideally in tech and any advancement that moves the needle even slightly is worth embracing.
2) Those who are so dogmatically opposed to AI that they leave logical conversation at the door in favor of just repeating whatever negative gotcha they heard last when the topic of AI comes up.
I'll say that the ratio of these engineers seems to be about 10:1 in real life versus 1:10 online, at least in my experience, roughly speaking. I don't think I've actually ever interacted with actual "AI/Tech Bros" who are obnoxiously pro-AI the same way that many of the people who are "anti-AI" come off.
The data I'm working with is biased (I've been working in high tech for a while now), but I'm left to conclude that the people who love to hate on AI online are those who love to hate online while most other people aren't particularly interested in defending a technology online that they feel is inevitable and doesn't really need defending.
What do you mean?
Open Source, hacker culture, scorn of "the suits", the love of sci-fi and it's attendant politics -- the bulk of actual technical people have a long history of being very politically aware and deliberately eschewing money for their values. It was rarely Marxist Class-Conciousness type stuff, but it certainly was aware of the political environment we were in.
Secondly, if you look at what Dario has actually written, then the cure to the big conditions like cancer is in his view hopefully going to be a result of scientists working 10x faster and therefore making big Nobel prize paradigm shifting discoveries 10x faster. So Anthropic suggesting “we are going to cure cancer” doesn’t make a lot of sense to me. According to his original vision it looks like a bunch of groups now using AI independently producing breakthroughs which cure cancer.
I'm an AI advocate but that question makes me feel that AI is simply "very useful" rather than being a historical game changer for humanity.
The earliest used models in this category would easily 10x (what did I just say) the creation of one-off short scripts, but you're not writing a 30-year app out of just a bunch of short scripts.
The METR time horizons graph suggests we can now get 10x (ahem) speedup on solving most coding problems that take us a few hours and about half of problems that take 2 days. Amdahl's law bites: even infinity speedup on half your problems is only 2x overall.
If you let an LLM loose, with a huge budget, what's the biggest artefact it can make before it drowns under the weight of bad decisions? The C compiler and web browser headlines a while back? The maths papers we see now that solve problems which stumped the maths world for decades?
But this is the other side of the same coin: teamwork. One person getting a thing made in twelve months vs a team of a hundred, you can scale up fast with money when you have a proof of concept, and an LLM can make a lot of proofs of a lot of concepts even in free accounts.
I do agree that the port would've taken a lot longer without LLMs though.
The actual hard part is getting it into idiomatic, safe rust, and I don't believe the LLM port makes the full transition any easier than doing things the old fashioned way: a dual lang code base like Linux.
...
how so?
let's say Linus opens a branch, rust-temp, and in 2 weeks pushes ~15 million lines of code deleting most of the old C code. and then it gets merged in a few weeks. and then there's still a few months of the "merge window" and RC process. and each day folks report bugs, and automatic fuzzers make sure that both versions "behave the same".
I was pretty skeptical (still am), but what the Bun team did is pretty great so far.
https://bun.com/blog/bun-in-rust details the process, we see the results (Node.js compat [0], more than 3 thousand of issues fixed since since 1.3 [1], more than 900 issues fixed in ~24 hours [2], and it's live/in-prod [3])
> 50 dynamic workflows in Claude Code run continuously over the course of 11 days.
so let's say 500 workflows could do something similar for the kernel. the reported cost is 165K USD, so let's say this would cost 1.65M USD, plus CI costs [4] (which is ~200K/year for bun, so let's say it's 2M USD for the kernel)
How much the world is spending on kernel bug bounties and various security programs each year? (rough estimate says that just the visible kernel testing programs cost at least 15-30M / year)
The bar is not perfect. The bar is something better.
[0] https://xcancel.com/bunjavascript/status/2087436767054213517...
[1] https://xcancel.com/bunjavascript/status/2082298681223680002...
[2] https://xcancel.com/bunjavascript/status/2080882189663907859...
[3] https://news.ycombinator.com/item?id=48966569
[4] https://news.ycombinator.com/item?id=49070494
I'm actually basing my opinion off the blog post, specifically the code. While they do get a few changes for free with the port, most of the code looks identical. And the number of unsafe statements certainly agrees.
And now they're going little by little, playing whack a mole with seg faults... My point is that the step they're at right here is the important part, and that mechanically converting the entire codebase to rust was an optional step when incrementally rewriting would've worked just as well if not better.
Being "only" 165k USD doesn't mean anything, because you didn't do a complete port, you did a port in a trench coat. If they went for the more targeted module by module approach, and did a clean and proper rewrite, LLM assisted or not, they would reach their end goal much less turbulently and without the effort of that initial frankenport.
for me it was already "don't run in prod" quality before. (at least after this I'm considering adding it to the CI to see how it fares.)
segfaults. I don't know. I looked at their CI and GH issues over the past few weeks. (though now GH is down so I can't do a search, but I didn't see thousands of segfaults.)
by all accounts and measures it seems it made their house of cards more manageable. despite the frankenport, no?
they already had a zig compiler fork, wanted to upstream it, but the zig maintainer(s) said it's low-effort. now they don't have to maintain their compiler fork. (or wait for the zig team to deliver the features they wish for.) no need to maintain a hybrid codebase. (though it has C++ because of the embedded JSC.)
also I have no idea what's the zig-rust FFI status, but getting over with a rewrite faster is usually better, even if you are left with non-idiomatic code.
Long term stability is the thing that matters, such a move would be immensely threatening to long term stability.
the point is that it's a quite obvious opportunity that was just some pipedream years ago.
by the time v8.0 comes around we'll have more data on the bun rewrite. also we'll see how LLMs will affect the productivity of kernel developers, and the overall stability/maintainability.
You're not wrong. We're already seeing something similar with all the outages and buggy software updates these days.
We little choice in the matter, you will have ze (software) bugs!
Doesn't matter.
The vibe coded rust port did not get them any closer to full compiler verified memory safety. They still need to go file by file, bit by bit, and make it memory safe, at which point, why not just do that from zig.
because for the same effort you get a better result if you can and do use a better tool?
But I don't think they did it in a way that actually provided a meaningful gain. My point is that the state of the codebase right after the code is just a worse version of the original zig code with few of the rust advantages. Going from that Frankenstein rust code to actual memory safe rust is a similar leap to going from zig.
After an aquisition earlier this year I got the task of doing an SAP-Integration for the new company, last time I did this 5 years ago it was a 6 month task, but with the experience and skills ive gained since I estimated it would be a 3 month project (with or without AI, most work is just logistics, AI cant help much there).
In those 3 months I was able to not only integrate SAP but also deliver a completely modernised user-facing software for that integration. While I could have written that software myself in a vacuum it would have never been worth it financially, since it would have delayed the launch of the integration by 6+ months. Building the software post-launch of the integration would have easily taken 2.5 years at minimum.
But this is also basically a "spherical cow in a vacuum" scenario, where I was essentially acting as a solo dev, in full operational control of the project, with deep domain knowledge of the topic and an allready fully set up codebase that I knew perfectly while working down ideas I've had in my backlog for 5+ years.
What you have said is correct, it lets you build software much faster. The question however is: is that software making money for the company? (Not talking about what you built but in general)
I think, with AI, companies are saying yes to a lot of things they would have said No to ik say 2020. And as a result realizing “just building it” is not the answer.
Previously your GTM team or Product team would say “If we ship some big project X, we unlock $Y in revenue” but now people are realizing that those projections were really more of a hope. So companies are spending so much more tokens and shipping so many more PRs based on hope but a lot of it just doesn’t turn into meaningful revenue, especially not in short term
This sort of system only works with internal software and an unusual amount of data. If we were in the business of selling that software we could not have charged a higher price for the new version over the old, the tweak could only be unlocked because we were able to control staff hiring and staff onboarding fully to make use of the new changes.
As part of the aquisition I got access to their previous codebase which was some sort of incomprehensible PHP monolith, with the persons who wrote that code long gone. Thanks to LLMs I was actually able to extract the core useful concepts (again, sufficiently deep domain knowledge that I knew exactly what to look for). Without LLMs i would have probably extracted the absolute minimum and let the rest rot.
There is no reason a dev of comparable skill and domain knowledge would not be able to do that for what I built here.
By that point it might no longer matter though, but I suspect that such code would have a lot more exposed edge cases than one where someone actually thought things through before coding.
So yes, many aspects of my job are now 10x as productive, but turns out that improves my overall throughput only very little.
Second, the Internet didn't show up much in GDP and similar measures either!
But your point stands. Where are the amazing digital products/stuff? I get that it might take time to arrive as we scale up compute and learn new paradigms. But so much infra already exists (deployment pipeliens, everyone reachable on a smartphone) that we should be seeing something.
I'm definitely seeing indie-sized games that appear to have had significant input from AI, though I'm not sure the balance between AI for coding and AI for assets. My experience attempting this directly suggests that the current level they work at can make very simple games as one-shots, but anything more than trivial will produce outputs only as good as the developer's combined willingness to put in effort tweaking things and taking it all one step at a time, and their taste about what "good" even is.
I'm using spare credits to build and improve an isochrone map renderer, which I otherwise wouldn't have had time for (apart from anything else, I'd have had to become skilled in JS+wasm, somewhat of a pivot from iOS). This also requires taking it all one step at a time, having UX and UI taste.
Having lived through GeoCities since before it was bought by Yahoo!, taste is… well. Most people make things that nobody else actually wants.
I've done amounts of refactoring and fixes and written tooling that just wouldn't have happened before.
I'm not sure what amazing new stuff y'all expect but the amount of technical debt in my projects is actually going down, cause I can finally get good enough test coverage, including E2E/load tests that actually prove whether the software works and scales or doesn't - just last week I diagnosed issues with SeaweedFS failing under concurrent writes when backing Sentry and could swap it out for Garage in a day, caught by a monitoring tool I slopped together that integrates with the Sentry API, no issues since.
The environment around me has gone from drowning in tech/ops debt to sort of swimming and at least holding above water for now (cause nobody will pay for 5x more tokens).
It's also insanely good for prototyping and being able to actually explore various ideas and shoot the bad ones down quickly instead of handwaving and looking at a loaded calendar, alongside being able to address well bounded tasks in parallel, better than human developers can - like I can give 5 GitHub issues to the slop machine and have it fix all of the annoying bugs. Issue with how some data shows up? Just feed it the DB dump and let it find out what's up.
Some projects have gone from around 500 code tests to around 4000, and before anyone says they're meaningless, at least 5% of those have caught real issues and helped a bunch, alongside linters and other tooling (including some tools I wrote myself). I've also written both native utilities and some web platforms for myself, side projects that I never would have gotten around to.
I'm measurably more productive than I've ever been (since I did measure that, looking at my commits over the last 2 years) but also burnt out. Still, it's the kind of burnout that's the consequence of context switching and lots of work, rather than the kind that I had years ago, where I had to manually untangle deeply nested Spring Boot service logic all over the place at like 2 AM cause the made up deadlines were kicking my butt.
In contrast to others, I don't need to move the goalposts - the productivity for me is here and now. Any future models will just make it better, unless we experience model collapse.
Disclaimer: you do need a LOT of code tests and validations, otherwise it all goes to shit. Maybe I'm just extending how much time it will be until it goes to shit for me as well, but go figure. You also have to babysit the models more than anyone would like or should, most of my work usually has 20-60 minutes of planning before dispatching the agent.
Implement features at a viable pace and drown any unrealistic expectations in reports of CVE fixes - usually a version bump of dependencies, but quite often also some migration work within the app; write tools to make that work visible like I did with scanning and reports, and enjoy living without having to complain about how you can’t keep up with releases while letting your software rot until it’s a multi-year migration.
Alongside that, do the same for test coverage as well because it’s the humans that have been sloppy for years and didn’t work enough on the test harnesses or good enough coverage, introduce Testcontainers and load testing tools and anything else that makes sense.
Use their money for tokens to build software that is actually good and make your life easier and ensure that their shortsightedness won’t sink the product on a few years.
Of course you could also get fired if they’re stupid, but then I don’t imagine that working for those that are delusional is exactly great either, unless the goal is explicitly to make slop to get investor money in which case the whole thing will probably need to be rewritten from scratch anyways so who cares.
But I am just as amazed with how little real life consequence it seems to have! Even software houses were hit more by interest rates than by this magical revolution.
If I couldn't directly observe Fable in action, I wouldn't believe in AI.
What would produce huge "gains" (depends on who you ask) again is the automation of the myriad non-digital roles/companies, but good luck with that...
>> What's moving the goalposts? I am very much amazed at what Opus 4.8 can do. I push its code straight to prod.
>>> What's moving the goalposts? I am very much amazed at what Opus 4.6 can do. I push its code straight to prod.
>>>> What's moving the goalposts? I am very much amazed at what GPT5 can do. I push its code straight to prod.
>>>>>> What's moving the goalposts? I am very much amazed at what Opus 3.5 can do. I push its code straight to prod.
The way it changes the game is by lowering the cost of making radical bets so we end up trying more moonshots.
Or an example of MS - their main cost like most software companies are people, especially software devs, which are to be replaced by AI so on the surface they would greatly benefit from it. But their products are centered around helping out people do stuff on the computer. Why would you need that when the AI will do it better and faster directly operating on the data or using e.g. Python?
Maximalists: 10x
Minimalists: were always close to 0x individuals, so opinion rejected 99% of the time
Realists: 30-50% productivity gains
The last 30 years of tech has been about big, the dot com boom, the next unicorn, the magnificent 7 or 11 or whatever.
The thing is that the missing solutions people want have never been the one sized fits all multi Tennant we need 6 rounds of seed funding kinds of problems.
The "app boom" should have been a clue that smaller, niche services are viable.
IF tech were like construction, then most of HN, most of SF Bay Area, most of "tech" seems to be concerned with building the tallest building, or a massive housing development. Your average office worker is more akin to a home owner, who has a busted outlet, a sticky door, a dripping faucet, they just work around it because the resources to fix it (programers) are going to build the next big thing.
Those workers now have tools to address those problems, themselves.
I have lawyers and writers (far removed from tech) talking to me about cron jobs and scripts they are running that solve real problems for them. AI has allowed them to accomplish this.
These things arent showing up on GitHub. They dont make quarterly reports at a publicly traded company. The mountain(s) of change are happening by stacking grains of sand, not one solid rock.
Hey, I just found 3 niche businesses that I can run because of AI and clear a million a year after costs.
Fight club rules apply, you dont talk about your advantages.
You get lawyers talking about their AI use (sometimes) because they are protected by the requirements of the bar. You people in niche businesses, or where it's about personal productivity, talking about it cause you cant replicate what they have done. You get folks in the sciences talking about how they are using LLM's to generate the python & scripts that make their world function with easy (rather than be a bottomless pit of time). These are people who might talk about their AI gains, but often have better things to do and don't run in programing / tech circles.
Unless you seek these people out you're not going to find their gains. It would be pretty hard for someone like anthropic to even begin to quantify time savings from generated code by "non programers" - because they don't have the data.
A product still requires a lot of handholding and human thinking, at least if one does not want everyone even throwing a glance at it to immediately be repulsed by the usual AI slop tells.
> I'm an AI advocate but that question makes me feel that AI is simply "very useful" rather than being a historical game changer for humanity.
It absolutely already is a historical game changer on par with the Industrial Revolution when it comes to the amount of jobs destroyed and economies screwed up - and the impact will be even worse in 10+ years as existing seniors retire but no new seniors rise as AI has destroyed entry level career paths.
Which economies are already screwed up?
IMO it can’t ever be on par with the Industrial Revolution because AI can only really affect the information economy. Things people do with their hands/bodies have either already been automated or can’t be with current tech. If you’d asked people decades ago they might say no one will ever work in factories by 2026 because they’ll all be automated. It didn’t work out that way. I think AI will go the same way: absolutely game changing to some industries (of which software engineering will be one) but a great many will still survive with less dramatic changes.
If anything it might result in more focus on the human aspects. How many people out there earn their stripes putting together slide decks? In a world where an AI can put together the snazziest presentation you’ve ever seen in a heartbeat it’s going to matter more how you stand at the front of the room and present those slides than it does today.
Law abiding US citizens won't be able to run them, but you didn't solve anything other than making sure US citizens pay Sam or Dario.
I rather let citizens of my country exercise our own authority and agency when deciding what is necessary for good societal outcomes (LLMs aren't necessary).
I don't think it's on them. Proximally, it's not their job to deliver the end-use value; it's their customers' job (if they're doing it, it would be a strong sign of tying). More generally, they operate as a commercial SaaS entity (non-profit or not), and net benefit to the world is not what any company is structured for (even public benefit corporations have mainly indirect effects).
So I think that "criticism" is just a back-handed way of promoting themselves in messianic terms.
IMHO Anthropic, OpenAI et al continue the practices that make people resent companies (US companies): You have to be either connected or nominal (show of diversity) to actually participate in the opportunities or protect yourself from their predations. That's just the nature of knowledge work: you need a ladder of opportunities that start very early to participate, and you'd need to understand subtleties of engagement to see how you're being fleeced as a customer. The divide of rich to poor is not about money but both knowledge and access; it's felt even by long-time experts in the field who are left out or ousted.
“Yet Dario blames public negativity on a long-standing loss of trust in institutions rather than his own messaging.“
Dario’s arrogance and blind spots are really something to behold.
Nobody is worried about any AI company in the abstract the way people are worried about other companies in the abstract (e.g. an oil company might ruin the environment just simply as a byproduct of engaging in their core business).
Everybody worried about their government, their utility companies, their employer, their landlord, all the faceless BigCos they do business with, etc, etc. getting their grubby mitts on AI and then using it to screw them even more than they're already being screwed.
The whole system is set up to screw people right now and it does it as fast and as hard as it can. AI is see as bad to the extent that it lets the screwing happen at a higher tempo.
> Open-weights do help some with this but are nowhere near a sufficient solution
Which is exactly why Anthropic contributes nothing to, and actively pushes for roadblocks and regulations for open weight models. Can't allow any hope to the masses.
Only people with $$$$$ are allowed to touch Fable. Which of course we have have aggressive guardrails for in case you even try to use it for something dangerous like AI model developement. Plus we are going to retain all data submitted to it just in case someone is trying to be sneaky.
It's very dangerous to Anthropic's bottomline!
This was primarily a clever way to try to shore up their revenue numbers. By preventing folks who don't have seven-to-twelve-figure bank account balances from using the tools that were most expensive to build, they might actually stand to make money from those tools... once they're forced to do accounting that includes the cost to make those tools.
As a bonus, R&D cost for serving the fixed-price-subscription rabble can be massively reduced by providing them with older tools that have had the goop around them tweaked just enough to make the tool's output have a different "flavor". That class of customer will never know that they're being conned.
Or even for the HN crowd, when will we see e.g. "Mythos aided research discovers 10 new viable battery technologies"
Even worse, it has made a significant number of people's lives worse. Some major, like layoffs, some less major, like PC prices, some are just annoyed by AI slop.
And then Dario himself wouldn't stop yapping about white collar work being gone in X years...
I'm way more worried about these guys breaking the economy in such a way that enriches them and makes me poorer, than I'm about any existential risk these guys cook up in their thought experiments.
Like if you are making more money than god in silicon valley I guess you don't really fear losing your job or house, but for most people that would be quite bad.
I never hear these guys articulate what their concrete vision for the economy if they make ASI / AGI happen (one of those is the correct shibboleth and I don't remember which).
This is fantasy, and I believe the vast majority of those actually working in pharmaceutical R&D would agree. Contrast Amodei's opinion here to the Derek Lowe post I shared recently: https://news.ycombinator.com/item?id=49313367 . Derek Lowe's opinion is closest to my own experience: AI, whether it be traditional ML or LLMs, can help here and there -- the former to help you to triage paths to explore in a way that's a little better than intuition in some cases, the latter mostly to generate code faster in the code-dependent aspects of pharma research -- but neither of these things are significantly widening the main bottlenecks. I don't think the data exists to do so, especially since so much of pharma R&D is looking for higher and higher hanging fruit (i.e. exploring avenues for which a trove of relevant training data does not already exist).
The way to do this is with large automated wetlabs running experiments and generating novel training data on the behavior of cells.
It's with partnerships with governments and universities that educate the masses to the point where many of us are better able to contribute towards the project of growing the available knowledge in a useful way.
It's with more NIH funding and fewer anti-science propagandists attacking that funding so they can sell their snake oil without pesky facts getting in the way.
What's actually happening is our data centers are spewing carcinogens into places where humans will breathe them and we're lying to ourselves about how if we just keep going this way we'll cure cancer.
Stop making the world less habitable by trying to squeeze another drop of model utility out of the existing knowledge, and start helping with the creation of new knowledge.
It's going to take decades to ramp up. Let's turn the page and get started.
- frontier llm access means you are at an economic advantage
- a big risk of this is ongoing wealth concentration
- the "open weights" approach can't solve the problem of wealth and llm access being linked; you need compute too, and compute is expensive, thus "open weights" still favors the wealthy
- instead we need "objective and fair institutional processes", as this will allow small labs cook up their stuff while frontier labs get regulated
why would i care about what these smaller players do, if economic advantage = frontier model access? also, the reasoning around "why bother with open weights because compute is expensive too" seems like the kind of logic a motivated 12 year old could work their way around in 30 seconds. things are not either/or dario, you said so yourself.
seems like a 400 word corpo misdirection essay. par for the course.
To paraphrase Sinclair "It's difficult to make a man realize the issues with regulatory capture, if they're to be the benefactor of regulatory capture".
> A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice.
Not so sure. After all lack of the latter did help establish the current tech overlord rule.
Especially when said individuals are rich.
> ...
> completely exempt any company below a certain amount of revenue or model training costs from being covered at all
One could argue that "Frontier AI" company know they have nothing to fear from company with less than XM$ revenue, and so their support for this type of regulation is still a way to force regulatory capture. In any case, whatever regulation you support, it's a regulation that you didn't have to handle when you were growing, but that incumbent will have to deal with.
Whatever it is Dario's intention or not does not matter. Capitalism push to consolidation and the eventual regulation that will need to be applied to mitigate the externality from a new industry, will mean that their will only be a handful of "very big" winner. Same story since the beginning of the industrial era. Even if that's not what Dario personally want, it's in the best interest of the shareholders, which will force Anthropic to do everything it can to be one of the big one.
> Overall my view is that AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws).
The same can be said about a lot of other industry (aviation, oil, chip manufacturing, ...). Every country / union big enough will finance their own champion to try to keep a foot in the industry even if they are not the best.
Edit: Does anyone else notice the switching between dashes and em-dashes between paragraphs? Tells you a lot about the man, doesn't it.
That he doesn’t know how to use basic word processing, or even agent SKILLS.md or whatever it’s called now. Or maybe it’s just the next generation of vagueposting.
The CEO of Anthropic not using AI extensively would be news.
The tech execs waking up on a random monday:
> I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.
No shit Dario, no shit...
Also ... Never read the tweets of your heroes.
The distillations of Qwen 3.8 down to a 27B model are good, but they're not on a par with frontier models.
When Dario talks about open weights not being a solution this is what he means - if you don't have 400GB of VRAM lying around the fact that there's an open model like Qwen3.8-2.4T-A95B doesn't really help much. If we're not regulating how models are available, or making sure access is open, then RAM prices will mean everything concentrates on a few very rich companies.
DeepSeek V4 Flash 0731 is 167 gigabytes from the developer and as a GGUF with no additional quantization. It limps along on my 192GB M2 Mac from several years ago [0]. This model tests better[1] than Claude Opus 4.6 released in February. That's six months ago - what will be available 6 months from now?
https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731/tr...
https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
So yeah, enthusiasts aren't going to run frontier models on their gaming machines, but a small office could easily justify the $30k - $100k cost to run something like this at high speed. The small company I worked for routinely spent that kind of money on Dec Alphas twenty five years ago, and that's not accounting for inflation adjustment.
And this is completely discounting the advances smaller models are making. You're right that Qwen 3.8 comes in different sizes. However, Qwen 3.8 27B and Qwen 3.6 27B do run on gaming cards, and they're better than the frontier models from twelve months ago.
I have no idea what will happen in the future, but I wouldn't base my guesses solely on the largest open weight models.
[0] Yes, it's unpleasantly slow (5-8 tok/sec)
[1] Yes, benchmarks should be taken with a lot of salt.
Does it have to be? There are plenty of coding tasks, where it's good enough.
However, if you're having a discussion about access to frontier models, and using Qwen 3.8 as an example of how open weights is a solution, then you should be honest and accurate about what you're talking about. Making an argument like "People can run Qwen 3.8 at home. That shows open weights are great." is a bit disingenuous if you're not also making it clear that you're not talking about Qwen 3.8 Max (or that you have a beast of a PC at home :D ).
I suppose the road to technical hell is paved with marketers and grifters. :)
[1]: https://ollama.com/library/deepseek-r1:1.5b [2]: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-...
Exactly. There are common coding tasks that these models can adequately do. They are absolute trash for anything that isn't coding. And even with coding, they are so, so far behind frontier models.
For folks who care about quality discussion and writing, it's a tragedy that people are using microblogging tools to publish essays. The length restrictions enforced by and inferior formatting tools provided by these things make it difficult to have serious talks about complex topics.
Of course -for a subset of the population- those limitations are a feature, rather than a bug.
this guy need to stop vibeposting nonsense.
Or are we still doing the lawfare strat since you can’t get any meaningful lead on Chinese models?
How about the involvement in the Iran bombing that killed 160, out of which 120 were kids?
Forgive me if Anthropic doesn’t really rhyme with trust over here.
How can you restore trust by ignoring the past deception, bad will and abuse - Is Dario saying you simply ignore that, and deliver on an outcome later?
If I break my agreements of how to treat people and they do not trust me, it's on me to restore that trust through acting different and showing them I've changed. It takes ownership of my past actions too, not just shoving them under a rug.
If only money thirsty capitals didn’t drain the HBM/DRAM/NAND capacity to rush to build out all the data centers so that they can make money off of inference, driving up memory prices like mad man leaving consumers with not only no chance of local inference setup, but also higher price of consumer electronics. Of course it has nothing to do with regulation, more to do with the greed.
It is so tragicomic to see people whose lives are built around companies coming so close to realizing that everything they do is bad, and then at the last minute swerving aside to convince themselves that no, if they just do more of it and somehow do it "better", then it will all be okay. The reason people don't trust companies and believe they are cooking up some new way to screw them over is because that is what they are doing. If Dario or anyone else really wanted to dispel that perception there's an easy way: do a total 180 and start fighting against everything you've been pushing. But none of them will do that because they still fundamentally believe that what they are doing is good, and are unable to see that fundamentally it's bad.
Both founders of Anthropic and OpenAI have talked about how AI will replace many jobs, and did so with a sense of doom and urgency.
And yet...they are (pikachu face) surprised that people do not want data centers (which will increase utility costs for consumers) and graduates boo tech leaders at commencement ceremonies?
Tech leaders are clueless.
Nothing beats the hubris of the Silicon Valley CEOs and self proclaimed “elite”. I’m missing people calling out “we are better than that”… NOT
Dario is always deeply worried.
I read somewhere he is the Woody Allen of AI.
Why does this need a master?
Do we regulate bio geniuses for the danger they pose? No, we do not.
Do we regulate other tools like hammers and drills? No. Books? No.
if claude still can’t yet write and refactor coherent code (on its own, replacing software engs, just like he said last year), why would we expect it to… fucking cure cancer?
I’d really appreciate it if folks stopped paying more attention to the bloviating or whinging of AI executives and instead focused exclusively on their actions, for therein lies the true nature of their goals or vision.
Ta-da. Trust crisis solved.
That's putting it mildly, very mildly. The guy all but wrecked the entire world's DRAM supply chain by using negotiation tactics that, if there were any semblance of regulatory authority left in the US, would lead to criminal charges for market manipulation.
Arrest Sam Altman, he is one of the most dangerous people on the planet. Not a single shred of respect for the 99% and the consequences his actions have on them. (And similar things apply to the other controversial figure you mentioned, Elon Musk actively enjoys hurting people by the looks of it.)
"Cami Clark, the low-profile wife of Anthropic CEO Dario Amodei, tried to court Jeffrey Epstein as an investor for her “luxury porn company” and women’s health startup, the Wall Street Journal and emails in the Epstein files reveal, as the Journal’s reporting raises new questions about Clark’s influence over her powerful husband and his company."
What a slimy couple.
He's clearly focusing on being on the news to provoke emotions on people, preparing for the Anthropic big blockbuster IPO.
Dario, Sam Altman and others should instead be praying every night that this will take us to the Singularity VERY SOON, because if it doesn't, the entire country will blame them for absolutely destroying their retirement and the US' economy once this AI Datacenter bubble pops.
Looks like Amodi is laying the groundwork so he can re-run Eric Schmidt's "but it happened" playbook. Do whatever you can to make money, wait for the dust to settle and walk off into the sunset after one final "whoopie daisy, didn't mean to do that to you. sucks to suck."
Basically he is probably a nice guy and well meaning from his point of view. But, now that they hold a huge power to impact "on demand" AI usage and regulation, and they are kind of the tip of a monopoly on advanced AI, they will see themselves as more important than anyone else, and with the moral duty to impose their view on everyone that hasn't the worth as them.
Just imagine if the company was still a small outsider, their official point of view would probably be a lot different. Especially with something like "regulation is a barrier to entry"...
Make it "Apologies for linking to Amodei" and we're talking.
I hope they have better sandboxes in their biology lab than they did in their cyber security lab.
More seriously, these sociopaths will soon release lab-made viruses into the world to prove AI's capability and to force governments into regulation of open models. It should be obvious by now that with their savior complex, the goals (cure cancer, AI safety, save humanity) justify any means.
It's not quite Sam Altman levels of delusional, but it's up there.
This is not how to swing the pendulum back in a positive direction.
Want to make a difference? Make AI use little to no natural resources, stop eating up RAM, stop making everything expensive, etc. Otherwise, don't talk.
I guess it’ll be great for Anthropic to have the cure for cancer, but what’s that gonna mean for people with cancer? Funny how he doesn’t talk about that part.
Oh gosh, I can't imagine why that might be, could it be the history of screwing us over?
If you want to build trust, I have a simple suggestion that Anthropic or OpenAI could implement today, if they wanted, and it would make me trust them probably 1000x more than I do today. That change is quite simple: stop selling to governments and enterprise. If you really believe in this product, you can earn enough money from regular people, you don't need government money and you don't need to sell to those corporations we don't trust. Just sell it only to individuals directly and add in your TOS that no company can force employees to use your product, or you won't sell it to anyone who works there anymore. This would strongly signal that you don't want AI to concentrate power, since you won't sell it to concentrations of power. It would build a lot of trust.
Oh, you want to "cure cancer" instead? Of course you do. Why am I backing away slowly? Maybe because I'm pretty sure you're cooking up a way to screw me.