From the ChatGPT subreddit: Gemini offers to pay me for a developer to fix its mess

Who exactly pays for it? Google? Or does Google send one of their interns to fix the code? Maybe Gemini does have its own bank account. Wow, I really haven’t been keeping up with these advances in agentic AI.

On one side, we have a trolley problem thought experiment involving hypothetical children tied to hypothetical train tracks and some people sending him rude emails. On the other side, we have actual dead children and actual hospitals and apartments reduced to rubble. I wonder which side is more convincing to me?

It’s the same pattern of thought as rationalists with AI, trying to fit everything they see into their apocalypse narrative while ignoring the real harms. Rationalists talk a good game about evidence, but what I see them do in practice is very different. First, use mental masturbation (excuse me, “first principles”) to arrive at some predetermined edgy narrative, and then cherry pick and misinterpret all evidence to support it. It is very important that the narratives are edgy, otherwise what are we even writing 10,000 word blog posts for?

OpenAI claims that their AI can get a gold medal on the International Mathematical Olympiad. The public models still do poorly even after spending hundreds of dollars in computing costs, but we’ve got a super secret scary internal model! No, you cannot see it, it lives in Canada, but we’re gonna release it in a few months, along with GPT5 and Half-Life 3. The solutions are also written in an atrociously unreadable manner, which just shows how our model is so advanced and experimental, and definitely not to let a generous grader give a high score. (It would be real interesting if OpenAI had a tool that could rewrite something with better grammar, hmmm…) I definitely trust OpenAI’s major announcements here, they haven’t lied about anything involving math before and certainly wouldn’t have every incentive in the world to continue lying!

It does feel a little unfortunate that some critics like Gary Marcus are somewhat taking OpenAI’s claims at face value, when in my opinion, the entire problem is that nobody can independently verify any of their claims. If a tobacco company released a study about the effects of smoking on lung cancer and neglected to provide any experimental methodology, my main concern would not be the results of that study.

Edit: A really funny observation that I just thought of: in the OpenAI guy’s thread, he talks about how former IMO medalists graded the solutions in message #6 (presumably to show that they were graded impartially), but then in message #11 he is proud to have many past IMO participants working at OpenAI. Hope nobody puts two and two together!

Hmm, should I be more worried and outraged about genocides that are happening at this very moment, or some imaginary scifi scenario dreamed up by people who really like drawing charts?

One of the ways the rationalists try to rebut this is through the idiotic dust specks argument. Deep down, they want to smuggle in the argument that their fanciful scenarios are actually far more important than real life issues, because what if their scenarios are just so bad that their weight overcomes the low probability that they occur?

(I don’t know much philosophy, so I am curious about philosophical counterarguments to this. Mathematically, I can say that the more they add scifi nonsense to their scenarios, the more that reduces the probability that they occur.)

There’s really no good way to make any statements about what problems LLMs can solve in terms of complexity theory. To this day, LLMs, even the newfangled “reasoning” models, have not demonstrated that they can reliably solve computational problems in the first place. For example, LLMs cannot reliably make legal moves in chess and cannot reliably solve puzzles even when given the algorithm. LLM hypesters are in no position to make any claims about complexity theory.

Even if we have AIs that can reliably solve computational tasks (or, you know, just use computers properly), it still doesn’t change anything in terms of complexity theory, because complexity theory concerns itself with all possible algorithms, and any AI is just another algorithm in the end. If P != NP, it doesn’t matter how “intelligent” your AI is, it’s not solving NP-hard problems in polynomial time. And if some particularly bold hypester wants to claim that AI can efficiently solve all problems in NP, let’s just say that extraordinary claims require extraordinary evidence.

Koppelman is only saying “complexity theory” because he likes dropping buzzwords that sound good and doesn’t realize that some of them have actual meanings.

I study complexity theory and I’d like to know what circuit lower bound assumption he uses to prove that the AI layoffs make sense. Seriously, it is sad that the people in the VC techbro sphere are thought to have technical competence. At the same time, they do their best to erode scientific institutions.

Username called “The Dao of Bayes”. Bayes’s theorem is when you pull the probabilities out of your posterior.

知者不言，言者不知。 He who knows (the Dao) does not (care to) speak (about it); he who is (ever ready to) speak about it does not know it.

AI research is going great. Researchers leave instructions in their papers to any LLM giving a review, telling them to only talk about the positives. These instructions are hidden using white text or a very small font. The point is that this exploits any human reviewer who decides to punt their job to ChatGPT.

My personal opinion is that ML research has become an extreme form of the publish or perish game. The most prestigious conference in ML (NeurIPS) accepted a whopping 4497 papers in 2024. But this is still very competitive, considering there were over 17000 submissions that year. The game for most ML researchers is to get as many publications as possible in these prestigious conferences in order to snag a high paying industry job.

Normally, you’d expect the process of reviewing a scientific paper to be careful, with editors assigning papers to people who are the most qualified to review them. However, with ML being such a swollen field, this isn’t really practical. Instead, anyone who submits a paper is also required to review other people’s submissions. You can imagine the conflicts of interest that can occur (and lazy reviewers who just make ChatGPT do it).

I have a lot to say about Scott, being that I used to read his blog frequently and it affected my worldview. This blog title is funny. It was quite obvious that he at least entertained, if not outright supported, rationalists for a long time.

For me, the final break came when he defended SBF. One of his defenses was that SBF was a nerd, so he couldn’t have had bad intentions. I share a lot of background with both SBF and Scott (we all did a lot of math contests in high school), but even I knew that it’s not remotely an excuse for stealing billions of dollars.

I feel like a lot of his worldview centers around nerds vs everyone else. There’s this archetype of nerds being awkward, but well-intentioned and smart people who can change the world. They know better than everyone else on how to improve the world, so they should be given as much power as possible. I now realize that this cultural conception of a nerd actually has very little to do with how smart or well-intentioned you really are. The rationalists aren’t very good at technical matters (experts in an area can easily spot their errors), but they pull off this culture very well.

Recently, I watched a talk by Scott, where he mentioned an anecdote when he was at OpenAI. Ilya Sutskever asked him to come up with a formal, mathematical definition to describe if “an AI loves humanity”. That actually pissed me off. I thought, can we even define if a human loves humanity? Yeah, surely all the literature, art, and music in the world is unnecessary now, we’ve got a definition right here!

If there’s one thing I’ve learned from all this, it’s that actions speak louder than any number of 10,000 word blog posts. Perhaps the rationalists could stop their theorycrafting for once and, you know, look at what Sam Altman and friends are actually doing.