On Signal vs Noise in ML discourse and the culture that surrounds it

The last decade in Deep Learning is, I believe, the most profound and relevant field of research in Computer Science of my lifetime (so far!). The culmination of almost a hundred years of computer science and mathematics. Certainly, it was the most surprising and wonderful thing I have ever seen since first using a computer in ~2002.

Unfortunately, being too close to it and having been on x.com for over a year and a half now, it is really hard to not be deeply grossed out by the majority of the field, both on the research and application side, due to how bastardized and diluted it has become.

There are a few issues: a lot of the mainstream discussions are led by the product managers, the “DevRels” (that’s pretty much 99% of the “researchers” you follow who seem like they are active researchers, btw), the CEOs with no real technical backgrounds or contributions, the people managers / team builders and various lizard-VCs who are shoveled onto timelines and TVs.

Then come the leeches, here to generate content fast and efficiently to generate views and to capture attention and monetize it. Inaccuracies, straight-up lies, anything goes. Heck, the companies themselves do that! (see: 90% of OpenAI employees on X or any chart they’ve ever released.) The distribution between noise and signal is more uneven than ever. 1signal being: actual pioneers & passionate, truthful technical people, opposite to the hyper-optimized corpo-speak lizards and various “paid in status or otherwise” niche ML influencers ↩︎

Part of the reason why this is the case is that we’re in the most efficient, hyper-connected and memetic era in human history. Another one is the shape of internet cultures, how they react when they become bigger and bigger and how they dilute to keep growing. Yet another one is the hierarchy of values that the people in the industry have, but that’s a different rant.

But my point is that in previous eras, the public figures we had tended to be the actual pioneers. And even when it wasn’t the case, we hadn’t reached peak algo-slop degeneracy yet, so everything was just a tad slower, a bit more palatable (even if just as manufactured, at times). Even the popularizers tended to be experts in their domains (see: Feynman).

It is fair to say that a public face 70yrs ago and today has very different implications: being a “public” figure has grown in scale and influence like the amount of transistors in computer chips. And there is no doubt there are lots of incredible, deeply technical and tasteful people in these teams too. They’re just (mostly) nowhere to be read or heard.

If they are, the incredibly morbid and perverse age of modern social media will usually let you peer too deeply into their souls and you will (hopefully!) quickly realize how human and imperfect they are and/or how misaligned their actual incentives are. That shatters most hopes of having legendary figures who have deep, positive influences like previous eras.

There are still a few around, but again, the ever-optimized internet-powered machinery of corporations and ads and deals affects a lot of them (see: Karpathy, his influence on the microculture on x.com and ML, coining vibecoding, the Cursor shilling incident, regularly posting about companies he’s invested in). Others, like Sutskever has locked themselves away in massive startups raising billions of dollars and basically do not participate in providing important points of data and feedback to people so they can conceptualize accurately what is happening at all.

How many truly brilliant people are in these companies (or outside of them! the kind of people who aren’t interested in building personal brands but whose insights and world models would be highly beneficial) who will simply never have a public voice (because of personality or choice or otherwise) due to the nature of our social platforms and the fabric of what makes the companies running these platforms tick? (e.g. ad money, attention time / time spent on platform, growth at all costs, “number goes up” which twists all potential discussions).

I guess I am trying to complain in a way about the cacophony that is x.com in general. [^As sad as it is to admit it, x.com’s ML circles do hold a disproportionate amount of influence on public thinking.] The “niche influencers” who truly believe they have coined terms that have existed for years or decades, the performative SF parties, the people who flanderize themselves at the first taste niche celebrity… the overreactions and inflammatory statements to cash in a bit more monetization money. And then the hundred of thousands of others, most joining it in a weird parasocial way. All of that nearly perfectly blended with LLM-run accounts to rehash posts or generate some more engagement.

The sheer amount of (often harmful) noise you consume on X is actually staggering. Take a second next time some “event” is happening on your part of twitter and see the memetics forming in real time, people aligning along thoughts, reactions basically falling into a couple of buckets, people immediately re-using other people’s arguments 5mn later. It’s actually astounding. It’s gross (and also beautiful in a terrible way).

On a smaller scale of influence than the biggest pioneers of the field, the exodus of truly passionate & talented individuals who let you peer into their mind and what they work on without the perverse effects of social media and monetization destroying it or them, has honestly mostly almost completely disappeared from social media. There’s just too many eyes, too many incentives and too much power to be captured for most to remain sane for long. Every community seems bound to devolve into some algorithm-optimized amorphous blob of noise only efficient at making number go up.