West Coast Stat Views (on Observational Epidemiology and more)

This is Joseph. It has been a while since we heard from Dean Dad. In his latest column he writes about Nikole Hannah-Jones and her concern about how sending her daughter to a school with educational challenges worked out less well then she'd hoped. As expected, his solution is quite pragmatic: > We wound up balancing an egalitarian sensibility with wanting our kids to get good educations by ruling out the bottom but ignoring fine-grained distinctions at the top. My sense of it was that the kids would do well at anything from a pretty good school on up, and we wound up living in pretty good school districts. We weren’t willing to make our kids martyrs to some sort of ideal, and we had no illusions about them saving otherwise-struggling schools on their own. But we didn’t want them growing up in bubbles, either. It seemed to work. This has the virtue of being the strategy I chose with my own son. Good family support and being engaged with a school that has a decent reputation, even if their specific learning philosophy makes me a bit batty at times. But it does raise the question of what to do about schools that are objectively failing. Matt Yglesias, who picked the same strategy as I did, has some nuance. But he still leaves out the hard part. Schools definitely have peer effects. There are students who are challenging in a classroom. It isn't easy to know how to handle this without creating new victims. There is also a very big social justice issue here. Buying ones way out of a school system that is in trouble is very objectively good for the rescued kid but it raises questions of why all students aren't being rescued. It's a tough problem. But ignoring it isn't the solution either. Mark is out sick but I look forward to his take when he is back.

The seminal parable of the AI doomster cult is the paperclip maximizer, a fairy tale (specifically, the magic mill Aarne–Thompson type 565) pretending to be a thought experiment. As with most rationalist writing, it has the logical trappings, but is at its core a story of magic, presupposing an omniscient and omnipotent machine no more grounded by the laws of nature and the principles of engineering than the box at the bottom of the ocean making seawater salty. Most rationalist doomsday scenarios, however, have a much more recent and obvious source: juvenile science fiction. For every existential threat proposed by these people, you can easily throw together a montage of clips from old sci-fi movies without even having to resort to deep cuts (though I do recommend The Satan Bug and its great Jerry Goldsmith score). The trouble with that approach (other than it being embarrassing) is that science fiction, particularly mass market science fiction, is an extremely poor way of thinking about technology. It leans heavily toward the sensationalistic and apocalyptic. Even in so-called hard science fiction, the demands of storytelling invariably take precedence. The influence of folklore and popular genres like westerns further distracts from serious thinking. But perhaps the biggest issue is that the appeal of science fiction (even in apocalyptic stories) rests largely in thinking about the possible, while any serious approach to technology starts with thinking about constraints, be they engineering or economic. There are exceptions. Recently, The Expanse did an excellent job building its world, at least the part not involving alien technology, around the constraints of actual physics. On the whole, though, constantly thinking in terms of "what if" instead of "how" is a bad way of approaching real-world problems. At this point someone in the audience is protesting, "What about all the great predictions science fiction has made over the years?" It is true that science fiction has a great reputation for prescience. It is also true that that reputation is largely undeserved. If you go back all the way to Jules Verne and look at virtually all of the famous cases, you will see that these amazing predictions almost entirely came down to ideas that were in wide circulation at the time of the writing, be they speculations appearing in the popular press or straightforward extrapolations of recent technological developments. Here too, there are exceptions. Arthur C. Clarke was genuinely ahead of the curve when it came to telecommunications, particularly the technological and societal impact of geosynchronous satellites, and Robert Heinlein deserves credit for his insightful take on AI in The Moon Is a Harsh Mistress, but on the whole, if you go back to the months preceding the writing of some famous "prescient" work of science fiction and look at what was being written about technological breakthroughs and their possible applications, you will see that while the authors may have added a great deal on the literary side, they seldom added much on the engineering side. This is by no means a damning criticism. No one expects this of other genres (as far as I know, for all their popularity, westerns were not responsible for any major innovations in cattle ranching). Science fiction as a literary art form has nothing to apologize for and it does deserve credit for inspiring an interest in science and technology, but given the extremism of the rationalists and the impact they are having, the extent to which they get their ideas from escapist literature and at best thinly disguised tales of magic is a genuine point of concern.

One of the keys to understanding Silicon Valley's profoundly dysfunctional culture is the bizarre antagonism toward expertise. You can find this stated explicitly in places like the techno-optimist manifesto and manifesting itself in the decision to hand countless important positions to men in their early twenties with no relevant expertise, and sometimes no apparent expertise whatsoever. It is an article of faith in this world that tech bros know more about medicine than doctors, more about finance than bankers, more about education than teachers, more about climate science than meteorologists, and the list goes on. It is no coincidence that the push for hydroxychloroquine started with Elon Musk and Larry Ellison. This would not be so bad if the press approached the opinions of Musk, Peter Thiel, and Mark Andreessen with an appropriate degree of skepticism, or at least some effort to bring actual experts into the conversation. Unfortunately, the combination of ill-informed but endless tech-sector money combined with credulous coverage has produced a number of huge disconnects between conventional wisdom and the consensus opinions of people who know what they're talking about. One of the most embarrassing of these gaps is in the humanoid robotics bubble. Billions of dollars are being poured into a design that the large majority of researchers in the field dismissed as lacking serious use cases years ago, and yet you could read a dozen stories of robots running marathons and winning dancing competitions without coming across one quote from an actual roboticist pointing out the limitations of the design. Perhaps an even larger and more consequential gap exists between the apocalyptic scenarios of the "rationalists" and the people who study these problems for a living. Bryan Cantrill explains just how bad things have gotten. From Fool's Expertise: > I listened to the recent Ezra Klein interview with Nvidia CEO Jensen Huang, > which seems to be something of a Rorschach test for AI doomerism: if you are > inclined to see doom, Jensen sounds dismissive, but if you (like me) do not > fear for the future of humanity at the (metaphorical!) hands of a computer > program, Jensen’s responses seem very reasonable. (With some exception: does > this guy really not know his own zip code?!) > > But I also think there was a lost opportunity. While Jensen rightfully > indicates that existing regulation (e.g., product liability laws) can act to > hold the frontier labs accountable for harms caused by their own products (a > point made more thoroughly by former FTC chair Lina Khan), > he didn’t challenge Klein enough. > > In particular, Klein repeatedly appealed to the authority conferred by AI > expertise; how could Jensen disagree with pioneers like Geoffrey Hinton? > While Jensen (rightfully) pointed out Hinton has been wrong in his predictions > (e.g., his infamously wrong 2016 prediction that radiology would cease to exist by 2021), > he missed an opportunity to explain why Hinton is wrong: it’s not (merely) > that Hinton is not seeing the future (or aspects of it) clearly, it’s that > Hinton is making claims that far exceed the range of his domain expertise. > > To take the radiologist prediction (conveniently made long enough ago that it > is now unequivocally wrong), we can fairly say that Hinton was wrong because > his prediction did not reflect what a radiologist actually does: had > Hinton bothered to ask the question instead of making an alarmist admonition > (Hinton explicitly said that no further radiologists should be trained in > 2016!), he would have learned that practitioners do much more than interpret diagnostic images! > Making image interpretation faster or better does not obviate the radiologist — to the contrary, it allows the radiologist to better use their expertise to > serve their patients! > > This pattern of Fool’s Expertise is repeated over and over again among AI > doomers: their experience in one kind of system (e.g., LLMs) lends them > unwarranted authority in another — authority that goes largely unchecked by > people like Klein who don’t delineate between different kinds of expertise. > > Fool’s Expertise is particularly perilous when discussing catastrophe, which > nearly tautologically involves long chains of causation that transcend > domains: fully understanding one link does not imply understanding the chain. > (This is why when the NTSB investigates an accident, the NTSB Go Team consists of experts across many different domains: they know that a > catastrophic accident may involve failures across several different domains, > interacting and cascading into broader system failure.) > > And to say it clearly: to the degree that AI-inflicted doom relies on > pathways into the real world, experts on those pathways must be consulted. > Is AI a cybersecurity threat? Please talk to cybersecurity experts — who > will be quick to point out that the ballyhooed OpenAI escape depended on a pedestrian failure of containment. > Are we concerned that AI is somehow going to fashion a novel pathogen? Let’s > please consult experts on viral synthesis — and listen to why they think that the fears are misplaced. > Or maybe it’s nukes that we’re afraid that AI will get a hold of? Great news: > seven decades of living with the possibility of global thermonuclear war gave > us not just extensive safeguards but also huge wells of expertise to draw > from!

Nicholas Grossman is definitely onto something...: > There have always been humans who thought: > > 1) The world will end soon > 2) It will happen in a cool, exciting way > 3) It’ll happen to me personally > 4) I’ll be a martyred prophet, or a warlord, or otherwise special > 5) People who don’t see this are lesser > > “Sexy apocalypse” is a great phrase for that > > [image or embed] > > — Nicholas Grossman (@nicholasgrossman.bsky.social) September 18, 2026 at 6:17 AM Grossman was commenting on this Techdirt piece by Mike Masnick and Dave Willner. The latter was the head of trust & safety at OpenAI which should make his comments about his former employer as or more newsworthy as Coxon's comments on Anthropic. > When we picture the end of the world, the first thing that springs to mind is typically the cinematic version of “things going wrong.” We picture the musical score rising to a crescendo as the climactic events take place. Our heroes are fighting to stop some big giant, unstoppable threat. Even if they are ultimately doomed, there is something noble and exhilarating in going down fighting. And if our heroes somehow survive failure and make it to the post-apocalypse, they usually still have great hair, even though civilization has collapsed. > > Put another way, in the movies, the apocalypse is usually pretty sexy. > > ... > > A lot of attention is being paid at the moment to an entire culture that has grown up around the “AI Safety” world that is intently focused on stories of doom and the threatened end of humanity. That’s big, attention getting, and splashy. The tabloid version of this culture — the group houses, the polyamory, the recruitment dynamics that a lot of people are suddenly discovering — is even splashier still. > > Taken as a whole, this specific version of AI safety is getting a lot of oxygen right now. It’s easy to see why — as a kind of “sexy apocalypse” these narratives, while frightening, are an exciting fantasy, even if they would be an awful reality. But that heightened fantasy is not what AI safety is actually about. > > It’s not about protecting us from the unlikely sexy apocalypse. It’s about protecting us from the far more likely, and far more numerous, stupid apocalypses along the way. > > This mismatch between cinematic safety and actual safety happens all the time. During the Manhattan Project, Edward Teller got some attention by worrying that the Trinity Test might set the entirety of Earth’s atmosphere on fire. Very cinematic! Very concerning! But, as with most things, the reality was much more mundane. Actual experts looked at Teller’s concern, ran the boring calculations, and ruled it out as a threat long before the actual test. That is how safety often works: boring, process-heavy work… that kills off the really sexy scenario. > > Instead of global immolation, the real risks came from otherwise smart people doing dumb shit, such as the infamous Demon Core incident, where physicist Louis Slotin exposed himself, and a number of others, to a lethal dose of radiation by trying to use a simple screwdriver head to separate a plutonium core from a reflector… and slipping. Incredibly, this was the second time a mistake like this had happened, with the exact same plutonium core, and Enrico Fermi himself had warned Slotin he’d be dead within a year if he kept it up. Turns out he was right. > > ... > > The [Hugging Face] incident wasn’t Terminator-style “rogue AI” doing anything. All of the mistakes that led up to this were, as Eryk Salvaggio points out at the Bulletin of the Atomic Scientists, fundamentally human mistakes. Employees at OpenAI deliberately disabled the guardrails, they gave the tool an effectively impossible task while telling it not to stop until it had accomplished the task, and they (accidentally!) left a door open for the tool to reach the internet and fan out. These are all human mistakes. Stupid, disappointing, easily preventable. Like using a screwdriver to prop open a plutonium core. > > And if we know anything, we know that humans will keep making mistakes like this, and not just because we’re stupid. Slotin wasn’t stupid. The OpenAI researchers who turned off the guardrails weren’t stupid. They were smart people doing dumb things. These sorts of “banal disasters” are the failure mode that we can and should be focused on minimizing because, while they can be just as damaging as the grand and cinematic versions, they are (self-evidently) far more likely to happen, because they only require stupid decisions to accomplish. No brilliant breakthroughs or novel technologies are required — just people doing stupid things because they’re overconfident in their own abilities — or overconfident because the talking machine confidently told them to be.