The Evolution of the Physicist’s Picture of Nature - Paul Dirac
Dirac’s 1963 Scientific American essay traces physics’ conceptual evolution and champions mathematical beauty over empirical conformity. Eric Weinstein interprets it as a critique of rigid empiricism, arguing Dirac foresaw modern physics’ impasse and urged deeper, structurally driven theorizing beyond ad hoc fixes like renormalization—a “Right Freeway, Wrong Exit” moment.
Overview[edit]
In The Evolution of the Physicist’s Picture of Nature, Dirac presents a historical and forward-looking narrative of how the conceptual framework of physics has evolved, and conjectures how it may proceed in future theoretical development.
Dirac argues that major shifts in physics arise when more symmetric or mathematically coherent formulations supplant earlier, less unified ones. He traces the path from Newtonian mechanics to Einstein’s relativity (special and general) and onward through quantum mechanics, highlighting limitations and tensions in current theoretical structures.
He draws attention to two classes of difficulties facing quantum theory:
- Problems of interpretation or ontology (e.g., measurement, wavefunction collapse) — what he calls Class One difficulties.
- Failures or singularities in extreme regimes (high energies, short distances) — Class Two difficulties, indicating the breakdown of existing theories.
Dirac also critiques the prevailing use of renormalization techniques, viewing them as unsatisfactory patching of infinities, and suggests that a future theory should eliminate the need for such ad hoc fixes. He speculates on mechanisms by which constants such as charge or the speed of light might emerge from deeper structure, contemplates models of discrete field lines, extended electron models, or even a refined concept of “ether” consistent with quantum principles.
He emphasizes that mathematical insight and internal consistency should guide theorists, even when experimental support is lacking, and he cautions that the ultimate theory may require forms of conceptual change currently beyond our anticipation.
Key Themes and Quotes[edit]
- Dirac asserts: “It is more important to have beauty in one’s equations than to have them fit experiment.”
- He regards spacetime symmetry (the four-dimensional structure) as powerful but notes that in quantum measurement we necessarily refer to three-dimensional “slices.”
- He views current quantum electrodynamics and renormalization as interim tools, expecting that a future formulation will avoid their conceptual weaknesses.
Interpretation Through Eric Weinstein’s Lens[edit]
Eric Weinstein’s analysis treats Dirac’s 1963 essay as an argument about the enduring methodological problems in modern theoretical physics rather than as a historical reflection.
Central Claims of Weinstein’s Analysis[edit]
- Weinstein highlights Dirac’s distinction between a theory’s guiding idea and its specific formulation. He argues that experimental or mathematical inconsistencies in a given version do not invalidate the underlying concept.
- He presents Dirac’s emphasis on mathematical coherence as a counterpoint to the empirical rigidity that, in Weinstein’s view, limits theoretical creativity.
- Weinstein points out that major figures such as Einstein, Schrödinger, and Dirac often introduced incomplete or inconsistent models that later proved structurally sound once refined.
- He interprets Dirac’s position as a warning against confusing the failure of a model with the failure of the broader theoretical direction. The metaphor of Right Freeway, Wrong Exit refers to pursuing a sound conceptual path but settling on inadequate frameworks.
On X[edit]
2021[edit]
@lpfeed @monadical @lexfridman @Zev__Weinstein You aren’t understanding then. Paul Dirac made this point in 1963 (Sci am). I’d read that before trying to dismiss it. Theory vs instantiation.
Notice this style of article. It confuses the *instantiation* of an idea which experiment *can* probe w/ the idea *itself* which experiment *cannot* probe. This is one of the most basic errors in science, philosophy of science & science reporting.
Read Dirac’s 1963 SciAm essay.
I’m sorry but what’s being addressed is closer to Naive Mildly Broken Spacetime SuperSymmetry models based on SUSY extensions of the symmetries of flat spacetime. Which many, if not most, sane theorists didn’t believe. But that seems to be a mouthful to say. Hence this silliness.
The bottomline is that the scientific method doesn’t work on ideas. It only works on instantiations of ideas & executions of experiments. That is why I call the Scientific Method the “Radio Edit of Great Science”. It’s science’s Golden Calf. It isn’t how top science works at all.
So why do we keep making this error. Because the real issue is keeping out bad ideas and keeping order. The Scientific Method can be invoked selectively against loons and heretics and suspended selectively for those we believe in. Read Dirac on Schrodinger. Or Einstein&Grossman.
You will see that General Relativity actually has Grossman as a coauthor at the level of ideas. The main mind blowing insight is in a co-authored 1913 paper seldom discussed. All that changes after that is the instantiation. Science fetishizes instance over insight. So bizarre...
2024[edit]
Ya know, I disagree with @elonmusk here because I don’t know how he got to such a strong conclusion. I wish he would say more. Seems unwarranted.
But @martinmbauer is clearly also not right here either! Examples:
1915: Einstein’s first explicit equation for General Relativity was mathematically wrong; it set a divergence free 2-tensor equal to a non-divergence free 2-tensor. But it wasn’t fundamentally wrong. It needed a small fix reversing the trace component.
In the 1920s E. Schrödinger’s theory didn’t agree with experiment. Why? Because the spin wasn’t properly incorporated. It wasn’t fundamentally wrong, and was patched. Same theory.
In 1928, P. Dirac’s Quantum Field Theory gave nonsense answers? Why? A small goof conflating bare and dressed masses. Harder to fix…but in no way a fundamental error. The theory of Quantum Electrodynamics or QED still stands.
Etc. Etc.
Not a big deal…but this point is just so wrong as to be unsalvageable. Very curious error to make.
Martin (with whom I usually deeply disagree) is normally pretty great. But sometimes I think pretending that all outsiders talking about the current physics disaster are cranks, causes insiders to say very simplistic unnuanced and wrong things. This feels like that. And I’m not even a physicist.
It’s like the insiders don’t realize that the outsiders have any validity. All outsiders don’t immediately become cranks by virtue of disagreeing at a profound level with the abjectly failing communities from which they came.
[Note: this is *NOT* a gotcha. I fully expect Martin to realize the error and just admit it. No big deal. We all say incautious things. And this is just obviously wrong. Not an indictment.]
In physics, theories are "fundamentally wrong" if they're mathematically inconsistent or contradict experimental evidence
Here it means *doesn't feel right to me*
And Nature absolutely doesn't care for personal feelings
Not sure what's the argument here. I didn't say every wrong theory must be fundamentally wrong ?
My point is that personal feelings from 'outsiders' or 'insiders' (weird distinction) don't have any bearing on whether a theory is wrong or not
You wrote: “In physics, theories are "fundamentally wrong" if they're mathematically inconsistent or contradict experimental evidence.”
That is simply untrue. I mean it sounds superficially reasonable in a kind of Wolfgang Pauli hard ass way…but it is clearly wrong. And I gave 3 examples which I could be sure we both knew. I could have given 10 more without too much effort. Feel free to challenge them.
Combatting this hardline belief and any simplistic reliance on the Scientific Method was the entire point of Dirac’s famous 1963 essay quote about mathematical beauty being more important than agreement with experiment. We don’t appreciate Dirac’s revolutionary point if all we repeat is the quote. Here is the context for the quote which makes the argument against the danger of letting experiment or consistency dictate that something is ‘fundamentally wrong’ as you say in your reponse to Elon:
“I think there is a moral to this story, namely that it is more important to have beauty in one's equations than to have them fit experiment. If Schrodinger had been more confident of his work, he could have published it some months earlier, and he could have published a more accurate equation. That equation is now known as the Klein-Gordon equation, although it was really discovered by Schrodinger, and in fact was discovered by Schrodinger before he discovered his nonrelativistic treatment of the hydrogen atom. It seems that if one is working from the point of view of getting beauty in one's equations, and if one has really a sound insight, one is on a sure line of progress. If there is not complete agreement between the results of one's work and experiment, one should not allow oneself to be too discouraged, because the discrepancy may well be due to minor features that are not properly taken into account and that will get cleared up with further developments of the theory.”
P.A.M. Dirac
I have no illusion that the point will ever die. But I was scratching my head when YOU made it, just as I was scratching my head watching you and @CburgesCliff hosted by some guy who seems to rely on strawmanning and personal invective as his schtick or act. I find you are usually pretty reasonable. That discussion was painfully biased and was pretty anti-collegial low level internet bullshit in my opinion. Yuck.
Anyway, here is the source:
https://scientificamerican.com/blog/guest-blog/the-evolution-of-the-physicists-picture-of-nature/
Related Pages[edit]
- Eric’s Recommendations
- The Scientific Method is the Radio Edit of Great Science
- Right Freeway, Wrong Exit
- String Theory
- Quantum Gravity
- The Only Game in Town (TOGIT)
References[edit]
- Dirac, P. A. M. The Evolution of the Physicist’s Picture of Nature. Scientific American, May 1963. Scientific American link