Hidden assumptions

Six years ago I wrote a paper called “Remarks on the group-theoretical foundations of particle physics” that was later published in the International Journal of Geometrical Methods in Modern Physics, and can be found on the arxiv at https://arxiv.org/abs/2205.13390/. I wrote the paper after conversations with one of the editors of the journal during a meeting at the Isaac Newton Institute in Cambridge, just before the first covid lockdown. But the ideas go back several years earlier, to 2015, when I analysed the group-theoretical obstructions to quantisation of gravity, and realised that the conventional identification of the Dirac group SL(2,C) with the Lorentz group SO(3,1) was mathematically impossible. I therefore proposed the group SL(4,R) of all 4×4 real matrices with determinant 1 as the only symmetry group that could plausibly unite the quantum theory of Dirac with the gravity theory of Einstein.

In the intervening period, I have examined all sorts of alternatives, that are closer to various parts of various established theories, and therefore might have a better chance of being accepted, but in the end I come back to the fact that my first instincts appear to have been correct. The group SL(4,R) is the only group that can describe the symmetries of all fundamental theories simultaneously. It is the only group that has any chance at all of being “the” symmetry group of the whole of physics, on all scales from sub-proton scale to the cosmological scale.

I have identified five different scales on which this group appears to be the correct group to describe physics: (1) the gravitational scale of Einstein’s theory, (2) the classical/relativistic scale of Hamiltonian mechanics, (3) the quantum (atomic) scale of Dirac’s theory, (4) the weak (nuclear) scale and (5) the strong (sub-proton) scale. At different scales, different subgroups of SL(4,R) become more important, and others less so. In relativity, SO(3,1) is paramount, while in classical Hamiltonian mechanics it is SL(3,R). In Dirac’s theory it is SL(2,C), embedded in the Clifford algebra Cl(3,1), and on the weak nuclear scale SO(4) describes lepton and quark symmetries for three generations. At the strong scale, the group SU(3) is usually used, but in order for the strong force to generate the mass of the proton it is necessary to use a non-compact group, most likely SL(3,R).

I’ll start at the biggest scales, where SL(4,R) is the group of symmetries that is supposed to apply to General Relativity (Einstein’s theory of gravity). In principle, this is the transformation group between spacetime coordinates that are natural for two mutually accelerating observers, although many other interpretations are current. If it transforms spacetime coordinates, then it must also transform the components of the gravitational field, which in Einstein’s theory is a 10-dimensional field written as symmetric 2-tensors on a 4-vector for SL(4,R). This is an irreducible representation of SL(4,R), that breaks up as 1+9 for SO(3,1), a different 1+9 for SO(4), 1+1+3+5 for SO(3), 1+3+6 for SL(3,R), 1+3+3+3 for SU(2), and 4+6 for SL(2,C). Similarly, the same groups transform the components of the electromagnetic field, written as anti-symmetric 2-tensors. This representation is again irreducible for SL(4,R), and for SO(3,1), and splits in different ways as 3+3 for SO(4), SO(3) and SL(3,R), but splits as 1+1+1+3 for SU(2) and as 1+1+4 for SL(2,C).

In this formulation, the electromagnetic field is self-dual, and SL(4,R) acts on it as SO(3,3). It is therefore very clear to see the separation into two 3-dimensional fields, one electric and one magnetic, that mix together under all non-Euclidean symmetries of spacetime. The gravitational field, on the other hand, is not self-dual. Unfortunately, Einstein assumes it is self-dual, which means his theory, despite its name, actually contradicts the general principle of relativity. It is an effective theory for small deviations from SO(3) symmetry, but it is not correct in general. There are other theories that do a better job for other symmetry groups – for example, Asher Yahalom’s “Retarded Gravity” is an SO(3,1) theory that does a good job of correcting Einstein’s theory out to whole galaxy scales, where Einstein’s theory is known observationally to break down. But it is not a complete SL(4,R) theory, so cannot explain the cosmological constant. In other words, it explains dark matter away, but not dark energy.

Next I’ll turn to Hamiltonian mechanics. In its original non-relativistic form, it generalised the SO(3) symmetry of Newtonian mechanics to SL(3,R). This allows you to scale distance up, as you scale momentum down, in three directions independently, so as to transform circular orbits into elliptical orbits and explain that the underlying physics is independent of the shape of the orbit. This means that the 3-dimensional representations on position and momentum are dual to each other, and are no longer self-dual as they were in the Newtonian case. The duality is quantised by the Heisenberg Uncertainty Principle, which allows you to specify either position or momentum as precisely as you like, but not both simultaneously. Planck’s constant tells you exactly what the granularity of position x momentum actually is.

Relativistic Hamiltonian mechanics extends position to include time, and extends momentum to include energy, so that the symmetries of SO(3,1) and SL(3,R) all apply, and generate the full group SL(4,R). Any relativistic Hamiltonian theory therefore has SL(4,R) symmetry. Since the principles of relativity and of Hamiltonian mechanics are absolutely fundamental to the whole of theoretical and mathematical physics, it follows that every proposed new theory of physics must have SL(4,R) symmetry in order to be taken seriously. Again I emphasise that the 4-position and 4-momentum representations are dual to each other, not to themselves.

Moving down to the next level, the Dirac theory of the electron, and its extension to theories of quantum physics and quantum chemistry, we find the Dirac algebra as an algebra of 4×4 complex matrices, identified as the complex Clifford algebra of Minkowski spacetime. The complex structure here makes no mathematical sense, but is needed for including the weak force, as well as various technical constructions such as the Laplace transform. For pure quantum electrodynamics, however, we need only the real Clifford algebra Cl(3,1), which is an algebra of 4×4 real matrices, generated by SL(4,R) plus scalars. Incidentally, this is not the same as Cl(1,3), which is an algebra of 2×2 quaternion matrices, so the signature of spacetime in the Dirac model must be (3,1) and not (1,3).

This version of the Dirac algebra is generated by the gamma matrices i.gamma_0, i.gamma_1, i.gamma_2, i.gamma_3 and i.gamma_5, but the conventional choices for these matrices are not all real, so quite a different choice of basis is required in order to see the isomorphism with SL(4,R). In particular, the four real spacetime directions are quite hard to extract from the standard basis of the spinors. A subgroup SL(3,R) can be obtained by fixing the real “scalar” gamma_0.gamma_1 + i.gamma_2 + gamma_5.gamma_3, while SO(4) can be generated by a “left-handed” copy of SU(2) on i.gamma_0, i.gamma_5 and gamma_0.gamma_5 and a “right-handed” copy on gamma_1.gamma_2, gamma_1.gamma_3 and gamma_2.gamma_3. The chirality here is obtained by relating the generators to SL(3,R), not to SL(2,C), since it is SL(3,R) that treats gamma_1, gamma_2 and gamma_3 in three different ways.

Indeed, the conventional interpretation of SL(2,C) as the Lorentz group makes no sense. Each of the two copies of SU(2) in SO(4) extends to three independent copies of SL(2,C), which means that the choice of SL(2,C) in the Dirac model not only distinguishes left-handed from right-handed (neutrinos from electrons) but also makes a choice of one of the three generations. This is the reason for the difficulty in including three generations in the Standard Model, and in Grand Unified Theories based on it. It is the decision to make SL(2,C) fundamental for all particles, rather than Dirac’s original SL(2,C) which was specifically devoted to the electron, that is responsible for this difficulty. We must recover the original Dirac philosophy, that the SL(2,C) model is a model of the electron, not a model of quantum mechanics in full generality.

Well, I seem to have already included quite a lot of what should have been the next level down, that is the weak force. At this level we treat SO(4) = SU(2) x SU(2) as the important symmetry group. Since the space symmetries have been moved from SU(2)_R to SL(3,R), the group SU(2)_R is available for internal symmetries instead. My proposal is to use SU(2)_R for the three generations of neutrinos (where unbroken symmetry appears to hold), and SU(2)_L for the three types of charged particles (charge -1, -1/3 and +2/3), so that the other 9 degrees of freedom in SL(4,R) are available for mass parameters for these 9 particles. The conventional choice of the “third component of weak isospin” is a choice to treat the electron as fundamental, and the quarks as less important. But a full theory of three generations of leptons and quarks requires us to use all three components of weak isospin also.

Finally, we come to the strong force. In my 2022 paper with Manogue and Dray “Octions: an E_8 description of the Standard Model” (see also https://arxiv.org/abs/2204.05310/) we used SL(3,R) for the strong force gauge group, in place of the Standard Model SU(3). I have argued strongly against this on many occasions, most recently because I need SL(3,R) for the weak force. But what if we are both right? What if the strong force and the weak force are not independent parts of the model, but just different ways of looking at the same model? The main argument in favour of using SL(3,R) for the strong force, instead of the conventional SU(3), is that SU(3) is completely independent of mass, and yet it is said that the strong force is responsible for generating 99% of the mass of the proton. It is mathematically impossible to describe mass with a compact group, so the conventional SU(3) is inadequate for the task it has been given.

In the octions model, however, we only use SL(3,R), and do not extend to SL(4,R). Nevertheless, SL(4,R) is available in the model, so we might as well use it. The extra 7 degrees of freedom are: one real scalar, 3 compact degrees of freedom that can be identified either with weak SU(2)_L, or with the generation symmetry, and 3 boosts that extend from quarks to include leptons, and therefore could reasonably be identified with electron masses for three generations. In other words, SL(4,R) provides a model in which the weak and strong nuclear forces can be united in a single gauge group, that contains two triplet symmetries plus nine independent “fundamental” masses.

This discussion of the five different scales of interpretation of SL(4,R) leads us to the beginnings of unification. The Einstein and Hamilton pictures are very similar, and easily united, and quantised, with only one correction to existing theory required. This is the requirement to correct the false assumption that the gravitational field is self-dual. Of course, this is a very fundamental false assumption, and invalidates the Einstein field equations in their entirely, so won’t be easy to correct.

Similarly, the Dirac and weak/strong nuclear pictures overlap to a large extent. Two corrections to the existing Standard Model are required. First, the identification of SL(2,C) with SO(3,1) must be recognised as observer-dependent, and dependent on the observer’s choice of the definition of the electron as “the” free fundamental particle of matter. Of course, this identification is a fundamental assumption of the entire corpus of quantum mechanics and quantum field theory, so it won’t be easy to give it up. But it has always been known that this equivalence principle, like all others, is only a local equivalence principle, so we must just adopt the Copernican philosophy that we are not (despite our hubris) the centre of the universe.

Second, we must understand that the Yang-Mills assumption that gauge groups are compact is a mathematical assumption that removes the discussion of mass from the model in its entirety. Hence this assumption is not only useless, it is highly counterproductive, and must be abandoned.

So, now, let’s get to the take-home message. The symmetry group SL(4,R) can be used as a symmetry group for all theories of physics at all scales from the interior of a proton to the entire visible cosmos, provided three fundamental assumptions are corrected:

1. Self-duality of the gravitational field;
2. Equivalence of SL(2,C) and SO(3,1);
3. Compactness of the gauge groups.

That’s it. Once you’ve corrected those three fundamental errors, you can get back to shutting up and calculating.

When Hamilton invented quaternions in 1845, he was looking for a way of multiplying triples, which we would now call 3-vectors, representing things like position in space, momentum, angular momentum, and other physically important vectors. He found, as we all know, that he couldn’t do it with 3-vectors, but he could do it with 4-vectors. But the idea of unifying space with time was too far ahead of its time (if you get my drift), and did not catch on. Maxwell’s equations (1865) could be written in quaternionic form using Hamilton’s formalism, but Heaviside separated space from time again, and created the formalism of vector calculus that is now standard in physics.

The result of Heaviside’s iconoclasm was that when unification of space with time was actually needed in the theory of special relativity (1905) and general relativity (1915), Hamilton’s quaternionic method for doing so was largely regarded as a failed experiment, and forgotten in the dustbin of history. All that remained of Hamilton’s quaternions was the use of the letter q to denote position in space, because, of course, position was, to Hamilton, a quaternion. Physicists still use q to this day, but have forgotten why.

So, if you want to understand “curvature of spacetime”, as Einstein did, you’d be much better off following Hamilton (who, you have to admit, was a much better physicist and/or mathematician than you are) using Hamilton’s formalism, which means differentiating spacetime with respect to itself. Flat spacetime, by definition, satisfies the differential equation dq/dq = 1. Well, this is a tautology, so let’s write dQ/dq = 1, where Q and q are spacetime variables for two different observers. In general relativity as usually presented, one of these observers is us, and the other one is God, but it is perfectly possible to work out the equations without believing in the existence of God. In the presence of matter (m), as a variable source term for the shape of spacetime, the equation becomes dQ/dq = m. It doesn’t matter (pun intended) whether m is God-given (the standard assumption) or just an accident of our position in the universe (the truly relativistic interpretation, as advocated by Mach).

Now we need to understand what it means to differentiate one quaternion with respect to another. Sometimes physicists use a square to denote a derivative with respect to spacetime, just as they use a triangle to denote a derivative with respect to space. This unifies div, that is the real part of d(space)/d(space), curl, the imaginary part of d(space)/d(space), and grad, d(time)/d(space), with the time derivatives (no special names). Altogether there are 4×4=16 components in these derivatives, and we need to organise them and keep them under control.

Algebraists use Lie groups to organise differential equations and keep them under control. That’s what Lie invented them for in the first place. If you assume, as Einstein did, that the spacetime coordinates are arbitrary (dependent on the observer), then the Lie group that controls the spacetime derivatives is GL(4,R). The derivative dQ/dq is a linear map from one four-dimensional spacetime (q) to the other (Q), and the 16-dimensional space of all such linear maps is called Hom(q,Q) by mathematicians. It is the space of linear maps from spacetime to “itself”, so let’s just write Hom(Q,Q), with the understanding that Q is a quaternionic representation of spacetime. As a representation of GL(4,R), the space Hom(Q,Q) splits 1+15 into a scalar plus the “adjoint” representation of SL(4,R), represented by the matrices with trace zero.

All this is bog standard mathematics, and can be found in the works of Hamilton and Lie, written long before Einstein came on the scene. Which makes it all the more difficult to understand how Einstein came to be persuaded to use Q x Q instead of Hom(Q,Q). Well, actually, it is quite easy to understand, because if Q is a self-dual representation, then Q x Q and Hom(Q,Q) are the same representation, so it doesn’t matter. So if we assume that (God-given) spacetime “is” Lorentzian, or “is” Euclidean, and ignore the principle of relativity, then yes, you can use Q x Q instead of Hom(Q,Q), and still get the right answer. But then, you have to accept that another observer using different spacetime coordinates will get a different answer. As, indeed, observations of the wider universe confirm.

Which all goes to show that the widely used term “general relativity” is a misnomer. General relativity is not a relativistic theory. It depends on a particular definition of inertial frame that is specific to us as privileged observers. General relativity must use Hom(Q,Q), splitting 1+15, and not Q x Q, which splits into 6 Maxwell equations plus 10 Einstein equations. The latter give only a special relativistic theory of gravity, which is not invariant under acceleration. Einstein was, I think, vaguely aware of this, because he knew that he had not incorporated Mach’s Principle, and therefore could not deal with a rotating (i.e. accelerating) universe. This is important, because it means that the Strong Equivalence Principle fails in Einstein’s theory.

Mach’s Principle, as I think I already said, is the equation dQ/dq=m, which tells you how to use the curvature of spacetime (aka the gravitational field) dQ/dq to determine the mass m of the matter you encounter. Or, conversely, how to use the mass m of the matter you encounter to determine the gravitational field. But remember that this is a quaternionic equation, so m is a quaternion, not a real number, or even a complex number.

Anyway, in order to keep track of 16 variables at once I need some good notation. Let me take H,I,J,K as a basis for Q and h,i,j,k as a basis for q, so that I can use quaternion multiplication on both Q and q, and also distinguish the identity element H of Q from the identity element h of q. There are 16 partial derivatives, starting with dH/dh, which describes how the time measured by an atomic clock in an orbiting satellite differs from the time measured on the ground. This is the one you need for calibrating your GPS system, and ensuring that you don’t drive into a field instead of a motorway.

More generally, I will use h,i,j,k to describe our ordinary everyday time and space measurements, and because we are talking about gravity, I’d better distinguish the directions i (East), j (North) and k (up) from h (here). The interpretation of H,I,J,K depends on where the other observer is. I’ll take H (Him/Her) to be God, so H can be anywhere, and any size. So I could take H,I,J,K to describe the universe as a whole (if there is such a thing), or the interior of a proton, or anywhere in between. Which of the terms in dQ/dq (for large Q) or dq/dQ (for small Q) are important in any given situation depends on the scale, but all the terms are there on all scales, even if we don’t necessarily detect all of them.

The terms that appear in any given theory are determined by the symmetries of that theory. Special relativity has SO(3,1) symmetry, with Lorentz transformations in dH/di – dI/dh etc, relating the scales of His three directions I, J, K to the three directions i,j,k here. The group also includes three terms like dI/dj – dJ/di that describe changes in direction. So the Lorentz transformation correspond to the electric field, and the three rotations correspond to the magnetic field. Einstein’s theory of gravity uses the other 10 terms, including dH/dh already mentioned, three terms like dH/di + dI/dh corresponding to the Newtonian gravitational field, three more “diagonal” terms like dI/di, corresponding to “expansion” of the universe in three directions, and finally the symmetric spacelike terms dI/dj+dJ/di etc. But they don’t really split up 1+3+3+3 like this, they split 1+9 for the Lorentz group, and 1+1+3+5 for the rotation subgroup.

Anyway, this is enough to distinguish forces that involve H and/or h, and therefore involve energy transfer (6 degrees of freedom altogether, forming the electric and Newtonian fields), from the forces that do not involve energy transfer (9 degrees of freedom, forming the magnetic and tidal fields, plus a “scalar” dI/di+dJ/dj+dK/dk that distinguishes between inertial and gravitational mass), and the Higgs term dH/dh (aka E=mc^2), that does not distinguish between inertial and gravitational mass.

A better way to make this distinction between “real” (electric and gravitational) forces and “fictitious” (magnetic and tidal) forces is to make the distinction between time and space explicit, and use SL(3,R) for the symmetry group. This divides the 16 degrees of freedom into 1+1+3+3+8, where 8 is the magnetic/tidal fictitious force, and 3+3 splits attractive forces from repulsive forces, but does not split electricity from gravity. The two scalars 1+1 split naturally into one overall scalar in GL(4,R), plus a scalar for GL(3,R) inside SL(4,R). The former is Lorentz-invariant, while the latter is not. The former therefore seems to be (inertial) “mass” in the usual sense, while the latter seems to be a “gravitational” mass that is SL(3,R) invariant but not SO(3,1) invariant. But perhaps the words are misleading, and we need a finer study of the 2-space of SL(3,R) scalars to really get to the bottom of what it represents.

At this point I really think we need to re-introduce the Hamiltonian duality between position (q) and momentum (p). In quantum mechanics, this becomes a strict mathematical duality defined by Planck’s constant and the Heisenberg Uncertainty Principle. In other words, the space Hom(q,q) is mathematically equivalent to p x q and q x p. Hamiltonian mechanics is based differentiating p with respect to q. It is a generalisation of F=ma, which describes a force F as the derivative of momentum (3 components of p) with respect to time (1 component of q). So a force is something that lies in Hom(q,p), which is equivalent to q x q.

So the question is whether gravity is a force or not. If it is a force, it is described by Hom(q,p); if it is curvature of spacetime, it is described by Hom(q,q). If it is a force, the forces split 6+10 into electromagnetism and gravity. If it is not a force, then the splitting is 1+15, and electromagnetism and gravity form a unified entity. How are we to decide which is “correct”? Well, we could try changing scale to the quantum scale, and see what happens.

At the quantum scale, “spacetime” appears as a Euclidean space, because the path integral only converges in a Euclidean space. So we have to work with SO(4) as the (internal) symmetry group of quantum spacetime. A whole heap of nonsense is talked about how to “convert” Minkowski spacetime with SO(3,1) symmetry into Eucildean spacetime with SO(4) symmetry. People invoke “Wick rotation” as a magic incantation to do this, but even Peter Woit admits he does not understand the mathematics behind Wick rotation, and is not at all convinced that it actually works. That is why (he says) he is looking for alternatives.

Well, here is an alternative. The full symmetry group of spacetime is (as we assumed at the beginning of this post) GL(4,R), and every theory of physics must in the end be GL(4,R) invariant (or “generally covariant”). But in any given scenario, or theory, one or another subgroup of GL(4,R) is used for an “effective” theory that ignores the symmetries that do not play a significant role in that scenario. In quantum mechanics, only the compact subgroup seems to play a significant role, so that’s the one we use. It is SO(4). No problem, no drama, that just happens to be the group we need in this particular context.

This means that in quantum mechanics we split q x q and q x p into representations of SO(4). The two splittings are the same, as 1+3+3+9 (compared to 1+6+9 for SO(3,1)). Here 3+3 is compact, and 1+9 are the boosts. Both 3’s mix time with space, one in a left-handed way (multiplication by quaternions on the left) and one in a right-handed way (multiplication by quaternions on the right). Multiplication by i maps h to i and -i to h, so describes motion of a photon (from h here) in the i direction, and maps j to k or -k, so describes a left-handed or right-handed polarisation. All fine and dandy, and the photon is described by anti-hermitian matrices, as befits a boson. The rest of SL(4,R) is hermitian, so describes fermions. Again, we have three directions of motion, and 9 degrees of freedom altogether, so three “flavours” of neutrinos in each direction. Again, all is fine and dandy, exactly the number of particles that particle theory says there should be. We do not need the scalar, except for renormalisation of the theory, so let’s not bother with it for now.

Hence in quantum gravity, it does not matter whether we interpret gravity as a force (q x q) or a curvature (q x p), both give the same effective local theory, that agrees with general relativity on a Euclidean spacetime. But not on a Minkowski spacetime. But that doesn’t matter, because no single observer uses (or needs to use) Minkowski spacetime coordinates. Minkowski spacetime is only required when two observers communicate with each other, and is in any case inadequate for the task, because it cannot deal with two observers in different parts of the world (rotating around each other) trying to observe the same neutrino in their own frames of reference. Moreover, the Standard Model, based on Minkowski spacetime, gives the wrong answer for neutrino observations.

So quantum electrodynamics, or the theory of the photon, has symmetry group SO(4), but classical electrodynamics, or Maxwell’s equations, has symmetry group SO(3,1). To try and pretend that these are describing the same physics is just bonkers. The symmetry groups are different, so the physics is different. That is obvious. Sophistry like Wick rotation doesn’t help – it converts photons into neutrinos, so is obvious nonsense. You don’t need to read the highly technical literature on the subject, to get blinded with science, and follow Woit into thinking that Wick rotation “might work”. Wick rotation does not work. It is a classic case of proof by intimidation. Wick rotation, pace Oscar Wilde, is a useless attempt to interfere with a scientific law.

Similarly, the whole problem of quantum gravity boils down to a confusion between the spin (1,1) representations of SO(3,1) and SO(4). The former cannot be quantised, but the latter can. The splitting between classical electrodynamics and classical gravity using SO(3,1) is different from the splitting between quantum electrodynamics and quantum gravity using SO(4). They only agree on a subgroup SO(3), which splits the whole of q x q (or (q x p) into 1+1+3+3+3+5. You may think you can choose a particular 3+3 for electromagnetism, and leave the rest for gravity, and you may think it doesn’t matter which 3+3 you take. But you’d be wrong, because nature got there first and made the choice for you.

Full unification of the forces of course requires us to understand how the nuclear forces fit into this picture. Most of the above goes through either in the force paradigm (p x p) or the curvature paradigm (p x q), because both SO(3,1) and SO(4) fix an isomorphism between p and q. But the nuclear forces are (mathematically) of the form q x p, not q x q, so only the curvature paradigm is likely to be of use in this case. The irony that Einstein used the (mathematical) force paradigm, while his theory is nowadays interpreted as curvature, is quite delicious. There is a converse irony in the theory of nuclear forces. But that needs a whole new post, I think, because to debunk the Standard Model requires a thorough investigation of the confusion between SO(3,1) and SL(2,C), as well as the confusion between SO(3,1) and SO(4) that we have already addressed. And it needs a translation from differential (Lie group) equations to algebraic (Lie algebra) equations, in order to be able to quantise everything into discrete “particles”.

Hamiltonian mechanics is, as you know, just a reformulation of Newtonian mechanics in a more sophisticated mathematical language. One of the features of this reformulation is that there is a symmetry of the equations that swaps position with momentum. In mathematical terms, this symmetry is called a duality. It is obviously not a physical symmetry, because position and momentum are completely different concepts, but it is a mathematical symmetry that is very useful in physics, especially quantum physics.

To explain this duality, we must first note that position and momentum are only relative. So we need to measure position relative to a particular point, that we might call “here”, expressed mathematically as 0. And we need to measure momentum relative to a particular momentum, that we write as 0, and we might call “still”, or in physicists’ language “at rest”. Then the position vector P has three “degrees of freedom” (how far away is it, in the North, East and up directions?), and the momentum vector M also has three degrees of freedom. The duality is an invariant scalar in M x P, so has units of angular momentum.

That doesn’t mean it actually “is” an angular momentum that you can actually measure, it means it is a natural unit of angular momentum, that is taken in quantum mechanics to be Planck’s constant. The principles of quantum mechanics imply that angular momentum comes in units of Planck’s constant, and any angular momentum smaller than this simply does not exist. The consequence of this is that if you try to measure both the position and momentum of something simultaneously, there is a limit to the simultaneous accuracy of the two measurements, given by Planck’s constant. This is the Heisenberg Uncertainty Principle, which, as physicists know, is just a quantum version of Hamiltonian duality.

This doesn’t mean there is a limit to how accurately you can measure the position of something, or a limit to how accurately you can measure the momentum. But if you measure the momentum of a photon extremely accurately, by measuring its frequency with an atomic clock, then you have no idea where it is.

Anyway, mathematicians have three equivalent ways of describing this kind of duality. One is M x P, as above. Another is Hom(M,M), which means transformations between two momentum vectors. If you think of this as an actual physical change in momentum, then by Newton’s law (F=ma), it is equivalent to a force. The third description is Hom(P,P), which means transformation between two position vectors. If you think of this as an actual physical change in position, it is a distortion of space.

For example, any force, such as the force of gravity, can equally well be interpreted as a distortion (or “curvature”) of space (or spacetime), rather than a “force” in the Newtonian sense, and can equally well be interpreted as a duality, in which case it can be quantised in terms of Planck’s constant. Hamiltonian mechanics, combined with representation theory of transformation groups, implies that these three interpretations of the equations of motion are equivalent.

So what is the problem? Why can’t curvature of spacetime be quantised? There is no mathematical obstruction to doing this, so what has gone wrong? I’ll tell you what has gone wrong – somebody made a mistake, and refuses to own up to it. To be fair, the mistake was made more than 100 years ago, and the perpetrators are all long dead. But that does not excuse us for indulging in ancestor worship instead of correcting their errors.

To explain properly, I need to explain a bit more group theory and representation theory. The first question is, what transformations of position coordinates are allowed in Hamiltonian mechanics? The usual answer is the group SO(3) of coordinate rotations, but that is a Ptolemaic answer, that says everything is made of circles. We must listen to Kepler and allow ellipses. The equivalence of the mechanics of an elliptical orbit to the mechanics of a circular orbit is fundamental to Newtonian mechanics. So we must extend the group from SO(3) to SL(3,R). Then the duality between the two 3-dimensional representations M and P of SL(3,R) can be expressed either in the language of forces, as Hom(M,M) -> 1, or in the language of curvature, as Hom(P,P) -> 1, or in the language of quantum mechanics as M x P -> 1.

All three interpretations are represented as 3×3 real matrices, with the duality being associated with the scalar matrices. So it is easy for a mathematician to translate between the three. The 9-dimensional representation MxP splits as 1+8 into the scalar plus the adjoint representation of SL(3,R). If we find the group SL(3,R) difficult to understand, we can restrict to the rotation group SO(3), in which case the 8 splits as 3+5, which may be easier to interpret.

Let us begin with the force interpretation. Notice that the group acts on momentum, but not on the energy coordinate of the 4-momentum. Hence it does not contain the electric or gravitational forces themselves, which do change (or “transfer”) the energy. What it does contain therefore is the (3-dimensional) magnetic field and the (5-dimensional) tidal field, which do not change the energy of the “test particle”, but only the internal “shape” or internal dynamics. This is bog standard 19th century Hamiltonian mechanics. But the unification of the magnetic and tidal fields into an 8-dimensional irreducible representation of SL(3,R) is not. This is the new ingredient that comes from adopting a Keplerian approach. In the Ptolemaic system, electromagnetism and gravity remain completely separate; only in the Keplerian system are they unified.

Let us now consider the duality or quantum interpretation. The magnetic field is quantised by photons, but without energy transfer. So we get only 3 of the 6 degrees of freedom normally associated with the photon. We have a notional direction of motion, together with a polarisation with respect to that direction. What is it that quantises the 5-dimensional tidal field? A “spin 2 graviton”? We’ll come back to that question later, after we’ve included energy and time in our analysis.

Let us move on to the curvature interpretation. Here the splitting 3+5 represents the separation of the rotation part of the distortion of space from the stretching and squashing part. Again, in the Ptolemaic system, these are different concepts, and kept entirely separate. Only the Keplerian system unifies the two into an 8-dimensional whole. Einsteinian gravity sticks to the Ptolemaic paradigm, and keeps 3 separate from 1+5. The 3 (rotations) is what is needed for Mach’s Principle, so that Einstein’s theory of gravity does not adequately implement Mach’s Principle. Instead, Einstein unifies 1+5 to include a cosmological constant (independent of time and energy). BIG mistake, which implies a BIG BANG!

I’ll tell you why it is such a BIG mistake. It is because Einstein unified 1+5 into a representation of SL(3,R), that is the representation on the symmetric part of M x M (stress tensor) and P x P (metric tensor), and used the fact that these two tensors are mathematically equivalent to equate them physically. But M x M does not describe a force. Nor does P x P. Forces are described by M x P. Only in the Ptolemaic system are P and M equivalent representations. Only in the Ptolemaic system can you do what Einstein did.

Hamiltonian gravity, based on the Keplerian paradigm, unifies 3 with 5, so implements Mach’s Principle without a cosmological constant, implements tidal forces exactly as in Newtonian gravity, and unifies tides with magnetism. Let me say that again: Hamiltonian gravity unifies tidal forces with magnetic forces in the 8-dimensional adjoint representation of SL(3,R).

I’ll pause to let that sink in, before we move on.

Hamiltonian gravity unifies tidal and magnetic forces. It must therefore also unify direct electric and gravitational forces.You will have noticed that I have not yet mentioned time or energy, only space and momentum. But of course the standard formulation of Special Relativity unifies time with space, and energy with momentum, into a four-dimensional Lorentzian spacetime and 4-momentum, with symmetry group SO(3,1). So we can do the whole analysis again in this larger group, in which case we get the full Ptolemaic epicycle version of Einsteinian gravity, complete with curvature of spacetime, cosmological constant, and infinitely many dark matter epicycles to deal with the fact that the Ptolemaic system does not get to the heart of the matter.

Then we can use the Keplerian insight (aka the principle of general covariance) to extend the group to SL(4,R), and unifying all four forces: electricity and gravity, magnetic and tidal forces. But wait a minute, that is only 3+3+3+5=14 dimensions, and SL(4,R) has 15 dimensions. Where is the 15th dimension? What is it? It comes from the fact that the 9-dimensional representation of SO(3,1) splits as 1+3+5, in which the 3 is Newtonian gravity (with change of energy), and the 5 is the tidal field (without change of energy). The 1 is what physicists call “dark matter”. But it isn’t “matter”, it is a force.

Hamilton said so.

The Principle of Relativity is a basic principle of physics, that physical reality does not depend on the observer’s point of view. In particular, it says that the equations of physics must always give the “same” answer, regardless of the coordinate system that is used for space and time. The original “special” theory of relativity was designed to deal with the special case when two observers are moving at “constant velocity” with respect to each other. The solution that was adopted was an algebraic one, based on the Lorentz group that transforms between the “natural” coordinate systems for the two observers.

In “general” relativity, on the other hand, designed to deal with arbitrary relative motion of two observers, the solution is usually presented geometrically, in order to deal with the case when the observers wander about all over the place. The way it is usually done, the relative motion is made up of small segments of motion at “constant velocity”, pieced together, to give the appearance of arbitrary motion. The problem with this approach is that where you join the segments together, there is infinite acceleration, and there is then no mathematics that can tell you what happens during that (unphysical) infinite acceleration.

What you actually need to do to get a proper theory of “general” relativity is to consider small segments of “constant acceleration” as well. Then you will find, when you do the mathematics, that you have to separate the cases where the acceleration is in the direction of motion, and perpendicular to the direction of motion. Then you have to extend the Lorentz group, that deals with the “constant velocity” segments, to include the “constant acceleration” segments. When you do this calculation, you will find that the group of coordinate transformations that you get out is SL(4,R). Of course, this is still only a “local” transformation group, applicable in the situation in which the acceleration can be treated as constant. But it is enough to rescue the theory from the “ultraviolet catastrophe” of infinite acceleration.

Physicists will often say that Special Relativity is “really” a theory of electromagnetism, and that General Relativity is “really” a theory of gravity. What this means is that once you have decided on what the group of coordinate transformations is, you can use it to transform the “force field” as well. In the case of Special Relativity, this is very well understood, and has the effect that if one observer observes a static charge, with an associated electric field but no magnetic field, then a moving observer will observe a magnetic field, and a changed electric field. The reason this is so well understood is because we use it to generate electricity with which to power so many things.

The mathematics behind this is the simple fact that when the Lorentz group acts on spacetime coordinates in the “natural” way (technically, the spin (1/2,1/2) representation), then there is a corresponding action on the six components of the electromagnetic field, that is, technically, the spin (1,0)+(0,1) representation. Leaving aside the technicalities, what it says is that the “internal” symmetries of the electromagnetic field are described by the same group as the “external” symmetries of spacetime. That is what the Principle of Relativity says: the group of symmetries of the “external” variables that a particular observer uses to describe spacetime is the same as the group of symmetries of the “internal” variables that describe the ways the physics behaves.

I cannot emphasise this enough. The group of external symmetries is the same group as the group of internal symmetries. That is what the Principle of Relativity actually says. But it is not enough to have the same group, you also have to have the correct representation.

The correct representation is specified by the Action Principle, which says that the action of a force field lies in the space of linear transformations on energy and momentum. This is mathematically equivalent to saying that the action lies in the tensor product of the 4-momentum with its dual, that is spacetime. If your transformation group is the Lorentz group SO(3,1), then this tensor product splits as 1+6+9, which it is natural to think of as 6 for the electromagnetic field, and 1+9 for the gravitational field. So far, so good. Then you may notice that the 4-momentum is mathematically “self-dual”, so that you might as well tensor the 4-momentum with itself, and the spacetime with itself. It looks innocuous, to a mathematician. So it should be OK, shouldn’t it?

No, it is not OK, because it is not compatible with the Action Principle, that is the founding principle of Hamiltonian mechanics. It may work if you only use SO(3,1) coordinate transformations, but we need to use SL(4,R) coordinate transformations. Then spacetime and 4-momentum remain dual to each other, but are no longer self-dual. What GR does is to use the 2-tensors of spacetime and 4-momentum separately, which split 6+10. What the Action Principle tells you to do is to use the tensor product of the two, which splits 1+15. What this means is that, if you treat gravity as a force, you have to put it in the 1+15 representation..Since GR puts it in the 6+10 representation, GR does not treat gravity as a force.

What annoys me about this is not so much that the theory is wrong (which it is), but the vehemence with which physicists will tell you that gravity “is not a force”, as though this were a fact about physical reality, when it is actually a fact about the mathematical theory. It is, moreover, a fact about the mathematical theory that tells you the theory is incontrovertibly, irredeemably, utterly wrong, from the ground up. The combined electro-magnetic-gravitational field is an irreducible 15-dimensional representation of the coordinate transformation group. It does not split into 6 dimensions of electromagnetism and 9 of gravity. It is not possible to have a correct theory of gravity without including electromagnetism within it.

And what makes it all the more shocking, is that it is an elementary, schoolboy error, made by mathematicians (geometers, to be precise) who did not understand the basic principles of Hamiltonian dynamics. To construct a correct theory of gravity, that reduces to GR in the special case of uniform observer motion, you only need two fundamental principles: the Principle of Relativity and the Action Principle. GR conforms to the Principle of Relativity only, but does not conform to the Action Principle.

This is why GR cannot be quantised. The representation that is quantised is the 1+15, not the 6+10. The Hamiltonian duality between spacetime and 4-momentum is quantised by the Heisenberg Uncertainty Principle into units of Planck’s constant. This quantises gravity, but of course does not quantise GR. So for a correct theory of quantum gravity, you need three principles: the Principle of Relativity, the Action Principle and the Uncertainty Principle. Nothing else. It isn’t complicated at all. But if you forget the Action Principle, you are doomed.

Next time, I will explain why the Standard Model of Particle Physics is wrong, because it conforms to the Action Principle and the Uncertainty Principle, but does not conform to the Principle of Relativity.