What is it, exactly, that you’re supposed to do when the predictions of your best scientific theories don’t match what you observe? The first step is to reproduce your results independently, ensuring that you haven’t made an error. The second step is to find whether this mismatch occurs for a wide variety of conditions, quantifying it in an attempt to learn exactly what it means. And the third step — if you’re bold enough — is to try and find a theoretical explanation that brings things back into line.

In general, there are only two theoretical explanations that are worth considering: either you’ve got the rules wrong, and they need to be modified from what you thought they were before these critical measurements, or you’ve got the ingredients wrong, and something else is at play above and beyond what you’d considered earlier. Yet, when it comes to the problem of gravitational effects based on the matter we see not matching our predictions, scientists almost always invoke dark matter, and rarely even consider altering the law of gravity: General Relativity. It seems unfair on the surface, but there’s a very compelling reason why professionals overwhelmingly do this. There’s a reason scientists are so accepting of dark matter, and it’s time the rest of us knew exactly why.

If we go all the way back to the 1800s, we can easily find two examples of an older version of this exact problem. Within our Solar System, Newton’s laws of gravity were known to be incredibly successful. They explained, without any error greater than the precision of our measurements, the orbits of every celestial body. From the Earth/Moon system to the orbits of the planets, asteroids, and comets around the Sun to the moons of other planets, Newton’s equations predicted the positions and velocities of each of these objects correctly.

But in the mid-19th century, two problems began to emerge. The first was Uranus. Our planets had all been around and accurately tracked for a very long time, excepting Uranus, which was first discovered in just 1781. Initially, Uranus moved at a slightly greater speed than Newton’s (and Kepler’s) laws predicted, but from the early 1800s through the 1820s, that phenomenon went away, as the planet moved at the correct speed. Perhaps those earlier measurements were in error. It was only in the 1830s and beyond that scientists became alarmed, as Uranus began traveling at the wrong speed again: this time, too slowly.

Independently, two scientists — Urbain Le Verrier (in France) and John Couch Adams (in England) — had the same idea: perhaps there was an additional planet out there beyond Uranus, and perhaps its gravitational influence causes these anomalous speeds. In particular:

  1. when the slower-moving outer planet is ahead of Uranus, it pulls Uranus forward in its orbit, causing it to accelerate,
  2. when Uranus begins to overtake the outer world, it gets accelerated outward (along the line-of-sight), which cannot be observed,
  3. and once Uranus is past the outer planet, the gravitational tug pulls it backward, causing it to decelerate.

Le Verrier sent the correct prediction to the Berlin observatory in 1846, where Neptune was discovered the very night the letter arrived. In this instance, “dark matter” was successful.

At the same time, Mercury’s orbit didn’t quite match Newton’s predictions either, with many astronomers conducting searches for an inner planet, Vulcan, thought to be responsible. But Vulcan turned out not to exist! Instead, Einstein’s formulation of General Relativity, a new theory of gravity to supersede Newton published in 1915, pointed the way forward. This time, modifying the law of gravity was the correct solution.

So why, then, are we so certain that modifying the law of gravity is an inferior approach to hypothesizing a new form of mass in the Universe? It seems like a prejudicial choice on the surface, as in the face of our cosmic ignorance, we should be open to all possibilities equally.

It’s true, in a sense: if there were only one problem or puzzle to consider, both of these options would be equally reasonable as potential solutions. If you consider a system like an individual galaxy, and you measure the matter that’s present — stars, gas, dust, plasma, etc. — you’ll arrive at a prediction for how the various objects within that galaxy should orbit around its center.

Again, we find a mismatch between what theory predicts and what we actually observe. The farther away we move from the galactic center, the slower the rotational speeds ought to be. But when we measure what we actually observe, we find that the rotational speeds don’t obey that rule, and are too high at the edge. This is an observational fact that’s true of spiral galaxies in general (and many non-spirals as well), and is often used as evidence for dark matter.

On its own, however, this isn’t particularly good evidence for dark matter. The reason is this: it’s equally plausible, for this type of system, that

  • there’s a missing ingredient to the Universe responsible for this additional gravitational influence, and that it doesn’t interact with light or (normal) matter, which explains why it’s invisible,
  • or there are no missing ingredients to the Universe, and instead the law of gravity, which has been so well-tested on laboratory, terrestrial, and Solar System scales, might break down on even larger cosmic scales.

If this were the only evidence we possessed, it would be appallingly flimsy. Galaxies have different masses, rotational speeds, formation histories, amounts of star-formation, etc. Either one of these options offers a fine conceptual framework for making sense of what’s happening, with each posing unique quantitative challenges for this particular problem.

The thing we have to do, if we want to be responsible scientists, is to examine the implications and consequences of these potential solutions for the rest of the Universe.

We can devise a modification of gravity, if we’re clever enough, that behaves as Einstein’s laws of gravity on Solar System-sized scales and below, but where an additional behavior appears on larger scales to explain what we see for galaxies. That modification, then, needs to be applied to the remainder of the Universe, and has to explain the dynamics of galaxy clusters, the cosmic web that forms, and all the phenomena that appear on larger scales.

Similarly, we can hypothesize adding an additional ingredient — some form of dark matter that doesn’t interact very much (or at all) with light, with normal matter, and with itself — and explain the dynamics of galaxies that way. This additional ingredient would be too diffuse to affect Solar System-sized scales and below, but could affect the larger scales significantly. Again, we’d have to apply that to the remainder of the Universe, and look for the cosmic implications.

This has been, traditionally (for nearly the past 40 years), where attempted modifications to gravity fall apart, but where dark matter truly shines in its successes.

The simplest modification you could make to the law of gravity — MOND, for MOdified Newtonian Dynamics — enables you to correctly predict the rotation curves of a wide variety of galaxies, all with the same universal modification to gravitation. But when you apply that modification to larger cosmic scales, the successes cease. The speeds you predict for individual galaxies moving around in a galaxy cluster are all wrong; an additional modification is required to get those right. The predictions for the structure in the cosmic web is way off, and the spectrum of fluctuations in the cosmic microwave background has the wrong number of peaks-and-valleys entirely.

While that doesn’t mean that a more sophisticated modification couldn’t work (and in fact, many have been proposed), this idea that one modification could explain a whole host of problems doesn’t appear to work out that way. For modifications to gravity, the simplest, most straightfowards, and indeed most compelling way to go about it doesn’t get you very far in the grand scheme of the Universe.

But for dark matter, the complete opposite is true. By adding one ingredient to the Universe — a new form of matter that gravitates, but doesn’t have interactions through any of the other fundamental forces with either itself, photons, neutrinos, or normal (atom-based) matter — we’d arrive at an entirely new picture for how structure formed in the Universe.

In the earliest stages of the Universe, matter would attempt to collapse as the overdense regions would gravitationally draw additional mass in, but radiation would push back against that growth. Whereas the normal matter would interact with that radiation, being “bounced” back out when the density got too great, the dark matter would be insensitive to that effect. As a result, you would have two distinct types of behavior, superimposed atop one another:

  • the behavior of the normal matter, which responded to gravity, radiation pressure, interactions with photons, as well as particle-particle interactions,
  • and the behavior of dark matter, which responded to gravity and the effects of the changing environment around them, without any other interactions at all.

This natural laboratory — of the early Universe — is actually a phenomenal testing ground for dark matter. The reason is simple: when the gravitational imperfections in the Universe are small, there is a negligible amount of chaos. If we start with a small set of gravitational imperfections and a few simple ingredients (like normal matter, dark matter, neutrinos, and photons), we can calculate precisely how these imperfections will evolve as long as these imperfections are small compared to the overall matter density.

When are the imperfections small? In two places:

  • at early cosmic times, before they’ve grown too significantly,
  • and on large cosmic scales, which take much longer to experience large amounts of gravitational growth.

This is why it’s so vital to look at both the large-scale structure of the Universe, where the predictions of dark matter can be extraordinarily well-calculated, and at the fluctuations imprinted in the cosmic microwave background, whose features are a relic of the Universe from just 380,000 years after the Big Bang. With modern data sets from enormous large-scale structure surveys like SDSS and all-sky cosmic microwave background surveys like those conducted by WMAP and Planck, dark matter’s exquisite agreement between theory and observations are slam dunks for cosmology.

If the observational successes weren’t so profound and unambiguous, dark matter would never have become the prevailing, accepted theory that it is today. A scientific consensus would not have arisen unless the direct evidence in favor of dark matter’s existence were overwhelming, and it is. While we still lack — and fervently search for — the critical direct-detection evidence we hope to find in terms of the particle theorized to be responsible for dark matter, the indirect evidence is so strong as to be decisive.

Astrophysically, dark matter (or something hitherto indistinguishable from it) explains an enormous suite of observations, including on the largest cosmic scales and at the earliest cosmic times: where there is the least theoretical uncertainty of all. At later times and on smaller scales, there are a lot of complications that arise, making simulations a necessity but also inherently fraught with uncertainties. When we look in the place where uncertainties are smallest, we also find the evidence that’s the strongest. In science, we often say that extraordinary claims require extraordinary evidence. When that evidence is present, however, you ignore it at your own peril.