Actually, turns out the math was right with a cosmological constant: that's how the accelerating expansion of our universe is explained. Einstein actually blundered twice: first by putting in the constant for the wrong reason (because he wanted a static universe), but then taking it out again when it turned out that reason wasn't the case (when the expansion of the universe was discovered).
If you just look at how to derive the Einstein Field Equation of General Relativity from first principles, the constant should be there; it isn't an add-on to General Relativity at all, it's part of it. It's just that there's no way to know from those first principles what its value is. That we had to figure out from observations.
Thanks for your point about the constant being actually required. I do not understand the math, but is this similar on how integrals always have a constant as a free parameter that need to be determined by other means?
I assume that Einstein originally set the constant to exactly balance the expansion, but later set it to zero. In bot cases you are picking arbitrar values but the actual value need to determined by empirical observeations.
Not really, no. It's a consequence of the assumptions made when the Einstein Field Equation (EFE) is derived from a Lagrangian using the principle of least action.
The assumptions are that the Lagrangian should be a Lorentz scalar (which is required of any Lagrangian) and that it should include no more than second derivatives of the metric. The Ricci scalar R meets this requirement and is the Lagrangian that was originally used by David Hilbert to derive the EFE (without a cosmological constant). But a simple constant (the cosmological constant) also meets the requirement, and therefore should be included in the Lagrangian; including it leads to the cosmological constant term in the EFE.
Einstein didn't include the constant at all in his original equation, published in 1915.
In (IIRC) 1917, he realized that his original equation did not allow a static solution for the universe as a whole. He also realized that including the cosmological constant term in his equation would be mathematically valid, and that if he picked just the right value for the constant, he could obtain a static solution for the universe. At that time, it was generally believed that the universe was static on large scales.
Then, later, when it was discovered that the universe is expanding, Einstein dropped the cosmological constant term. He later called including that term in 1917 "the greatest blunder of my life", because if he had just gone with his original field equation, without the constant, he could have predicted the expansion of the universe more than a decade before it was discovered.
> In bot cases you are picking arbitrar values
When the expansion of the universe was discovered, yes, it was already recognized that it is valid to include the constant in the Einstein Field Equation, so it couldn't just be un-included. Its value was just assumed to be zero since that was consistent with all observations that were known then.
In Einstein's original 1915 field equation, however, the constant wasn't "set to zero". It wasn't included at all; nobody even realized at that time that it was valid to include it.
I wouldn't go that far: Rather, adding it doesn't violate any of the heuristics used to come up with the field equations or action. So to avoid bias, one should keep it around. However, in the absence of observational evidence to constrain its value, it's also justified to start any investigation with its value assumed 0...
Yes, that's what I was trying to say. I didn't mean that including it mathematically in the equations necessarily requires one to adopt a non-zero value for it; you are quite correct that one shouldn't do that unless one has observational evidence to back it up (and cosmologists in fact didn't adopt a non-zero value until observational evidence required it).
