The Bland and Altman limits of agreement (LoA) method is almost universally used to compare two measurement methods, when the outcome is continuous. The method relies on strong statistical assumptions, which are unlikely to hold in practice. Given the popularity of this simple method, it is timely to explain when it can be safely used and when it should not be used.
Based on a small sample of simulated data where the truth is known, we illustrate what happens when the LoA method is used and the underlying assumptions are violated.
When each measurement method has a different precision or the systematic difference between the two methods is not constant, the LoA method should not be used. For this setting, we refer to an alternative unbiased statistical method, which comes at the cost of having to gather repeated measurements by at least one of the two measurement methods.
The LoA method is valid under very restrictive conditions and when these conditions do not hold the only way out is to gather repeated measurements by at least one of the two measurement methods and use an alternative existing statistical methodology.