Why did the .406 hitter disappear after 1941, and how luck has become a bigger factor in batting average in modern baseball?

Since Ted Williams’ .406 batting average in 1941, the .406 batting average has disappeared. This isn’t because hitting skills have deteriorated, but because the overall quality of the game has improved, reducing skill differences and allowing luck to play a bigger role in batting average.

 

In 1941, Ted Williams of the Boston Red Sox finished the season with a .406 batting average. Since then, no one has ever posted an OPS above 0.4. Even though Williams hit .406, a .406 batting average wasn’t particularly unusual at the time. Between 1901 and 1930, a number of hitters posted an ERA over 0.4, but starting in the 1930s, the record faded away and hasn’t been seen again since Williams’ 0.4 ERA in 1941. In baseball, a .400 batting average is an honorable and blessed achievement, and the disappearance of the .400 batting average is a clear indication of this, so much time and effort has gone into analyzing the reasons for its disappearance from baseball history.
The most popular explanations for the disappearance of the .400 hitter boil down to two things. The first is that hitting skills have absolutely regressed, and the second is that pitching, defense, and team management have outpaced hitters’ abilities. I think both of these explanations are wrong. In his book Full House, Stephen Jay Gould argues that the reason for the disappearance of the .400 average is not a decline in hitting skills, but an improvement in the quality of baseball as a whole. I completely agree with him. I would go further and argue that the overall improvement in the game has led to the demise of the .400 hitter, as batting averages have become more influenced by luck than skill.
Some people believe that the hitting skills of modern baseball players have regressed from the past. This is the opinion of legendary players from the past, such as Ted Williams and Stan Musial. Williams claims that nothing has changed in the game of baseball except hitting, and that modern hitters are less intelligent than in the past. Musial also criticizes modern hitters for their lack of ability to analyze defenses. However, this explanation doesn’t really fit the bill as the reason for the disappearance of the on-base percentage. Rather, today’s baseball players are drawn from a larger pool, players of different ethnicities receive organized training, and they train continuously, even outside of the season. The larger population increases the likelihood of identifying talented players, and the systematic training has improved their skills, so the argument that batting has regressed is not convincing.
The question of whether pitching and defense have improved is answered somewhat differently. Pitchers have developed new pitches, such as sliders, and the rise of relief pitchers has made pitching staffs stronger. Defense has also improved significantly, thanks to advances in equipment like gloves. Despite the undeniable improvements in pitching and defense, the average batting average remained constant throughout the 20th century at around 2.6 pounds. This means that hitting, pitching, and fielding developed together in balance. Therefore, improvements in pitching and defense do not directly explain the disappearance of the .400 hitter.
Now, let’s see if the overall improvement in the game of baseball itself could be the reason for the disappearance of the .400 hitter. If we plot the batting averages of hitters as a normal distribution, the mean is roughly constant, but the standard deviation is decreasing over time. This makes a 40% batting average an extremely rare phenomenon. In fact, from 1880 to 1980, the standard deviation of players’ average annual batting averages decreased regularly. This shows that the overall quality of the game of baseball has improved, reducing the differences between players’ skills. While the best hitters of the past were able to hit .400 because they faced relatively less developed pitchers and defenses, modern baseball has made it harder to reach .400 because they have to face higher quality pitchers and defenses.
In addition, luck has become a bigger factor in batting average than skill, which is one of the reasons for the disappearance of the .400 hitter. While the variance in skill among modern players is shrinking, the influence of luck remains. As the distribution of skill boxes converges to the mean, the lucky box has become more influential on batting average. In other words, no matter how skilled a modern player is, it’s hard to hit .400 without some luck.
In conclusion, the disappearance of the .400 hitter is not due to a decline in hitting skills, but to an overall improvement in the quality of the game of baseball. As a result of the decreased variance in skill and the greater influence of luck, the modern game has seen the demise of the .400 hitter.