立夏 is one of the 24 words called 二十四節気 (Solar term). It specifically refers to May 5 or May 6 (depending on the year; see the linked article). Despite its appearance, it's not a word that simply refers to the start of the summer. Some Japanese seasonal events including setsubun are based on 二十四節気.
二十四節気 was defined many years ago based on the climate of a certain region of China, and the terms generally do not match how Japanese people usually sense seasons today. Most Japanese people think May 6 is still spring even though 立夏 is technically the start of the summer. The start of the spring (立春) according to this system is February 4, but this is usually the coldest time of a year in Japan. So 立春, 立夏, etc. are not very useful concepts in our daily life.
Solar terms originated in China, then spread to Korea, Vietnam, and Japan, countries in the East Asian cultural sphere. Although each term was named based on the seasonal changes of climate in North China Plain, peoples living in the different climates still use it with no changes.
However, haiku is one of the fields where those 24 terms and the sense of seasons based on them have traditionally been considered important. If you know the kigo system, it still respects this 二十四節気 system as explained here, and that's why 立夏 is a common word in traditional haiku. 夏に入る is an wago-paraphrased version of 立夏 in haiku, and it typically refers to May 5-6, too.
In daily life, people simply say 夏が始まる, and that's normally no sooner than June 1.