There is simple mistake in this argument: it wrongly treats life expectancy at birth and average age of death during an epidemic situation as interchangeable terms. They are not.
I was about to write a more complete explanation for this but then I stumbled across a perfectly fitting paper in the reputable journal PNAS: Goldstein, Joshua R. and Lee, Ronald D. 2020. Demographic perspectives on the mortality of COVID-19 and other epidemics. Proceedings of the National Academy of Sciences
https://www.pnas.org/content/117/36/22035
I think this is highly worth reading for being prepared for statements as the one in the question or "people are just dying with Covid, not from Covid". The authors write:
In the context of epidemic mortality, life expectancy at birth is a misleading indicator, because it implicitly assumes the epidemic is experienced each year over and over again as a person gets older. [...] what we really want to know is how a one-time epidemic affects the remaining life expectancy of the actual population.
We calculate [...] that the average person dying of COVID-19 had 11.7 y of remaining life expectancy [...]
The sentence continues and takes this into a population-level context (in the US):
[...] so if the epidemic kills an additional 1 million people, it will result in a loss of 11.7 million y of remaining life expectancy. This represents a loss of less than 1/1,000th of the population’s remaining years to live.
So actually there is a considerable loss of life expectancy among the people who died from/with Covid19. However, it will probably (and hopefully) not heavily affect the current overall population's life expectancy.