The answer is: Latin. Scholars of Lowth’s period were in thrall to the idea that Latin and Ancient Greek were the quintessence of language. England was taking its place as a world power starting in the 17th century, and English was being spoken by ever more people and used in a widening range of literary genres. This spawned a crop of grammarians dedicated to sprucing the language up for its new prominence, and the assumption was that a real and important language should be as much like Latin as possible. And in Latin, as it happens, one did not end sentences with a preposition. “To whom are you speaking?” was how one put it in Latin; to phrase it as “Who are you speaking to?” would have sounded like Martian.
The problem, though, is that English is not Latin, just as it is not Arabic, Swahili or Thai. As natural as it seemed to certain overeducated souls 300 years ago that English would be best off dressing up as Latin, this was ultimately a local and even parochial fetish. It was held by people with no way of knowing the vast and diverse array of languages in the world, among which Latin in no way stands out as the best, most interesting or most elegant. The sheer awkwardness of the idea that English should not end sentences with prepositions is captured in the fact Lowth himself wrote, when arguing against it, “This is an idiom which our language is strongly inclined to,” apparently not catching the irony.
Why did English start ending sentences with prepositions? Another short answer: the Vikings. They started invading Britain in the eighth century and stayed on for good in massive numbers. Their language, Old Norse, seeped into English over the succeeding generations. Old Norse “strands” prepositions, and thus English started to do the same. Old Norse today has evolved into languages like Swedish, in which “the person I arrived with” is “personen jag kom med,” with “med” meaning “with.” Crucially, no one has a problem with this in Scandinavia.
As ordinary as it may seem, then, the idea that ending a sentence with a preposition is slumming makes no sense and never has. Clarity is not at issue: “The person I arrived with” is in no way confusing compared with “the person with whom I arrived.” One may prefer the latter aesthetically. But that preference translates no more gracefully into a rule than stipulating that cadet blue is better than chartreuse, or that it is better that credits appear before a film as in the old days than after it. (Although it’s a nice touch in last year’s “The Holdovers” that the credits play up front as they would have in the 1970s that the film is set in.)
It is alternately bemusing and frustrating that the story is similar with so many of the other “rules” of English that get around. The idea that you shouldn’t split an infinitive is also based on Latin. But in Latin the infinitive is a single word; you literally can’t split it. In English it is the verb plus the word to which, at times, one might like to separate from the verb: To boldly go where no man has gone before.