There’s a kind of half truth to that, in that a trait already developed is unlikely to simply disappear. Even if it becomes vestigial, it will probably stick around until something forces it out.

Thus we get whale and snake hips, ecidna eggs, human ear muscles, and so on. All can tell us of the conditions in the past, and it would usually be more difficult to remove them entirely as opposed to simply not getting very big.