I think we are basically on the same wavelength; but our recognition of the sickness is similar yet diagnoses differ in one key respect.
I agree re: lack of realism: my belief is that the reason they are not realistic are the same reasons you put forth - they cannot be trained realistically because the majority are not putting the effort in to develop the skill to begin to train them realistically.
I DO think kata are supposed to end up as combative simulation: they represent a progressive cycle of training that should ultimately be almost freestyle in essence - and back to the start. Wayne Muromoto's article on tea ceremony on his blog is an excellent illustration of this.
I personally feel that I learned a lot about what kata are supposed to be when I began getting much deeper into force on force tactical and combatives training.
I totally agree that due to the nature of combat sports, within the MA paradigm the latter tend to do this better. I believe this is because TMA has allowed this aspect to be almost completely co-opted by combat sports. I don't think it used to be like this, I don't think there was the division there is now.
The primary TMA shortcoming today is lack of antagonistic/competitive training. TMA may be "kata only" today but I don't even a reading of history supports that, at least for jujutsu.
NOT just classical jujutsuka doing Judo, but doing their own ryu against judo (and sumo, and other ryu....) under competitive conditions - whether just in the dojo or in a taryu jiai format.
We have enough sources that tell us that Takeda, Ueshiba, and their classical forebears in various ryuha (all the "beat the sumotori" stories....) did so - and did gekken with grappling - that it is rather surprising to not see a balancing focus on similar resistive grappling practice alongside the "internal power exercises" hysteria within the IP movement.
The grappling is what paints the eyes on the proverbial dragon, so to speak...
Since the earlier practice was mixed, with all types of grapplers then, it should work now under Judo, BJJ, wrestling, submission grappling, and any other rules or lack thereof. All those other things DO adapt to each arena.... there is no valid reason that with the variety of technique shown in just the videos linked here that Daito-ryu, or many other ryu, cannot have many of its techniques practiced in a freestyle format - from an in-house practice to even competing in open sumo to submission grappling tournaments. Many of the techniques shown are completely legal in Judo, BJJ, and sub grappling tournaments today.
I would argue that this is the missing link in TMA, the role that actually grappling has in developing body power in grappling arts. Not that just anybody grappling will develop it, but those mindful of it need to develop it in exercises and then test it in actual resistive practice to hone it.
This is in fact the way it used to be practiced when TMA weren't traditional.....kata is one wheel of the cart - combat and competition the other. Perhaps some of the mixing we see is because people have to go to Judo or whatever to get it because it is no longer practiced within much of traditional jujutsu.