One reason for the transmission heating up "could be" that your transmission oil cooler is compromised. You said the dipstick was melted and since there was no leakage of fluid, you assumed that it was still full of transmission fluid.
I had a similiar problem with my Amel SM. The cooler developed a hole in the cooler tubes that permitted the seawater and the transmission fluid to mix. Eventually the hole in the cooler tubes was big enough, that the suction from the engine seawater pump sucked the fluid out of the transmission. Prior to the complete failure of the transmission, the seawater mixed with the transmission fluid and rusted the inner workings of the transmission. So, in my case, the transmission had to be replaced.
Have you ever noticed that the engine exhaust/water leaves an oil sheen on the water?
Pull some of the fluid out of the transmission and see if it looks right, or is milky or watery.
Here's the thing, if you have to replace the transmission, be sure to replace the transmission cooler. I'm not sure of a reasonable interval to replace them, but for $200 or so, it's good insurance. Mine failed at about 2,000 engine hours.