I realize it's been some time since this posting and surprised there hasn't been an earlier reply. Often gas and diesel powerplants require opposite turning props Left Hand verses Right Hand rotation. Possibly the origional gas engine ran a left hand prop and you are now unknowingly running the diesel engine in reverse to go forward, compensating for not having installed a Right Hand turning prop.The other factor at play is Prop Pitch. From the description it sounds like a cavation problem resulting from have too little pitch on the origional gas prop to match the prop required for diesel engine. Generally there should be sufficient pitch on your diesel engine prop to stall engine RPM at about 3600. My interest in this topic is, I am re building and converting a badly damaged keel to hull join,older Herreshoff 18 cat rig into a power boat installing a 3GM Yanmar diesel. If there is someone out there that has a diesel powered version of the 18 (Pilot,Launch,Bay Fisherman,) would you be so kind to pass on your prop diameter and pitch as stamped on the props hub. I want to compare with my calculations before ordering, With Thanks --VN