OK I think, before this gets into a slanging match, that part of the problem is where you are measuring the swell.
I agree that in the mid atlantic a 3 ft wave spread out over a 16 second period is a hell of a lot of water, and that same wave when it reaches the shore it will have caught up with other waves and essentially the same amount of water will be squashed and the only way to go being upwards, it will be a hell of a lot bigger!.
However, as you mentioned there are so many other factors to take into account. As you said, if the waves are hitting the beach at an angle or wrapping around a point into the bay, the energy (which is essentially all that waves are) will be dissipated and the height will decrease.
North Devon and Cornwall consistently get good waves because they have an unobstructed fetch out into the atlantic.
However, you can't predict waves from buoys alone. Waves are made by wind! If the buoy readings you are looking at are where in the middle of the 'wind area' then energy is still being given to these waves. If the wind continues almost all the way to the shore, then the waves reaching the beach will still have a lot of power.
If the buoy is positioned just as the waves leave the wind area and are still a long way offshore, then the waves have a long time to lose their power and dissipate.
I think the best way of predicting waves is to be aware of the following pieces: Pressure Areas (these give information about wind speeds and directions) Swell forecasts, buoy information, other people's predictions and webcams.
I tend to look at the web cams on a daily basis, then check the pressure charts. Over a period of time you will be able to marry up the pressure areas with the swell they produce. You also get an idea for how long a swell will last etc.
The thing is it's Nature, which can basically do whatever the hell it likes. All the signs can point to a great swell coming, and it can suddenly disappear. Likewise, you can be predicting poor surf and overnight everything can change.
You can NEVER know for sure!!