With all due respect, I disagree. I've had huge dudes who could not hold me down. And i've gone with BJJ black belts who embodied the 'empty jacket' principle with their top game and could keep me there with virtually zero effort (I am myself 173 pounds).
I think the most important single factor in maintaining top position (although the BJJ and judo definitions change because bjj awards a score for taking the back and knee on stomach position, they are otherwise identical) is the ability to shift one's weight, which is mostly a function of movement of the hips. There exists a maximal size difference at which a 'pinning' mentality/posture is effective. Once you reach that (i'm sure we could calculate exactly where it lies), you can't hold uke down there. However when you switch to 'floating', for lack of a better term, you are much more secure because you're mobile, able to respond to the different directions you're being pushed/pulled in, to slide into different positions.
You were talking about flaws in the ne waza pedagogy earlier. Another one to add to that list, I think, is the teaching of pins as static positions. This, I believe, retards (heehee) the student's development. It makes them stiffer in ne waza, and thus more easily turned. Furthermore they have a hard time using transitional submissions because they've been taught to always randori for the win, rather than to maximize their learning time. This is stressed in tachi waza but due to a lack of knowledge, forgotten in ne waza!
While you're right in saying that staying in kami shiho wouldn't have been able to prevent what happened to you, that doesn't mean there was nothing you could have done. Now i'm just a young whippersnapper and all but I used to be much smaller than I am now and gained a lot of experience with dudes trying to do pretty much what you described.
In order to bench press you off, he's got to extend his arms. You could have shifted your hips 90, slid your knee in and tried for an armbar. You could have brought your hips forward and up, used a knee to pin a bicep in place then slid that same knee through the gap of his arm to get a reverse triangle as you stabilize a grip on his pants that keeps him from getting his hips under him (by bringing the gripped leg toward you like a cradle). You could have skull rode him (although if he was an olympian that could be hazardous for you!). You could have swung your hips in a wide arc so as he sat up you took his back. You could have shifted like you were going for an armbar but instead slid your chest along his tricep and brought your head down to his shoulder so you ended up with a 'seatbelt' grip.
Hindsight is always 20/20, but just off the top of my head I can think of quite a few options there. So I think your example was flawed.