put perspective handles on the outside instead (if you invert the frame, they'll switch place with the scale handles, oh well)
This is the proof of concept code I wrote before gsoc, with very naive behaviour and simple interface.