the man who comes out of the greenhouse with his trousers open looked extremely toppy to me.

I figured that was the role he was playing for Gwen, not for Ianto. I don't think we can ignore that there was a third person present, and that the third person made it into a 'work' environment. And the professional dynamics of Jack/Ianto are naturally going to be different than the private.