compositor: fix the over operator
If a vterm is on top of a vcalc and you decrease the opacity of the
first (or rotate/resize it), the result composite image is obviously wrong
(the vcalc's buttons become black…).
This is because of some mistakes in the over operator's implementation.
This patch fixes the problem by reimplementing it.