先ほどまでの憂いは影をひそめ、その仕草に優雅ささえ漂わせる.
The hardships from before hide the shadow, (and through) that gesture give off elegance.
Is this a correct way to understand it?
Is 仕草 the agent?
Is 漂わせる a causative form.
Yes 漂わせる is causative, but, no, 仕草 is not the agent. 優雅さ is the agent (causee). It's because 漂う is an intransitive verb, and when in the causative form, its agent has to be marked with を. その仕草に is just an adverbial phrase that describes where the action (漂う) happens.
Literally:
(I don't know how to translate this naturally, but I think "her gesture (even) gives off elegance" is basically correct at least as a free translation)
Compare:
おまけ: