Causal Decision Theory (CDT) is a popular choice among practical decision theorists. While its successes and failings have been extensively studied, a less investigated topic is how CDT's choices hinge on the theory of causation used. The most common interpretation, temporal CDT, understands causation as a description of physical processes ordered in time. Another emerging view comes from the graphical framework of Structural Causal Models (SCM), which sees causation in terms of constraints on sources of variation in a system. We present an adversarial scheme where a CDT agent facing a Bandit problem can be tricked into sub-optimal choices, if it follows temporal CDT. We then propose an axiom to ground the orientation of arrows in the causal graph of a decision problem. In doing so, we resolve an ambiguity in the theory of SCMs, and underscore the importance of agent-perspectives, which have been largely ignored in the causal inference literature. We also demonstrate how this structural CDT avoids our adversarial trap, and outperforms temporal CDT in a series of canonical decision problems.