Societies are increasingly dependent on Cyber Physical Systems (CPSs), which are exposed to natural and human-made attacks. Attacks on CPSs can result in security breaches and behaviors that may impose harm on their environments. Understanding attack mechanisms is crucial to preventing losses or damage to people, assets or information. We develop a computational environment that allows to implement attacker agents under a stochastic optimization framework, allowing to use different techniques to model attacking behavior (i.e., policies), including approximate dynamic programming, reinforcement learning, or stochastic programming, as well as arbitrary policies (e.g., rules of thumb). We rely on the ADVISE formalism to represent attack paths on CPSs, leveraging their Markov Decision Process structure to build an environment that allows to test attacker and defender policies. The proposed environment is tested by simulating attacks on a SCADA system previously addressed in the literature, demonstrating satisfactory convergence for a Q-learning algorithm, which allows to identify the attack steps that most frequently lead to successful attacks. The proposed approach allows flexibility in modeling attackers, and allows to conceive models with interacting attacker and defender agents, which is left as the main goal of future work.