RDDL (pronounced as “riddle”) stands for the Relational Dynamic Influence Diagram Language.
It is the domain modeling language utilized in the International Conference on Automated Planning and Scheduling (ICAPS) in the years 2011, 2014, 2018,
and most recently in 2023 for the Probabilistic Planning and Reinforcement Learning track of the International Planning Competitions.
RDDL was designed to efficiently represent real-world stochastic planning problems, specifically Markov Decision Processes (MDPs), with a focus on factored MDPs characterized by highly structured transition and reward functions.
This tutorial aims to provide basic understanding of RDDL, including recent language enhancements and features, through a practical example.
We will introduce a problem based on a real-world scenario and incrementally build up its representation in RDDL, starting from the raw mathematical equations up to a full RDDL description.
Furthermore, we will introduce “pyRDDLGym,” a new Python framework for the generation of Gym environments from RDDL descriptions.
This facilitates interaction with Reinforcement Learning (RL) agents via the standard Gym interface, as well as enables planning agents to work with the model.
In a series of exercises, we will explore the capabilities of pyRDDLGym, which includes generation of the Dynamic Bayesian Networks (DBN) and eXtended Algebraic Decision Diagrams (XADD)-based conditional probability functions,
as well as offering both generic and custom visualization options. We will also generate a functional environment for the example problem.
To close the loop from a mathematical representation to a fully operational policy, we will utilize the built-in model-based anytime backpropagation planner,
known as “JaxPlan.” This will enable us to obtain solutions for the example problem, effectively closing the gap between theoretical description and a practical working policy.
