Stochastic intervention for causal inference via reinforcement learning

Causal inference methods are widely applied in various decision-making domains such as precision medicine, optimal policy and economics. The main focus of causal inference is the treatment effect estimation of intervention strategies, such as changes in drug dosing and increases in financial aid. Existing methods are mostly restricted to the deterministic treatment and compare outcomes under different treatments. However, they are unable to address the substantial recent interests of treatment effect estimation under stochastic intervention, e.g., “how all units health status change if they adopt 50% dose reduction”. In other words, they lack the capability of addressing fine-grained treatment effect estimation to empower the decision-making applications. In this paper, we advance the causal inference research by proposing a new effective framework to estimate the treatment effect under the stochastic intervention. Particularly, we develop a stochastic intervention effect estimator (SIE) based on nonparametric influence function, with the theoretical guarantees of robustness and fast convergence rates. Additionally, we construct a customised reinforcement learning algorithm based on the random search solver which can effectively find the optimal policy to produce the greatest expected outcomes for the decision-making process. Finally, we conduct extensive empirical experiments to validate that our framework can achieve superior performance in comparison with state-of-the-art baselines. For reproducing experimental results, all the source codes and data are available at https://github.com/tridungduong16/Interpretable-Machine-Learning/tree/master/Source%20Code. Copyright © 2022 Elsevier B.V. All rights reserved.