Practical Control Design for the Deep Learning Age: Distillation of Deep RL-Based Controllers
Published in 2022 58th Annual Allerton Conference on Communication, Control, and Computing, 2022
This paper proposes a control-design workflow in which deep reinforcement learning produces a high-performance reference policy, then imitation learning distills that policy into a simpler non-neural controller for validation and deployment.
Why it matters
The workflow separates performance-oriented learning from the final controller representation. That makes it possible to use deep RL during design while deploying models that are smaller, more interpretable, and more amenable to verification and validation.
Key contributions
- Shows the existence of exactly equivalent soft decision trees for a class of neural controllers.
- Distills deep-Q-network policies into decision trees and Gaussian-kernel support vector machines.
- Introduces metrics for evaluating the fidelity and performance of distilled controllers.
- Studies MountainCar, CartPole, and CarRacing benchmarks.