Poster
Belief-State Query Policies for User-Aligned Planning under Partial Observability
Daniel Bramblett · Siddharth Srivastava
West Ballroom A-D #6505
Planning in real-world settings often entails addressing partial observability while aligning with users' requirements. We present a novel framework for expressing users' constraints and preferences about agent behavior in a partially observable setting using parameterized belief-state query (BSQ) constraints in the setting of goal-oriented partially observable Markov decision processes (gPOMDPs). We present the first formal analysis of such constraints and prove that while the expected cost of a BSQ constraint is not a convex function w.r.t its parameters, it is piecewise constant and yields an implicit discrete parameter search space that is finite for finite horizons. This theoretical result leads to novel algorithms that optimize gPOMDP agent behavior with guaranteed user alignment. Theoretical analysis proves that our algorithms converge to the optimal user-aligned behavior in the limit. Empirical results show that BSQ constraints provide a computationally feasible approach for user-aligned planning in partially observable settings.
Live content is unavailable. Log in and register to view live content