(189e) Infrastructure Maintenance and Inspection Scheduling with a Time-Variant Transition Probability Under a State Observation Uncertainty

For the safe and cost-effective operation of an infrastructure system, a scientific model based management is required. The overall objective of this management problem is to minimize the life-cycle operation cost, to take a physical property of the system partition into account, and to reflect the real-time dynamics with accompanying optimal action implementation. This work suggests optimization and data processing scheme for such problems.

A Markov Decision Process (MDP) framework is incorporated to solve the sequential optimization problem which evaluates the minimum operation cost. To address the scheduling problem, Markovian transition probability between each state, the action and cost effect should be defined. In the infrastructure system, however, the exact measurement of the state is nearly impossible due to the difficulty of the thorough understanding of a large-scale system, and the intrinsic uncertainty of the non-destructive inspection method. Partially Observable MDP (POMDP) scheme on which the additional probability of the state observation is applied is then suggested in this work to overcome the subotimality caused by the uncertainty.

The exact solution of large-scale POMDP takes an exponential computation time. Thus, in the actual implementation, a point-based solver called gapMin with the following steps is used: (1) Transforms the objective value function into the piecewise-linear function (2) improves the lower bound with the consecutive search of the belief space in a breadthwise manner (3) improves the upper bound with the LP interpolation of belief-value convex hull mapping [1], [2].

Water distribution system is analyzed as the illustrative example. There are five states which indicates the physical condition of the pipe, and the combination of maintenance actions (do-nothing, rehabilitation, and replacement) with inspection actions (do-nothing, inspection) [3]. A structural deterioration rate over a pipe age can be modeled with a two-parameter Weibull probability distribution [4].

Discretization of the continuous probability model yields the time-variant Markovian transition probability set. Unlike the do-nothing and rehabilitation actions, the replacement action initializes the pipe age. Defining the (state, age) pair makes it possible to reflect the action-dependent age dynamics, and the state augmentation integrates the time-variant transition probabilities into a single time-invariant matrix.

The simulation result yields the optimal action sequence which makes the minimum life-cycle operation cost for the infinite horizon POMDP. The next decision step optimization can be employed by the updated data result from the calculated former action. Results show that the suggested POMDP framework for the infrastructure management successfully fulfilled the overall objective.

References

[1] Poupart, Pascal, Kee-Eung Kim, and Dongho Kim. "Closing the Gap: Improved Bounds on Optimal POMDP Solutions." ICAPS. 2011.

[2] Shani, Guy, Joelle Pineau, and Robert Kaplow. "A survey of point-based POMDP solvers." Autonomous Agents and Multi-Agent Systems 27.1 (2013): 1-51.

[3] Papakonstantinou, K. G., and M. Shinozuka. "Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory." Reliability Engineering & System Safety 130 (2014): 202-213.

[4] Kleiner, Yehuda. "Scheduling inspection and renewal of large infrastructure assets." Journal of infrastructure systems 7.4 (2001): 136-143.