(231b) Markov Decision Process Based Dynamic Programming for Colloidal Self-Assembly Process Optimal Control

Markov
Decision Process based Dynamic Programming for Colloidal Self-Assembly Process
Optimal Control

Xun Tang¹, Yuguang Yang², Michael A.
Bevan² and Martha A. Grover¹

1. School of
Chemical & Biomolecular Engineering, Georgia Institute of Technology, 311 Ferst Dr. NW, Atlanta, GA. 30332-0100.

2. The Department of Chemical and Biomolecular
Engineering, Johns Hopkins University, 221 Maryland Hall 3400 North Charles
Street, Baltimore, MD. 21218.

Self-assembly broadly refers to the process of a disordered
system converging to a well-arranged state without human intervention [1].
Using self-assembly on the nano-/micrometer scale to
produce defect-free crystals for the use of metamaterials,
particularly photonic crystals, has received a significant amount of interest.
However, due to the formation of polycrystalline assemblies, it is challenging
to achieve such regularly arranged crystals without defects from self-assembly
processes.

During the past decade, various approaches have been reported on
producing photonic crystals. Techniques include patterned template directed
colloidal crystallization, anisotropy-driven nanorod
assembly, colloidal deposition, magnetic field and electric field mediated self-assembly,
laser or optoelectronic and optical tweezers directed assembly, etc. have been
widely studied for defect-free crystal manufacturing [2, 3, 4].

In spite of all these different approaches, the idea of using
control theories to manufacture defect-free colloidal crystals has not received
much attention yet due to the difficulty of real-time sensing. Control
algorithms with system state as feedback can significantly enhance the
performance of a dynamic system, especially a stochastic system like colloidal
self-assemblies on the micro/nano scale. Feedback
control is a process of monitoring a system by taking the present or past
output information as a consideration in sequential action to achieve the
ultimate desired output from the system. Considering its strength in dealing
with stochasticity and nonlinearity of the system
being studied, feedback control has been widely used in both academic studies
and real-world applications. Unfortunately, colloidal self-assembly processes
with feedback control have not been studied much yet, with a few exceptions [5,
6].

In our study, we propose
to apply a Markov decision process (MDP) based dynamic programming optimal
control algorithm to manipulate a SiO₂ colloidal self-assembly
process for two-dimensional defect-free crystals [6]. The movement of the
particles is manipulated by the particle-particle and particle-wall
interactions, which are controlled by changing the magnitude of the voltage on
the electrodes. The
movement of the particles is manipulated by the dipole-dipole and dipole-field
interactions, which are controlled by changing the magnitude of the voltage on
the electrodes surrounding the system. A two-dimensional Langevin equation is developed
to simulate the dynamics of this SiO₂ colloidal system. Markov
transition matrices associated with 6 discrete, constant control variables
(voltages) are developed based on simulation data from the low dimensional
Langevin equation. With the predictions on the system state evolution from the
Markov chain model via Markov chain Monte Carlo simulation, an infinite-horizon
MDP optimization problem is formulated and the optimal control policy is solved
by dynamic programming with policy iteration. Our MDP-based dynamic programming
control policy is able to accelerate the SiO₂ colloidal self-assembly
process for two-dimensional defect-free crystals.

References

[1] Whitesides, G.
M.; Grzybowski, B., Self-assembly at all scales.
Science 2002, 295, (5564), 2418-2421.

[2] Velev, O. D.;
Gupta, S., Materials fabricated by micro- and nanoparticle assembly?the challenging
path from science to engineering. Adv. Mater. 2009, 21, (19), 1897-1905.

[3] Grzelczak, M.;
Vermant, J.; Furst, E. M.;
Liz-Marzán, L. M., Directed self-assembly of nanoparticles.
ACS Nano 2010, 4, (7), 3591-3605.

[4] Arpin, K. A.; Mihi, A.; Johnson, H. T.; Baca, A. J.; Rogers, J. A.;
Lewis, J. A.; Braun, P. V., Multidimensional architectures for functional
optical devices. Adv. Mater. 2010, 22, (10), 1084-1101.

[5] Juárez, J. J.;
Mathai, P. P.; Liddle, J.
A.; Bevan, M. A., Multiple electrokinetic actuators for
feedback control of colloidal crystal size. Lab on a Chip 2012, 12, (20),
4063-4070.

[6] Juárez, J. J.;
Bevan, M. A., Feedback controlled colloidal self-assembly, Adv. Funct. Mater. 2012, 22, (18),
3833-3839.