(654d) Decentralized Multi-Agent Control of Distributed Reactor Networks

Control of complex networked systems can be achieved via hierarchical multilayered agent-based structures benefiting from their inherent properties such as modularity, adaptability, flexibility and robustness. Our recent work implemented a centralized multi-agent supervision framework for reconfiguration of interconnected reactor networks hosting multiple autocatalytic species where a global planner agent calculates the strategy for reconfiguration and dynamically updates the objectives of local control agents as the reactor network conditions change. Alternatively, this paper proposes a decentralized multi-layered agent structure where local control agents individually decide on their own objectives allowing the framework to achieve multiple local objectives concurrently at different parts of the network. On top of that layer, a global observer agent continuously monitors the system for conflicting local control strategies and deadlocks throughout the network and assigns regional mediation agents for solution of the conflicts.

Manufacturing processes producing specialty chemicals such as pharmaceuticals, high-end polymers etc. are usually produced in small volume, have high value, and require expensive equipments. An efficient way to reduce the production costs is to use the same production line for producing similar products. Scheduling back to back production of different grade products may usually result in off-spec products during transition from one product to another. Minimizing such transients is a challenging task and may require reconfiguration of the whole process.

In addition, such specialty chemicals may require different physical and chemical properties in different parts of the product. Developing an interconnected network of smaller scale production units (reactors) gives the flexibility of obtaining distributed physical and chemical properties throughout the product.

One example of such systems is the network of interconnected continuous stirred tank reactors (CSTRs). Supervising CSTR networks is a challenging problem because of their highly nonlinear behavior and multiple steady state operating conditions. Our earlier work demonstrated autocatalytic reactions in these networks to simulate population dynamics, multiple species of organisms that compete on same resources, or chemical manufacturing problems [1, 2, 3, and 4].

Using agent-based approach for supervising grade transitions in a networked manufacturing process has several advantages over traditional methods such as distributed parameter systems involving model reduction and controller synthesis techniques. Determining possible control strategies for different scenarios in a large nonlinear reactor network can be a very challenging problem if not impossible to solve. Agents are autonomous decision making modules capable of adapting their responses to dynamically changing environment conditions and other agent's behaviors. Having a pool of different types of agents embedded in a hierarchical multi-layered framework allows the emergence of control strategies for the cases with unpredicted behaviors and unexpected deviations. These emergent complex control strategies are achieved by combination of simple local actions (strategies) with predefined set of rules and priorities. Another advantage of using multi-agent systems is their modularity. A modular framework gives the flexibility to replace failing or underperforming modules without disturbing the whole process thus making the system more robust. Also modular approach increases the scalability of the framework.

The decentralized multi-layered framework focuses on the problem changing spatial distribution of autocatalytic species in a CSTR network which is equivalent to grade transition in a reactor network of a manufacturing process. Implemented framework consists of several layers of multiple agents. At the lowest level a local controller agent is assigned for each reactor in the network. The objective of each local controller agent is to maximize targeted species in the reactor by manipulating interconnection flow rates. The local controller agents communicate with other local controller agents for searching and requesting targeted species in the network. Since the structure is decentralized, a queuing algorithm is employed to each agent for managing and ordering multiple requests received from different local agents. Requests are ordered and prioritized according to the distance between the source and the destination nodes. A well known algorithm in graph theory, namely Dijkstra's shortest path algorithm, is implemented to calculate the distance between reactors. Each agent takes local control action (adjusting interconnection flow rates) according to the highest prioritized request in its ordered request list and moves to the next request when the previous one is complete. Since each agent sends and receives requests concurrently, the network moves from initial configuration to the targeted one more efficiently than the centralized approach.

On the other hand, conflicting local strategies may result in intersection of trajectories which causes deadlocks to occur. A global tracking agent is implemented to observe local control objectives and detect conflicts. In the case of deadlocks, conflicting request are removed from related local controllers request list and a mediation agent is assigned for supervising the region where the deadlock occurs. A mediation agent computes a solution to a portion of the overall problem and recommends local strategies to the agents involved in the mediation section [5].

The decentralized framework implemented has several advantages over the centralized one proposed in our earlier work. As the size of the network considered increases, planning local control strategies from a global point of view gets harder to achieve. In the decentralized approach, local controller agents plan their strategies locally making the whole system more robust and scalable. Secondly, giving the local planning duty to individual local agents allows the system to execute multiple tasks in different parts of the network which reduces the transients compared to the centralized framework.

The control framework is implemented using the agent based modeling and simulation environment RePast. The RePast toolkit is built upon the object oriented programming language Java which makes the implementation modular, scalable and flexible. The autocatalytic CSTR network simulator containing differential equations that describes autocatalytic reactions is coded in C and the visual representation of the network is prepared in RePast environment.

References:

[1] E. Tatara, I. Birol, F. Teymour, and A. Cinar, ?Agent-based Control of Autocatalytic Replicators in Networks of Reactors,? Computers & Chemical Engineering, vol. 29, pp. 807?815, 2005.

[2] E. Tatara, C. Hood, F. Teymour and A. Cinar, ?Adaptive Agent-based Control of Product Grade Transitions in Reactor?, Prepr. IFAC Intl Symposium on Advanced Control of Chemical Processes (ADCHEM 03), Gramado, Brazil, April 2-5, 2006.

[3] E. Tatara, F. Teymour and A. Cinar, ?Agent-based Control of Spatially Distributed Chemical Reactor Networks?, AIChE Annual Meeting, Cincinnati, OH, November, 2005.

[4] M. D. Tetiker, A. Artel, E. Tatara, F. Teymour, M. North, C. Hood, A. Cinar, ?Agent-based System for Reconfiguration of Distributed Chemical Reactor Network Operation?, Proc. American Control Conf., June 14-16, 2006, Minneapolis, MN.

[5] R. Mailler, V. Lesser, ?Solving Distributed Constraint Optimization Problems Using Cooperative Mediation?. Proceedings of Third International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2004), IEEE Computer Society, pp. 438-445. 2004.