(508b) Hierarchical Bayesian Approach to Decision-Oriented Experiment Design and Evaluation for Selection Among R&D Alternatives

Generally, evaluation of R&D alternatives does not provide a clear cut choice that is best under all possible scenarios. In such cases, the decision maker may either  make the selection based on a user defined utility function, or conduct more experiments to further reduce uncertainty in the most promising alternatives, so that a better decision can be reached.

The latter option has more intrinsic appeal, but conducting experiments to reduce uncertainty can be both time consuming and expensive. Hence, it is important to have a systematic approach and accompanying design of experiment (DOE) strategies that are geared towards a particular selection criterion adopted. In this paper, we will present a hierarchical Bayesian approach to address this problem.

The hierarchical Bayesian approach consists of three major components. The first component quantifies the uncertainties as probability distributions.  The second component provides information about the contribution of various uncertainty factors to the downside risk. The third component addresses the allocation of resources and design of experiments which have the aim of making the best selection among the alternatives under a given criterion, rather than the traditional designs that are geared towards minimizing some measure of the parameter uncertainty. The overall framework is shown in the figure 1. All the three components are embedded into a Bayesian framework, which makes it possible to quantify risk in terms of probability density functions.  

Fig. 1 Hierarchical Bayesian Framework for improved R&D decisions

The first component is the quantification of uncertainty in various R&D alternatives. This is done by using all sources of information, such as pre-existing data and knowledge of experts. For this component we present tools such as ?Expert Opinion Elicitation' and ?Expert Opinion Aggregation' from the field of Risk and Decision Analysis along with statistical tools like copula's to accurately quantify the uncertainty.

The second component consists of tools to estimate the key uncertainty factors contributing to the overall variances. For this, we can use Global Sensitivity Analysis (GSA), while keeping in mind the discrimination between the epistemic and aleatory uncertainties. However, the traditional GSA measures the contribution of each factor to the overall uncertainty, while the decision maker is mainly concerned about their respective contributions to the downside risk of each alternative. Hence, we propose a new form of GSA coined as ?c-GSA?, which accounts for the second lower partial moment contribution of each factor.

Once the key uncertainty contributors are found, the decision maker allocates the limited resources to reducing various uncertainties appropriately. As a next step, it is important to channel the allocated resources to design of experiments, directly geared towards resolution of the decision maker's dilemma, which in this case is the selection of a most appropriate alternative from the expected profit and risk viewpoint.  In order to achieve this goal, we suggest using Bayesian decision oriented design of experiments. This comprises the third component of the Bayesian approach.

After the DOEs are performed, experiments can be conducted. The new experimental observations can be integrated with the prior estimates to obtain the posterior estimates. Using these posterior estimates one can again evaluate the various R&D alternatives. If an acceptable alternative is found the decision maker's job is complete but if not, the decision maker must return to perform the second and the third steps as mentioned above, starting with estimation of the c-GSA index.

In order to demonstrate the utility of this approach, we will use a real biofuel related case study (Hemi-cellulose pre-extraction to produce ethanol in a thermo mechanical pulp producing mill).