Bayesian Regression Methods for Ordered Categorical Data
Hugh A. Chipman

This thesis explores regression models for ordered categorical data,
with special attention to data arising from industrial experiments.
There has been considerable interest in models to deal with this kind of
response, and significant advances have been made by Nair (1986) and
Hamada and Wu (1990).  These papers provide an in-depth look at the
existing methods for analyzing ordinal data, including accumulation
analysis and scoring, clarifying their strengths and weaknesses.
Despite the advances made, both papers still rely on assigning a numeric
score to each response category, and analyzing the scores as though they
were normal.  This heuristic approach is an approximation, and in some
cases can be less than optimal.  One alternative suggested by Hamada and
Wu is to fit models to the data via maximum likelihood instead.  This is
a more sound approach, and the regression models of McCullagh (1980) for
ordinal data would seem appropriate, since they are an extension of
binary regression.

In the industrial setting, where models for experimental data typically
have as many terms as observations, the MLE's quite often do not exist.
This renders the maximum likelihood approach to fitting McCullagh models
inappropriate when the models are to be used for optimization.  An
exciting alternative procedure is to assume prior knowledge about the
effects of the variables, and fit the model via Bayesian techniques.
This is reasonable, since the estimated effects are known to be finite
on the transformed scale.  The fitting of Bayesian McCullagh models is
itself a challenge, since the posterior distributions are not available
in closed form.  The Gibbs sampler, a recent computational technique, is
used to obtain the posterior distributions of the parameters and any
functions of them.  A second important step is considered in addition to
modeling, namely optimization of the process using the fitted model.  In
addition to optimization of plain factorial experiments, a response
model strategy (Welch, Yu, Kang and Sacks, 1990 and Shoemaker, Tsui and
Wu, 1991) for optimization of Taguchi's ``robust parameter design''
experiments is developed.  An important advantage of the Bayesian
approach is that the posteriors capture model uncertainty, which is
incorporated in the optimization process to produce a more realistic
choice of optimum levels.

One of the key issues in fitting this model is variable selection.
Unlike ANOVA methods, which can determine variable importance
marginally, it is necessary to search the entire model space.  A
generalization of the basic model that incorporates automatic Bayesian
variable selection (George and McCulloch, 1993) is explored.  This
method searches the entire model space, and identifies the most probable
models, with the a priori assumption that selection of terms is
independent.  In industrial problems, many terms in the model are
related to one another, so that independence is not a reasonable
assumption.  For example, if neither main effect A or B is
identified as important, models containing an interaction AB may be
considered implausible.  Prior structures that represent this and other
relations among predictors are developed and applied to industrial
problems and data.  The relation considered include dummy variables,
polynomial predictors, mutually exclusive predictors, and restrictions
on the numbers of active predictors.  Although the variable selection
part of the thesis is developed for ordered categorical data, it may be
applied to other regression problems, including linear models.