【latex版】水贴

互联网+、众创空间、获得感、非首都功能、网约车、红通、小短假、阅兵蓝、人民币入篮、一照一码

A minimum relative entropy based correlation model
between the response and covariates

A semiparametric model is presented utilizing dependence between a response
and several covariates.We show that this model is optimum when the marginal distributions of
the response and the covariates are known. This model extends the generalized linear model
and the proportional likelihood ratio model when the marginal distributions are unknown. New
interpretations of known models such as the logistic regression model, density ratio model and
selection bias model are obtained in terms of dependence between variables. For estimation of
parameters, a simple algorithm is presented which is guaranteed to converge. It is also the same
regardless of the choice of the distribution for response and covariates; hence, it can fit a very
wide variety of useful models. Asymptotic properties of the estimators of model parameters are
derived.Real data examples are discussed to illustrate our approach and simulation experiments
are performed to compare with existing procedures.

Luo and Tsai (2012) described neuropsychological scale data where the response variable is the
score from the trail making test (part A) measuring 334 patients’ processing speed in seconds,
and the covariates are years of education, age and diagnosis. Often these types of data have several
covariates (Figs 1 and 2 in Section 7 show scatter plots of scores versus years of education
and age), have unknown statistical distributions and known statistical procedures fail to work
properly. Stamey et al. (1989) examined the correlations between the level of prostate-specific
antigen and several clinical measures in 97 men who were about to receive a radical prostatectomy.
The goal (Hastie et al., 2009) is to predict the logarithm of prostate-specific antigen level,
lpsa, from a number of measurements including log-cancer-volume, lcavol, log-prostate-weight,
lcp, age and logarithm of capsular penetration, lcp (Figs 5–7 in Section 7 show scatter plots
of lpsa versus lcavol, lweight and lcp respectively). Although there are moderately high correlations
between the response and most covariates, a linear regression model does not consider
the non-linearity at the edges of the data.

For a response Y with covariates Zi, 1id, the correlation coefficient corr.Y,Zi/=ci
measures any linear relationship between Y and Zi. When |ci| is much smaller than 1, then
the linear relationship between Y and Zi is very weak. Then we may think of the relationship
between Y and Zis as non-linear in nature. In this sense, ci not only measures the strength
of the linear but also the non-linear relationship between Y and Zi with higher or lower ci
referring respectively to more linearity or more non-linearity. Thus ci can be considered as a
global measure of dependence between Y and Zi. For example, if E.Y|Zi/=Z2
i with Zi being
symmetrically distributed around 0, then ci =0, then the dependence of Y on Zi is entirely
non-linear. Fig. 8(c) in Section 8 considers a similar situation with E.Y|Z/=|Z|.

In this paper, we develop a model for Y based on corr.Y,Zi/=ci, 1id. We shall see
that, when the specified cis are high, the model that is obtained is almost linear (e.g. prostate
data), whereas, when the specified cis are low, the model that is obtained is non-linear (e.g.
the trail making data). In any scientific study, the initial choice of covariates from a vast pool
may be difficult. Given a set of covariates, transformations of covariates or increasing the number
of correlation constraints with original or transformed variables might prove useful for a
better fit of the model developed. See Sections 7 and 8 for discussions on transformations of
covariates.
When the marginal distributions of Y and the Zis are known, our procedure has close connections
with the maximum entropy (ME) principle, which may be stated as follows:

The ME principle can be generalized to the concepts of Kullback–Leibler (KL) distance and
I-projection, as defined below (Csisz′ar, 1975). For two probability measures Q and P, the KL
distance (or, relative entropy) between Q and P is defined as
I.Q|P/=

ln
dQ
dP

dQ, if QP,
∞, otherwise:
.1:1/
(QP means that Q is absolutely continuous with respect to P.) Although I.Q|P/ is not a
metric, it is always non-negative and equals 0 if and only if Q=P. Hence it is often interpreted
as a measure of ‘divergence’ or ‘distance’ between Q and P. For a given P and a specified set of
probability measures C, it is often of interest to find the QÅ ∈C which satisfies

‘when selecting a model for a given situation  it is often appropriate to express the prior
information in
terms of constraints. However, one must be careful so that no information other than  these
specified constraints is used in model selection. That is, other than the constraints that we have,
the uncertainty associated  with the probability distribution to be selected should  be kept  at
its maximum’  (Jaynes,
1957).

The ME principle can be generalized to the concepts of Kullback–Leibler (KL) distance and I
-projection,  as defined below (Csisza′ r, 1975). For two probability measures Q and P , the KL
distance (or, relative entropy) between Q and P is defined as











I.Q|P/ =










( dQ
ln
dP










、 dQ,              if Q











× P ,











.1:1/












,                                   otherwise: