应用数学学报(英文版)
HOME | ABOUT JOURNAL | EDITORIAL BOARD | FOR AUTHORS | SUBSCRIPTIONS | ADVERTISEMENT | CONTACT US
 
Acta Mathematicae Applicatae
Sinica, Chinese Series
 
   
   
Adv Search »  
Acta Mathematicae Applicatae Sinica, English Series 2011, Vol. 27 Issue (4) :601-612    DOI: 10.1007/s10255-011-0110-x
ARTICLES Current Issue | Next Issue | Archive | Adv Search << | >>
Efficient Algorithms for Generating Truncated Multivariate Normal Distributions
Jun-wu YU1, Guo-liang TIAN2
1. School of Mathematics and Computation Science, Hunan University of Science and Technology, Xiangtan 411201, China;
2. Department of Statistics and Actuarial Science, The University of Hong Kong, Pokfulam Road, Hong Kong, China
Download: PDF (1KB)   HTML (1KB)   Export: BibTeX or EndNote (RIS)      Supporting Info
Abstract Sampling from a truncated multivariate normal distribution (TMVND) constitutes the core computational module in fitting many statistical and econometric models. We propose two efficient methods, an iterative data augmentation (DA) algorithm and a non-iterative inverse Bayes formulae (IBF) sampler, to simulate TMVND and generalize them to multivariate normal distributions with linear inequality constraints. By creating a Bayesian incomplete-data structure, the posterior step of the DA algorithm directly generates random vector draws as opposed to single element draws, resulting obvious computational advantage and easy coding with common statistical software packages such as S-PLUS, MATLAB and GAUSS. Furthermore, the DA provides a ready structure for implementing a fast EM algorithm to identify the mode of TMVND, which has many potential applications in statistical inference of constrained parameter problems. In addition, utilizing this mode as an intermediate result, the IBF sampling provides a novel alternative to Gibbs sampling and eliminates problems with convergence and possible slow convergence due to the high correlation between components of a TMVND. The DA algorithm is applied to a linear regression model with constrained parameters and is illustrated with a published data set. Numerical comparisons show that the proposed DA algorithm and IBF sampler are more efficient than the Gibbs sampler and the accept-reject algorithm.  
Service
Email this article
Add to my bookshelf
Add to citation manager
Email Alert
RSS
Articles by authors
Keywordsdata augmentation   EM algorithm   Gibbs sampler   IBF sampler   linear inequality constraints   truncated multivariate normal distribution     
Abstract: Sampling from a truncated multivariate normal distribution (TMVND) constitutes the core computational module in fitting many statistical and econometric models. We propose two efficient methods, an iterative data augmentation (DA) algorithm and a non-iterative inverse Bayes formulae (IBF) sampler, to simulate TMVND and generalize them to multivariate normal distributions with linear inequality constraints. By creating a Bayesian incomplete-data structure, the posterior step of the DA algorithm directly generates random vector draws as opposed to single element draws, resulting obvious computational advantage and easy coding with common statistical software packages such as S-PLUS, MATLAB and GAUSS. Furthermore, the DA provides a ready structure for implementing a fast EM algorithm to identify the mode of TMVND, which has many potential applications in statistical inference of constrained parameter problems. In addition, utilizing this mode as an intermediate result, the IBF sampling provides a novel alternative to Gibbs sampling and eliminates problems with convergence and possible slow convergence due to the high correlation between components of a TMVND. The DA algorithm is applied to a linear regression model with constrained parameters and is illustrated with a published data set. Numerical comparisons show that the proposed DA algorithm and IBF sampler are more efficient than the Gibbs sampler and the accept-reject algorithm.  
Keywordsdata augmentation,   EM algorithm,   Gibbs sampler,   IBF sampler,   linear inequality constraints,   truncated multivariate normal distribution     
Received: 2009-09-19;
Fund:

Supported by the National Social Science Foundation of China (No. 09BTJ012) and Scientific Research Fund of Hunan Provincial Education Department (No. 09c390). G.L. Tian's research was supported in part by a HKU Seed Funding Program for Basic Research (Project No. 2009-1115-9042) and a grant from Hong Kong Research Grant Council-General Research Fund (Project No. HKU779210M).

Cite this article:   
.Efficient Algorithms for Generating Truncated Multivariate Normal Distributions[J]  Acta Mathematicae Applicatae Sinica, English Serie, 2011,V27(4): 601-612
URL:  
http://www.applmath.com.cn/jweb_yysxxb_en/EN/10.1007/s10255-011-0110-x      或     http://www.applmath.com.cn/jweb_yysxxb_en/EN/Y2011/V27/I4/601
 
[1] Breslaw, J.A. Random sampling from a truncated multivariate normal distribution. Applied MathematicsLetters, 7: 1-6 (1994)
[2] Chen, M.-H., Deely, J.J. Bayesian analysis for a constrained linear multiple regression problem for predictingthe new crop of apples. Journal of Agricultural, Biological and Environmental Statistics, 1: 467-489 (1996)
[3] Chen, M.-H., Dey, D.K. Bayesian analysis for correlated ordinal data models. In: Generalized LinearModels: A Bayesian Perspective, D.K. Dey, S.K. Ghosh and B. K. Mallick, eds., Marcel Dekker, New York,2000, 133-157
[4] Chen, M.-H., Shao, Q.M., Ibrahim, J.G. Monte Carlo Methods in Bayesian Computation. Springer-Verlag,New York, 2000
[5] Chib, S. Bayesian methods for correlated binary data. In: Generalized Linear Models: A Bayesian Perspective,D.K. Dey, S.K. Ghosh and B. K. Mallick, eds., Marcel Dekker, New York, 2000, 113-131
[6] Chib, S., Greenberg, E. Analysis of multivariate probit models. Biometrika, 85: 347-361 (1998)
[7] Cohen, A., Kemperman, J.H.B., Sackrowitz, H.B. Unbiased tests for normal order restricted hypotheses.Journal of Multivariate Analysis, 46: 139-153 (1993)
[8] Devroye, L. Non-Uniform Random Variate Generation. Springer-Verlag, New York, 1985
[9] Fang, K.T., Wang, Y. Number-theoretic Methods in Statistics. Chapman & Hall, London, 1994
[10] Fraser, D.A.S.,Massam, H. A mixed primal-dual Bases algorithm for regression under inequality constraints:Application to concave regression. Scandinavian Journal of Statistics, 16: 65-74 (1989)
[11] Gelfand, A.E., Smith, A.F.M., Lee, T.M. Bayesian analysis of constrained parameter and truncated dataproblems using Gibbs sampling. Journal of the American Statistical Association, 87: 523-532 (1992)
[12] Gelman, A., Rubin, D.B. Inference from iterative simulation using multiple sequences. Statistical Science,7: 457-511 (1992)
[13] Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B. Bayesian Data Analysis. Chapman & Hall, London,1995
[14] Geweke, J. Efficient simulation from the multivariate normal and student-t distributions subject to linearconstraints. In: Computing Science and Statistics: Proceedings of the 23-rd Symposium on the Interface,E.M. Keramidas, ed., Interface Foundation of North America, Inc., Fairfax, 571-578, 1991
[15] Geweke, J., Keane, M., Runkle, D. Alternative computational approaches to inference in the multinomialprobit model. Review of Economics and Statistics, 76: 609-632 (1994)
[16] Geweke, J., Keane, M., Runkle, D. Statistical inference in the multinomial multiperiod probit model.Journal of Econometrics, 80: 125-166 (1997)
[17] Girard, P., Parent, E. Bayesian analysis of autocorrelated ordered categorical data for industrial qualitymonitoring. Technometrics, 43: 180-191 (2001)
[18] Hajivassiliou, V.A. Simulation for multivariate normal rectangle probabilities and their derivatives: Theeffects of vectorization. International Journal of Supercomputer Applications, 7: 231-253 (1993)
[19] Hajivassiliou, V.A., Ruud, P. Classical estimation methods using Simulation. In: Handbook of Econometrics,R. Engle and D. L. McFadden, eds., North-Holland, Amsterdam, Vol.4, 2383-2441, 1994
[20] Hajivassiliou, V.A., McFadden, D.L., Ruud, P. Simulation of multivariate normal rectangle probabilitiesand their derivatives: Theoretical and computational results. Journal of Econometrics, 72: 85-134 (1996)
[21] Hajivassiliou, V.A., McFadden, D.L. The method of simulated scores for the estimation of LDV models.Econometrica, 66: 863-896 (1998)
[22] Huang, J. A note on estimating a partly linear model under monotonicity constraints. Journal of StatisticalPlanning and Inference, 107: 343-351 (2002)
[23] Liu, C.H. Bayesian analysis of multivariate probit models. Comment on “The art of data augmentation”by D. Van Dyk and X. L. Meng. Journal of Computational and Graphical Statistics, 10: 75-81 (2001)
[24] McFadden, D.L. A method of simulated moments for estimation of discrete response models withoutnumerical integration. Econometrica, 57: 995-1026 (1989)
[25] McFadden, D.L. Lectures on simulation-assisted statistical inference. EC-squared Conference, Florence,Italy, December 12-14. http://elsa.berkeley.edu/ ~mcfadden, 1996
[26] McFadden, D.L., Train, K. Mixed MNL models for discrete response. Journal of Applied Econometrics,15: 447-470 (2000)
[27] Meyer, M.C. An extension of the mixed primal-dual bases algorithm to the case of more constraints thandimensions. Journal of Statistical Planning and Inference, 81: 13-31 (1999)
[28] Meyer, M.C. A test for linear versus convex regression function using shape-restricted regression. Biometrika,90: 223-232 (2003)
[29] Meyer, M.C., Woodroofe, M. On the degrees of freedom in shape-restricted regression. The Annals ofStatistics, 28: 1083-1104 (2000)
[30] Robert, C.P. Simulation of truncated normal variables. Statistics and Computing, 5: 121-125 (1995)
[31] Robertson, T., Wright, F.T., Dykstra, R.L. Order Restricted Statistical Inference. Wiley, New York, 1988
[32] Rubin, D.B. Using the SIR algorithm to simulate posterior distributions (with discussion). In: BayesianStatistics, 3, J.M. Bernardo, M.H. DeGroot, D.V. Lindley and A.F.M. Smith, eds., Oxford University Press,Oxford, 1988, 395-402
[33] Smith, A.F.M., Gelfand, A.E. Bayesian statistics without tears: a sampling-resampling perspective. TheAmerican Statistician, 46: 84-88 (1992)
[34] Tan, M., Fang, H.B., Tian, G.L., Houghton, P.J. Small-sample inference for incomplete longitudinal datawith truncation and censoring in tumor xenograft models. Biometrics, 58: 612-620 (2002)
[35] Tan, M., Tian G.L., Fang, H.B. Estimating restricted normal means using the EM-type algorithms andIBF sampling. In: Development of Modern Statistics and Related Topics — In Celebration of ProfessorYaoting Zhang's 70th Birthday, H. Zhang and J. Huang, eds., World Scientific, New Jersey, 53-73, 2003
[36] Tan, M., Tian, G.L., Ng, K.W. A non-iterative sampling method for computing posteriors in the structureof EM-type algorithms. Statistica Sinica, 13: 625-639 (2003)
[37] Tan, M., Tian, G.L., Fang, H.B., Ng, K.W. A fast EM algorithm for quadratic optimization subject toconvex constraints. Statistica Sinica, 17(3): 945-964 (2007)
[38] Tan, M., Tian, G.L., Ng, K.W. Bayesian Missing Data Problems: EM, Data Augmentation and NoniterativeComputation. Chapman & Hall/CRC (Monographs in Biostatistics Series), Boca Raton, USA,2009
没有找到本文相关文献
Copyright 2010 by Acta Mathematicae Applicatae Sinica, English Serie