Markowitz's modern portfolio theory is one of the major milestones in finance. However, the empirical performance of the resulting optimal portfolio has received much criticism by many researchers. Bai et al. (Math Finance 19(4):639–667, 2009a) proved that the estimated optimal portfolio always overestimates the expected return, particularly when the portfolio consists of a large number of assets and they proposed a bootstrap-corrected estimator. Based on their work, Leung et al. (Eur J Oper Res 222(1):85–95, 2012) then proposed a new estimator with a closed-form expression. To some extent, their estimator reduces the mean square error but does not minimize it. In this paper, we propose an improved estimator of Markowitz's optimal portfolio and our simulation studies reveal that the new estimators outperform Leung's estimator and other existing estimators, especially when the number of assets is large. Finally, we apply the new estimators to the US stock market and our methods can achieve both the largest utility and the highest return in most cases. Copyright © 2019 Grace Scientific Publishing.