BEYOND MODERN PORTFOLIO THEORY: DEEP LEARNING FOR NOISE-RESILIENT ASSET ALLOCATION

Abstract

The paper presents a comprehensive investigation into the integration of Deep Learning architectures with Random Matrix Theory to address the curse of dimensionality and estimation noise in Modern Portfolio Theory. Classical Mean-Variance Optimization relies on the sample covariance matrix, which, in high-dimensional financial settings where the number of assets is comparable to the observation period, becomes inherently unstable due to the spreading of eigenvalues. This study proposes a novel Neural-Nonlinear Shrinkage framework. It leverages Denoising Autoencoders and 1D-Convolutional Neural Networks to extract latent risk factors from raw return series, effectively filtering out the Marchenko-Pastur noise that typically plagues empirical financial data. The research bridges the gap between spectral analysis of random matrices and the non-linear feature extraction capabilities of deep architectures by providing a robust preprocessing pipeline for financial time series. The purpose of the work is to operationalize a robust covariance estimation pipeline that outperforms traditional shrinkage methods like Ledoit-Wolf by capturing non-linear tail dependencies and time-varying volatility clusters. The study aims to determine if neural-regularized matrices lead to more stable, efficient frontiers and lower turnover costs in rebalancing compared to standard statistical estimators. The methodology encompasses formalizing the noise-to-signal ratio in high-dimensional financial data, applying the Marchenko-Pastur law to identify the noise bulk in empirical return distributions, implementing a Bottleneck Autoencoder to compress n-dimensional returns into a lower-dimensional manifold of clean risk factors, and performing comparative back-testing using Quadratic Programming to solve for minimum variance and maximum Sharpe ratio portfolios across different market cycles. The scientific novelty. For the first time, the study formalizes a Spectral-Loss Function for Autoencoders, which penalizes the reconstruction not just on Mean Squared Error, but on the deviation of the resulting covariance matrix's condition number from theoretical stability bounds. This physics-informed approach to AI ensures that the model respects the mathematical constraints of positive semi-definiteness in financial matrices. The practical value lies in providing quantitative hedge funds and institutional asset managers with a defensible, automated pipeline for large-scale portfolio construction. It significantly reduces weight flipping and improves out-of-sample risk-adjusted returns by mitigating the impact of outliers and reducing the optimization engine's sensitivity to small changes in input data. Conclusions: Effective portfolio optimization in modern markets requires moving beyond static historical averages. The results demonstrate that neural-regularized portfolios achieve a significant reduction in realized volatility compared to standard Mean-Variance Optimization. By cleaning the covariance matrix through a deep-learning lens, the study transforms theoretical models into robust, production-ready tools for high-dimensional finance.