Efficient Deep Learning via Sparse Subnetwork Characterization
Deep neural networks (DNNs) are powerful tools for solving complex, nonlinear problems, yet their training and deployment often demand substantial computational and memory resources. Modern DNNs are typically initialized with random weights and overparameterized to ensure task performance, resulting in inefficiencies during training and inference. Neural network pruning has shown that over 90% of parameters in a trained model can be removed without loss in accuracy, offering significant reductions in compute and storage requirements. However, pruning is computationally expensive, data-specific, and typically performed post-training, limiting its utility in dynamic or resource-constrained environments.

This work aims to advance the efficiency of deep learning by empirically analyzing the structure and parameterizations of pruned, high-performing subnetworks. We hypothesize that these sparse networks exhibit consistent, learnable characteristics that can be exploited a priori—before or during training—to reduce computational cost while maintaining task performance. Our research will explore statistical, structural, and topological features of these subnetworks to identify the attributes that support effective learning and generalization.

By characterizing the essential components of performant neural architectures, we aim to enable more efficient DNN initialization strategies, accelerate convergence, and reduce the computational footprint of deep learning workloads. These findings have direct implications for scaling AI across HPC environments, where minimizing training time and maximizing throughput are critical. This work will contribute to the PI’s Ph.D. dissertation and help bridge gaps between machine learning performance and HPC efficiency.
IMPACT
This research will identify and leverage optimal parameter subsets within deep neural networks (DNNs) to enable more efficient, accurate, and robust AI systems across a wide range of applications, from pattern recognition to predictive analytics. By reducing computational demands and enhancing performance, this work will accelerate the deployment of advanced AI capabilities in Army and DoD missions—from edge computing and robotics to cyber situational awareness and automated knowledge extraction—directly supporting all six Army modernization priorities.
PRESENTER
Donovan, Jordan
jordan.t.donovan@erdc.dren.mil
601-988-3575
US Army Engineer Research and Development Center
CATEGORY
AI/ML for HPC
SECONDARY CATEGORY
Other
SYSTEM(S) USED
Carpenter, Narwhal, Nautilus