Many DOE science applications are now beginning to use models based on machine learning to complement traditional models based on the numerical solution of partial differential equations. Sparse matrix kernels are important for both machine learning based models and simulation based models. However, the performance of sparse-matrix computations can be very sensitive to the sparsity structure of the non-zero elements. Further, the sparsity patterns encountered in machine learning applications can differ considerably from those seen in PDE simulations. Sparse-matrix kernels available in libraries such as Intel's MKL and Nvidia's cuSPARSE exhibit significant performance variability across matrices of different sparsity structure, and ultra-high performance research implementation require non-standard data structures.
During the development of ES-SciLA we will design, implement, and commercialize a set of sparsity-adaptive linear algebra primitives that can effectively support a wide range of applications. We aim to reduce the significant variability in performance across different matrices seen with current state-of-the-art sparse-matrix libraries. We propose to develop a sparsity-adaptive approach for sparse linear algebra and sparse tensors. We seek advances in achieving high-productivity and high-performance with sparse matrix computations on heterogeneous architectures (multi-core processors and GPUs). The production library will include a wide range of linear algebra primitives. The focus of the initial Phase I proof of concept will be a sparsity-adaptive CPU and GPU implementation of SpMM. The Phase I work will leverage the concepts developed for optimized sparse linear algebra, and apply them to an optimized implementation with standard (CSR) data structures. This will allow users who require standard data structures to use a high performance implementation that is sparsity-adaptive. The sparsity-adaptive operators will be expanded during Phase II and technology transfer/commercialization, and will include state-of-the-art implementations that achieve high performance using standard and non-standard data structures. ES-SciLA will generate a matrix "signature" using sparsity and matrix features (e.g., sparsity, average non-zeros per row, non-zero block sizes, etc.) and the non-zero pattern.