Higher-order inverse regression methods are commonly known as more powerful sufficient dimension reduction (SDR) methods than the popularly used sliced inverse regression (SIR) in the population level. However, due to the convention of essentially conducting singular value decomposition on the ambient candidate matrices, these methods suffer from the excessive number of parameters in the sample level and have not been systematically generalized under the high-dimensional settings like SIR. In this paper, we break the convention of using the ambient candidate matrices in these methods, and instead apply a novel column-selection strategy on their candidate matrices that substantially lowers down the working number of parameters to being comparable with SIR. Then, for the first time of the literature, we generalize the higher-order inverse regression methods, as well as their ensembles, towards sparsity under the high-dimensional settings in a uniform manner. The dimension of the predictor is allowed to diverge with the sample size in nearly an exponential order, and no additional restrictions are imposed on the data other than those commonly seen in the high-dimensional literature. For com[1]pleteness of theory, we also study the column-selection strategy towards the estimation efficiency under the conventional low-dimensional settings. These results are illustrated by simulation studies and a real data application at the end.