In modern massive data modelling, distributed learning plays a critical role in enhance scalability, efficiency and privacy protection. Heterogeneity and robustness of a distributed learning algorithm are key aspects related to the accuracy and reliability of learning result. In this work, under the common framework of statistical learning, we propose the convex hull search technique and algorithm derived from it. The proposed algorithm has four main advantages: fast convergence, high accuracy, tuning friendness and Byzantine robust. The corresponding convergence and asymptotic normality result for our algorithm are established which show its adaptability on data heterogeneity. Examples of the application of our algorithms has been given on regression and clustering tasks through synthetic data. Furthermore, a real energy load data is implemented for Gaussian process regression hyperparameters optimization. Existing numerical result confirm superiority and exhibit wide applicability of our algorithms.