The coarse-grained soil is prone to particle breakage under external loads and other factors. A series of single-particle fragmentation tests is conducted for mudstone and sandstone particles. Based on the size effect and fractal model of particle fragmentation, the relationships between the fractal dimension and single-particle crushing strength, fragmentation energy, and Weibull modulus are investigated. A single particle crushing process is analyzed using PFC3D and the modelling results are compared with that from the experimental data to verify the reliability of numerical code. The crushing strength and crushing energy of large particle size are then analyzed by numerical models. The results show that the fractal dimensions of different materials are different under the same test conditions. The fragmentation degree of sandstone with different grain sizes is greater than that of mudstone. The crushing strength of a single particle has an obvious size effect. In addition, the crushing strength and energy of single particle can be predicted by fractal dimension and particle size. The modified Weibull modulus can be also predicted by fractal dimension. The numerical simulation results agree with the experimental results and also agree with the predicted results. Besides, the modelling results of the single particle crushing strength with large particle size are also consistent with the predicted results. The crushing energy, however, shows slightly different, which requires further experimental verification. The research results can provide a reference for obtaining the single particle strength and deformation characteristics of large-size coarse-grained soil.