Abstract: In the process of underground engineering construction, tuff in the fault fracture zone under a three-dimensional stress state loses particles under the action of fluid–solid coupling, causing the structural instability of the fault fracture rock. Finally, fault water inrush disaster occurs. Based on this, the field fault sampling has been conducted, and the broken rock triaxial seepage test system has been used to investigate the phenomenon of particle loss in samples with various particle sizes under triaxial load, as well as the effect of particle loss on pore structure and the time-varying evolution of seepage velocity. The following are the results: (1) The quality and time of the particle loss of broken tuff satisfy the exponential nonlinear relationship under different levels of triaxial stress, with a correlation coefficient of not less than 94%. Particle loss quality is inversely related to axial pressure and confining pressure, indicating that the higher the axial displacement, the smaller the decrease in particle loss mass with confining pressure. (2) The porosity increases rapidly between 0 and 60 s during the infiltration process. The seepage evolution process of the pore structure is related to the particle size gradation; that is, the overall porosity increases as the value of n (Talbot power exponent) increases. In the case of the same value of n, the porosity, which ranges from 0.33 to 0.52, decreases as the axial displacement and confining pressure increase. (3) Owing to the regular loss of particles in the sample, the time-varying evolution process of the seepage velocity of fractured tuff can be divided into three stages: stable seepage, sudden increase of seepage velocity, and approximate pipe flow. When the confining pressure is 0.8 MPa, each stage’s flow velocity is higher than that of the corresponding stage when the confining pressure is 1.4 MPa. The stable seepage stage has a short duration and low flow rate, and its occurrence times decrease as the n value increases. In the stage of seepage velocity surge, velocity surges to a peak value. The approximate pipe flow stage maintains a relatively stable and high flow velocity despite occasional fluctuations. The research results can offer a theoretical basis for studying the evolution law of fault water inrush disaster.