Taking the large-scale toppling failure counter-tilted slope in Changshanhao open-pit mine in Inner Mongolia as engineering background, a layered counter-tilted slope geological generalized physical model was constructed based on similar theory. Physical model tests were performed under three different conditions: unreinforced, ordinary Possion’s Ratio(PR) anchor cable reinforcement, and constant resistance and large deformation Negative Possion’s Ratio (NPR) anchor cable reinforcement. Then infrared thermal imaging system, strain monitoring system and digital speckle displacement measurement system were used comprehensively to monitor the temperature field, strain field and displacement field respectively during the entire counter-tilted slope excavation test. Finally the deformation characteristics and instability mechanism of layered counter-tilted slope were investigated, and the reinforcement effects under different reinforcement measures were compared and analyzed. The research results show that the deformation process of counter-tilted slope has obvious deformation characteristics of "superimposed cantilever beam". Its toppling mechanism is mainly manifested in three stages: initial crack formation, crack development, and sliding surface penetrating and slope instability. The PR anchor cable failed during the excavation and could not resist the large deformation of the slope. In contrast, the NPR anchor cable has the characteristics of high constant resistance, large deformation and energy absorption, which can effectively prevent the occurrence of counter-tilted slope instability and provide a new way for slope engineering reinforcement.