Speed of sound estimation in ultrasound imaging is a growing modality with several clinical applications such as hepatic steatosis stages quantification. A key challenge for clinically relevant speed of sound estimation is to obtain repeatable values independent of superficial tissues and available in real-time. Recent works have demonstrated the feasibility to achieve quantitative estimations of the local speed of sound in layered media. However, such techniques require high computational power and exhibit instabilities. We present a novel speed of sound estimation technique based on an angular approach of ultrasound imaging in which plane waves are considered in transmit and receive. This change of paradigm allows us to rely on the refraction properties of plane waves to infer the local speed of sound values directly from the angular raw data. The proposed method robustly estimates the local speed of sound with only a few ultrasound emissions and with a low computational complexity which makes it compatible with real-time imaging. Simulations and in vitro experimental results show that the proposed method outperforms state-of-the-art approaches with biases and standard deviations lower than 10 m s-1, eight times fewer emissions, and 1000 times lower computational time. Further in vivo experiments validate its performance for liver imaging.