Body temperature acting an important role in medicine, a number of diseases are characterized by a change in human body temperature. Monitoring body temperature also allows the doctor to track the effectiveness of treatments. But current continuous body temperature measurement (CBTM) system is mainly limited by reaction time, movement noise, and labor requirement. In addition, the traditional contact body temperature measurement has the problem of wasting consumables and causing discomfort. To address above issues, we present a noncontact, automatic CBTM system using a single thermal camera. By applying deep-learning based face detection, object tracking, and calibrated conversion equation, we can successfully extract subject’s forehead temperature in real-time. The experimental results show that the overall mean absolute error (MAE) and root-mean-squared-error (RMSE) of our proposed framework compared with industrial instrument are 0.375 °C and 0.439 °C, respectively.