Comparison between total least squares and ordinary least squares in obtaining the linear relationship between stable water isotopes

Jeonghoon Lee, Won Sang Lee, Hyejung Jung, Seung Gu Lee

Research output: Contribution to journalComment/debate

2 Scopus citations


The linear relationship between two stable water isotopes (δD and δ18O) has been used to examine the physical processes and movements or changes of three water phases (water vapor, liquid water and ice), including deuterium excess. The ordinary least squares (OLS) method has been the most commonly used method to fit the linear relationship between two isotopic compositions of water. However, an alternative method, the total least squares (TLS) method, has been proposed because it considers the presence of errors in the explanatory variable (horizontal axis, δ18O). However, not many studies have examined the differences of the relationship using two stable isotopes between the OLS and TLS for various types of water. In this work, these two methods were compared using isotopic compositions of three types of water (Antarctic snow, water vapor and summer and winter rainfall). Statistically, the slopes and intercepts obtained by the two linear regression methods were not significantly different except for summer rainfall, which has the smallest coefficient of variations (R2). The TLS method produced larger slopes than the OLS method and the degrees of difference between the two methods were greater when the coefficient of variation was lower. In addition, with a Monte Carlo method, we showed that the differences between the two methods increased as the uncertainty increased. Moreover, the results of Bayesian linear regression were consistent with the two linear regressions. Although the TLS method is theoretically more suited to the linear regression for the stable water isotopes than the OLS method is, the application of the widely used OLS method can be recommended in the case of small measurements uncertainties after testing whether the linear parameters, slopes and intercepts, derived from the two methods are statistically significant different.

Original languageEnglish
Article number11
JournalGeoscience Letters
Issue number1
StatePublished - Dec 2022

Bibliographical note

Funding Information:
This work was sponsored by a research grant from the Korean Ministry of Oceans and Fisheries (KIMST20190361) and partially supported by the principal Research Fund of the Korea Institute of Geoscience and Mineral Resources (GP2017-018).

Publisher Copyright:
© 2022, The Author(s).


  • Monte Carlo
  • Ordinary least squares
  • Stable water isotopes
  • Total least squares


Dive into the research topics of 'Comparison between total least squares and ordinary least squares in obtaining the linear relationship between stable water isotopes'. Together they form a unique fingerprint.

Cite this