On the relative accuracy of the skull sex estimation

Abstract

Introduction. There are no fewer than two hundred algorithms for sex estimation based on cranial morphology, relying on statistical analysis of non-metric, linear, angular traits, and their combinations. Nevertheless, many physical anthropologists prefer to rely on visual observations. The objectives of this research encompass exploring potential reasons behind the preference for a visual approach and conducting an analysis of the comparative effectiveness of visual and statistical methods for sex estimation.

Materials and methods. The study is grounded in an analysis of publications related to methods of sex estimation based on cranial traits, spanning the past 70 years. Comparison of accuracy estimates was conducted using non-parametric tests, considering differences in statistical methods, validation approaches (no validation, cross-validation, independent test), and variable types (non-metric traits, craniometry, geometric morphometrics).

Results. General reasons for skepticism towards algorithms include unrealistic expectations regarding their capabilities, greater susceptibility to errors by models compared to humans, lack of control over classification. However, algorithms generally surpass experts in predicting the target variable. The average accuracy of visual sex estimations based on cranial traits is slightly lower than the estimates of statistical models and exhibits noticeable variability. The accuracy of estimations made by experienced anthropologists is comparable to the average performance of models. Nevertheless, the effectiveness of algorithms significantly diminishes when applied to datasets originating from sources other than the training set, particularly when dealing with craniometric traits. In a substantial portion of studies, the size of the training datasets is insufficient for a reliable assessment of model effectiveness, and the sex distribution is skewed towards male skulls, leading to some inflation of the accuracy of their estimates. Model effectiveness can also decline due to errors in the evaluation of non-metric traits, and the assessment of inter-researcher discrepancies does not allow for an evaluation of their impact on model accuracy.

Conclusion. Despite an extensive bibliography, there remains a lack of data on both the accuracy of the visual approach to sex estimation and the reliability of models with claimed high effectiveness. The adoption of flexible methodologies enabling researchers to independently control both variable selection and the composition of the training set will help overcome algorithm aversion and enhance the quality of estimates. © 2023. This work is licensed under a CC BY 4.0 license.

References

Alekseev V.P., Debets G.F. Kraniometriya. Metodika antropologicheskikh issledovanii [Craniometry. Methodology of anthropological research]. Moscow, Nauka Publ., 1964. 128 p. (In Russ).

Zvyagin V.N. Metodika kranioskopicheskoi diagnostiki pola cheloveka [Methodology of cranioscopic diagnostics of human sex]. Sudebno-meditsinskaya ehkspertiza [Forensic Medical Expertise / Sudebno-Meditsinskaya Ekspertisa], 1983, 3, pp.15-17. (In Russ.).

Kahneman D., Sibony O., Sunstein C.R. Shum. Nesovershenstvo chelovecheskikh suzhdenii [Noise: A flaw in human judgement]. Moscow, AST Publ., 2021. 544 р. (In Russ.)

 Berg G.E., Tersigni-Tarrant A. Sex and ancestry determination: assessing the “gestalt”. Proceedings of the 66th Annual Meeting of the American Academy of Forensic Sciences; 2014 Feb 17-22; Seattle, WA. Colorado Springs, CO, American Academy of Forensic Sciences, 2014. pp. 414-415.

Berger B., Adam M., Rühr A., Benlian A. Watch me improve – algorithm aversion and demonstrating the ability to learn. Bus. Inf. Syst. Eng., 2021, 63, pp. 55-68. DOI: 10.1007/s12599-020-00678-5.

Burton J.W., Stein M., Jensen T.B. A systematic review of algorithm aversion in augmented decision making. J. Behavioral Decision Making, 2020, 33 (2), pp. 220-239. DOI: 10.1002/bdm.2155.

Camps F.E. Gradwohl’s legal medicine. Identification by the skeletal structures. 2nd ed. Bristol, John Wright & Sons Ltd., 1968, pp. 123-140.

Dawes R., Faust D., Meehl P. Clinical versus actuarial judgment. Science, 1989, 243 (4899), pp. 1668-1674. DOI: 10.1126/science.2648573.

Del Bove A., Veneziano A. A generalised neural network model to estimate sex from cranial metric traits: a robust training and testing approach. Applied Sciences, 2022, 12, 9285. DOI: 10.3390/app12189285.

Dereli A.K., Zeybek V., Sagtas E., Senol H., Ozgul H.A. et al. Sex determination with morphological characteristics of the skull by using 3D modeling techniques in computerized tomography. Forensic Sci. Med. Pathol., 2018, 14, pp. 450-459. DOI: 10.1007/s12024-018-0029-0.

Dietvorst B.J., Simmons J.P., Massey C. Algorithm aversion: People erroneously avoid algorithms after seeing them err. J.Exper. Psych.: General, 2015, 144 (1), pp. 114-126. DOI: 10.1037/xge0000033.

Đurić M., Rakočević Z., Đonić D. The reliability of sex determination of skeletons from forensic context in the Balkans. Forensic Sci. Int., 2005, 147 (2-3), pp. 159-164.

Giles E., Elliot O. Sex determination by discriminant function analysis of crania. Am. J. Phys. Anthropol., 1963, 21 (1), 53-68. DOI: 10.1002/ajpa.1330210108.

Henke W. On the method of discriminant function analysis for sex determination of the skull. J. Hum. Evol., 1977, 6 (2), pp. 95-100. DOI:10.1016/S0047-2484(77)80111-5.

Hrdlička A. Practical anthropometry. Philadelphia, The Wistar Institute of Anatomy and Biology, 1939. 231 p.

Kelley S.R., Tallman S.D. Population-inclusive assigned-sex-at-birth estimation from skull computed tomography scans. Forensic Science, 2022, 2, pp. 321-348. DOI: 10.3390/forensicsci2020024.

Krogman W.M., İşсan M.Y. The human skeleton in forensic medicine. Springfield, IL, Charles C. Thomas, 1986. 576 p.

Lewis C.J., Garvin H.M. Reliability of the Walker cranial nonmetric method and implications for sex estimation. J. Forensic Sci., 2016, 61 (3), pp. 743-751. DOI: 10.1111/1556-4029.13013.

Meindl R.S., Lovejoy C.O., Mensforth R.P., Carlos L.D. Accuracy and direction of error in the sexing of the skeleton: Implications for paleodemography. Am. J. Phys. Anthropol., 1985, 68 (1), pp. 79-85. DOI:10.1002/ajpa.1330680108.

Nakhaeizadeh S., Dror I.E., Morgan R.M. Cognitive bias in sex estimation: The influence of context on forensic decision-making. Sex Estimation of the Human Skeleton, 2020, pp. 327-342. DOI:10.1016/b978-0-12-815767-1.00020-1.

Renier L.A., Schmid Mast M., Bekbergenova A. To err is human, not algorithmic – Robust reactions to erring algorithms. Computers in Human Behavior, 2021, 124, 106879. DOI: 10.1016/j.chb.2021.106879.

Spradley M.K., Jantz R.L. Sex estimation in forensic anthropology: skull versus postcranial elements. J. Forensic Sci., 2011, 56 (2), pp.289-296. DOI:10.1007/ 978-1-59745-099-7_9.

Stewart T.D. Sex determination of the skeleton by guess and by measurement. Am. J. Phys. Anthropol., 1954, 12 (3), pp. 385-392. DOI: 10.1002/ajpa.1330120312.

Stewart T.D. Essentials of forensic anthropology. Springfield IL, Charles C. Thomas, 1979. 317 p.

Thomas R.M., Parks C.L., Richard A.H. Accuracy rates of sex estimation by forensic anthropologists through comparison with DNA typing results in forensic casework. J. Forensic Sci., 2016, 61 (5), pp. 1307-1310. DOI: 10.1111/1556-4029.13137.

Walker P.L. Sexing skulls using discriminant function analysis of visually assessed traits. Am. J. Phys. Anthropol., 2008, 136 (1), 39–50. DOI: 10.1002/ajpa.20776.

Walrath D.E., Turner P., Bruzek J. Reliability test of the visual assessment of cranial traits for sex determination. Am. J. Phys. Anthropol., 2004, 125 (2), pp. 132-137. DOI: 10.1002/ajpa.10373.

Weiss K.M. On the systematic bias in skeletal sexing. Am. J. Phys. Anthropol., 1972, 37 (2), pp. 239-249. DOI: 10.1002/ajpa.1330370208.

Williams B.A., Rogers T.L. Evaluating the accuracy and precision of cranial morphological traits for sex determination. J. Forensic Sci., 2006, 51, pp. 729-735. DOI: 10.1111/j.1556-4029.2006.00177.x.