Comparative Analysis of Interpretability of Simple and Complex Machine Learning Models in Presence of Noise
Abstract
This paper offers a comprehensive analysis of the interpretability of key Machine Learning models, including ElasticNet regression, Random Forest, and Neural Networks, when faced with various types of noise. Focusing on both synthetic and real-world datasets of diverse sizes (385 to 15,000 samples), the study probes the models' ability to detect hidden patterns, especially in the presence of varied noise conditions (Gaussian, Perlin, and Simplex). Through systematic evaluation using Permutation Feature Importance (PFI) and SHAP summary plots, our research reveals a strong correlation between dataset size and model robustness to noise perturbations. The results demonstrate that larger datasets consistently lead to more stable feature importance rankings and better preservation of model interpretability under noise conditions. While ElasticNet shows superior performance on larger datasets, Neural Networks prove most sensitive to noise, particularly with smaller datasets. The findings provide valuable insights for practical applications of machine learning, suggesting that emphasis should be placed on acquiring larger training datasets to ensure robust and trustworthy model interpretations in noisy environments. This work contributes to the broader understanding of ML model interpretability and provides guidance for model selection in real-world applications where data noise is inevitable.How to Cite
References
Hügle M, Omoumi P, van Laar J M, et al. Applied machine learning and artificial intelligence in rheumatology[J]. Rheumatology advances in practice, 2020, 4(1): rkaa005.
Baduge S K, Thilakarathna S, Perera J S, et al. Artificial intelligence and smart vision for building and construction 4.0: Machine and deep learning methods and applications[J]. Automation in Construction, 2022, 141: 104440.
Kononenko I. Machine learning for medical diagnosis: history, state of the art and perspective[J]. Artificial Intelligence in medicine, 2001, 23(1): 89-109.
Murdoch W J, Singh C, Kumbier K, et al. Definitions, methods, and applications in interpretable machine learning[J]. Proceedings of the National Academy of Sciences, 2019, 116(44): 22071-22080.
Guidotti R, Monreale A, Ruggieri S, et al. A survey of methods for explaining black box models[J]. ACMcomputing surveys (CSUR), 2018, 51(5): 1-42.
Doshi-Velez F, Kim B. Towards a rigorous science of interpretable machine learning[J]. arXiv preprint arXiv:1702.08608, 2017.
Gilpin L H, Bau D, Yuan B Z, et al. Explaining explanations: An overview of interpretability of machine learning[C]//2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 2018: 80-89.
Lipton Z C. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery[J]. Queue, 2018, 16(3): 31-57.Journal of Technology Innovation and Engineering (JTIE) ISSN 3058-9584 Vol.1 No.1 January 2025
Scott M, Su-In L. A unified approach to interpreting model predictions[J]. Advances in neural information processing systems, 2017, 30: 4765-4774.
Ancona M, Ceolini E, Öztireli C, et al. Towards better understanding of gradient-based attribution methods for deep neural networks[J]. arXiv preprint arXiv:1711.06104, 2017.
Hardt M, Price E, Srebro N. Equality of opportunity in supervised learning[J]. Advances in neural information processing systems, 2016, 29.
Boyd D, Crawford K. Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon[J]. Information, communication & society, 2012, 15(5): 662-679.
Datta A, Sen S, Zick Y. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems[C]//2016 IEEE symposium on security and privacy (SP). IEEE, 2016: 598-617.
Leo Breiman. Random forests. Machine learning, 45:5–32, 2001.
Thomas G Dietterich et al. Ensemble learning. The handbook of brain theory and neural networks, 2(1):110–125, 2002.
Ken Perlin. An image synthesizer.ACM Siggraph Computer Graphics, 19(3):287– 296, 1985
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
Eric Brochu, Vlad M Cora, and Nando De Freitas. Atutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794, 2016.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017.
Fischer Black. Noise. The journal of finance, 41(3):528–543, 1986.
Steven Worley. A cellular texture basis function. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 291–294, 1996
Copyright
No license provided.
