A Domain-Specific Language for Monitoring ML Model Performance

TY - GEN

T1 - A Domain-Specific Language for Monitoring ML Model Performance

AU - Kourouklidis, Panagiotis

AU - Kolovos, Dimitris

AU - Noppen, Joost

AU - Matragkas, Nicholas

N1 - Funding Information: The work presented in this paper has been partially supported by the Lowcomote project, which received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curiegrant agreement No. 813884. Publisher Copyright: © 2023 IEEE. This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.

PY - 2023/12/22

Y1 - 2023/12/22

N2 - As machine learning (ML) starts to offer competitive advantages for an increasing number of application domains, many organisations invest in developing ML-enabled products. The development of these products poses unique challenges compared to traditional software engineering projects and requires the collaboration of people from different disciplines. This work focuses on alleviating some of these challenges related to implementing monitoring systems for deployed ML models. To this end, a domain-specific language (DSL) is developed that data scientists can use to declaratively define monitoring workflows. Complementary to the DSL, a runtime component is developed that implements the specified behaviour. This component is designed to be easily integrated with the rest of an organisation's ML platform and extended by software engineers that do not necessarily have experience with model-driven engineering. An evaluation of the proposed system that supports the validity of the approach is also presented.

AB - As machine learning (ML) starts to offer competitive advantages for an increasing number of application domains, many organisations invest in developing ML-enabled products. The development of these products poses unique challenges compared to traditional software engineering projects and requires the collaboration of people from different disciplines. This work focuses on alleviating some of these challenges related to implementing monitoring systems for deployed ML models. To this end, a domain-specific language (DSL) is developed that data scientists can use to declaratively define monitoring workflows. Complementary to the DSL, a runtime component is developed that implements the specified behaviour. This component is designed to be easily integrated with the rest of an organisation's ML platform and extended by software engineers that do not necessarily have experience with model-driven engineering. An evaluation of the proposed system that supports the validity of the approach is also presented.

KW - Dataset Shift

KW - Machine Learning

KW - MLOps

KW - Model-Driven Engineering

UR - http://www.scopus.com/inward/record.url?scp=85182388899&partnerID=8YFLogxK

U2 - 10.1109/MODELS-C59198.2023.00056

DO - 10.1109/MODELS-C59198.2023.00056

M3 - Conference contribution

AN - SCOPUS:85182388899

T3 - Proceedings - 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2023

SP - 266

EP - 275

BT - Proceedings - 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion, MODELS-C 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, MODELS-C 2023

Y2 - 1 October 2023 through 6 October 2023

ER -