A Novel Approach to Automated Behavioral Diagram Assessment using Label Similarity and Subgraph Edit Distance

Reza Fauzan; Daniel Oranova Siahaan; Siti Rochimah; Evi Triandini

doi:10.7494/csci.2021.22.2.3868

Authors

Reza Fauzan Institut Teknologi Sepuluh Nopember https://orcid.org/0000-0002-4113-9848
Daniel Oranova Siahaan Institut Teknologi Sepuluh Nopember https://orcid.org/0000-0001-6560-2975
Siti Rochimah Institut Teknologi Sepuluh Nopember
Evi Triandini ITB STIKOM Bali https://orcid.org/0000-0001-7073-9829

DOI:

https://doi.org/10.7494/csci.2021.22.2.3868

Keywords:

automated assessment, behavioral diagram, label similarity, similarity assessment, subgraph edit distance, unified modeling language

Abstract

Unified Modelling Language (UML) is one of the standard languages used in modelling software. Therefore, UML is widely taught in many universities. Generally, teachers assign students to build UML diagram designs based on a predetermined project. However, the assessment of such assignments can be challenging and teachers may be inconsistent in assessing students’ answers. Thus, automated UML diagram assessment becomes essential to maintaining assessment consistency. This study uses a behavioral diagram as the object of research since it is a commonly taught UML diagram. The behavioral diagram can show a dynamic view of the software. This study proposes a new approach to automatically assessing the similarity of behaviour diagrams as reliably as experts. We divide the assessment into two portions: semantic assessment and structural assessment. Label similarity is used to calculate semantic assessment, while subgraph edit distance is used to calculate structural assessment. The results suggest that the proposed approach is as reliable as an expert in assessing the similarity between two behaviour diagrams. The observed agreement value suggests strong agreement between the use of experts and the proposed approach.

Downloads

Download data is not yet available.

References

Adamu A., Zainon W.: A review of UML model retrieval approaches. In: Indian Journal of Science and Technology, vol. 9(46), pp. 1-8, 2016.

Adamu A., Zainon W.M.N.W.: Matching and retrieval of state machine diagrams from software repositories using Cuckoo Search Algorithm. In: 2017 8th International Conference on Information Technology (ICIT), pp. 187-192. IEEE, 2017.

Adamu A., Zainon W.M.N.W.: Multiview Similarity Assessment Technique of UML Diagrams. In: Procedia Computer Science, vol. 124, pp. 311-318, 2017.

Adamu A., Zainon W.M.N.W.: Similarity Assessment of UML Sequence Diagrams Using Dynamic Programming. In: International Visual Informatics Conference, pp. 270-278. Springer, 2017.

Bloxham S., den Outer B., Hudson J., Price M.: Lets stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria. In: Assessment & Evaluation in Higher Education, vol. 41(3), pp. 466-481, 2016.

Buijs S., Heerkens J., Ampe B., Delezie E., Rodenburg T., Tuyttens F.: Assessing keel bone damage in laying hens by palpation: eects of assessor experience on accuracy, inter-rater agreement and intra-rater consistency. In: Poultry science, vol. 98(2), pp. 514-521, 2019.

Castro L.J.G., Berlanga R., Garcia A.: In the pursuit of a semantic similarity metric based on UMLS annotations for articles in PubMed Central Open Access. In: Journal of Biomedical Informatics, vol. 57, pp. 204-218, 2015.

Chonoles M.J.: OCUP 2 Certication Guide: Preparing for the OMG Certied UML 2.5 Professional 2 Foundation Exam. Morgan Kaufmann, 2017.

Daller E., Bougleux S., Gauzere B., Brun L.: Approximate graph edit distance by several local searches in parallel. In: 7th International Conference on Pattern Recognition Applications and Methods. 2018.

Fauzan R., Siahaan D., Rochimah S., Triandini E.: Class Diagram Similarity Measurement: A Dierent Approach. In: 2018 3rd International Conference on Information Technology, Information System and Electrical Engineering (ICITISEE), pp. 215-219. IEEE, 2018.

Fellbaum C.: WordNet. In: Theory and applications of ontology: computer applications, pp. 231-243. Springer, 2010.

Feng Y., Bagheri E., Ensan F., Jovanovic J.: The state of the art in semantic relatedness: a framework for comparison. In: The Knowledge Engineering Review, vol. 32, 2017. 2020/06/20;

Fischer A., Riesen K., Bunke H.: Improved quadratic time approximation of graph edit distance by combining Hausdor matching and greedy assignment. In: Pattern Recognition Letters, vol. 87, pp. 55-62, 2017.

Gwet K.: Kappa statistic is not satisfactory for assessing the extent of agreement between raters. Series: Statistical Methods for Inter-Rater Reliability Assessment 1 (1): 1-5. In: Gaithersburg: STATAXIS Consulting, vol. 4, 2002.

Harispe S., Ranwez S., Janaqi S., Montmain J.: Semantic similarity from natural language and ontology analysis. In: Synthesis Lectures on Human Language Technologies, vol. 8(1), pp. 1-254, 2015.

Jenkins D., Simpson S., Peacock A.: Investigating the consistency and quality of EPC ratings and assessments. In: Energy, vol. 138, pp. 480-489, 2017.

Jimenez A.M., Zepeda S.J.: A Comparison of Gwets AC1 and kappa when calculating inter-rater reliability coecients in a teacher evaluation context. In: Journal of Education Human Resources, p. e20190001, 2020.

Kutuzov A., Dorgham M., Oliynyk O., Biemann C., Panchenko A.: Learning Graph Embeddings from WordNet-based Similarity Measures. In: arXiv preprint arXiv:1808.05611, 2018.

Landis J.R., Koch G.G.: The measurement of observer agreement for categorical data. In: biometrics, pp. 159-174, 1977.

Majumder G., Pakray P., Gelbukh A., Pinto D.: Semantic textual similarity methods, tools, and applications: A survey. In: Computacion y Sistemas, vol. 20(4), pp. 647-665, 2016.

Park W.J., Bae D.H.: A two-stage framework for UML specication matching. In: Information and Software Technology, vol. 53(3), pp. 230-244, 2011.

Pressman R.S.: Software engineering: a practitioner's approach. Palgrave macmillan, 2005.

Riesen K., Bunke H.: GRAPH EDIT DISTANCENOVEL APPROXIMATION ALGORITHMS. In: Handbook of Pattern Recognition and Computer Vision, pp. 275-291. World Scientic, 2016.

Riesen K., Ferrer M., Bunke H.: Approximate graph edit distance in quadratic time. In: IEEE/ACM transactions on computational biology and bioinformatics, vol. 17(2), pp. 483-494, 2015.

Riesen K., Ferrer M., Dornberger R., Bunke H.: Greedy graph edit distance. In: International Workshop on Machine Learning and Data Mining in Pattern Recognition, pp. 3-16. Springer, 2015.

Robinson W.N., Woo H.G.: Finding reusable UML sequence diagrams automatically. In: IEEE software, vol. 21(5), pp. 60-67, 2004.

Salami H.O., Ahmed M.: Retrieving sequence diagrams using genetic algorithm. In: 2014 11th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 324-330. IEEE, 2014.

Siahaan D., Desnelita Y., et al.: Structural and semantic similarity measurement of UML sequence diagrams. In: 2017 11th International Conference on Information & Communication Technology and System (ICTS), pp. 227-234. IEEE, 2017.

Sommerville I.: Software engineering 9th Edition. In: ISBN-10, vol. 137035152, p. 18, 2011.

Triandini E., Fauzan R., Siahaan D.O., Rochimah S.: Sequence Diagram Similarity Measurement: A Dierent Approach. In: 2019 16th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 348-351. IEEE, 2019.

Wongpakaran N., Wongpakaran T., Wedding D., Gwet K.L.: A comparison of Cohens Kappa and Gwets AC1 when calculating inter-rater reliability coecients: a study conducted with personality disorder samples. In: BMC medical research methodology, vol. 13(1), p. 61, 2013.

Yuan Z., Yan L., Ma Z.: Structural similarity measure between UML class diagrams based on UCG. In: Requirements Engineering, pp. 1-17, 2019.