skip to main content
research-article

Measuring Performance in Knowledge-intensive Processes

Published:06 February 2019Publication History
Skip Abstract Section

Abstract

Knowledge-intensive Processes (KIPs) are processes whose execution is heavily dependent on knowledge workers performing various interconnected knowledge-intensive decision-making tasks. Among other characteristics, KIPs are usually non-repeatable, collaboration-oriented, unpredictable, and, in many cases, driven by implicit knowledge, derived from the capabilities and previous experiences of participants. Despite the growing body of research focused on understanding KIPs and on proposing systems to support these KIPs, the research question on how to define performance measures thereon remains open. In this article, we address this issue with a proposal to enable the performance management of KIPs. Our approach comprises an ontology that allows us to define process performance indicators (PPIs) in the context of KIPs, and a methodology that builds on the ontology and the concepts of lead and lag indicators to provide process participants with actionable guidelines that help them conduct the KIP in a way that fulfills a set of performance goals. Both the ontology and the methodology have been applied to a case study of a real organization in Brazil to manage the performance of an Incident Troubleshooting Process within an ICT (Information and Communications Technology) Outsourcing Company.

Skip Supplemental Material Section

Supplemental Material

References

  1. Peter C. Brewer and Thomas W. Speh. 2000. Using the balance scorecard to measure supply chain performance. J. Bus. Logist. 21, 1 (2000), 75--93.Google ScholarGoogle Scholar
  2. Joos C. A. M. Buijs, Boudewijn F. van Dongen, and Wil M. P. van der Aalst. 2013. Mining Configurable Process Models from Collections of Event Logs. Springer, Berlin, 33--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Evellin C. S. Cardoso, Paulo Sérgio Santos Jr., João Paulo Andrade Almeida, Renata S. S. Guizzardi, and Giancarlo Guizzardi. 2010. Semantic integration of goal and business process modeling. In Proceedings of the International Conference on Research and Practical Issues Enterprise Information Systems (CONFENIS’10).Google ScholarGoogle Scholar
  4. Felix Tung Sun Chan. 2003. Performance measurement in a supply chain. Int. J.Adv. Manufact. Technol. 21, 7 (2003), 534--548.Google ScholarGoogle ScholarCross RefCross Ref
  5. Mu-Yen Chen, Mu-Jung Huang, and Yu-Chen Cheng. 2009. Measuring knowledge management performance using a competitive perspective: An empirical study. Expert Syst. Appl. 36, 4 (2009), 8449--8459. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Claire Costello and Owen Molloy. 2009. Building a process performance model for business activity monitoring. In Information Systems Development, Challenges in Practice, Theory, and Education, Vol. 1. Springer, 237--248.Google ScholarGoogle Scholar
  7. Francisco M. del-Rey-Chamorro, Rajkumar Roy, Bert van Wegen, and Andy Steele. 2003. A framework to create key performance indicators for knowledge management. J. Knowl. Manag. 7, 2 (2003), 46--62.Google ScholarGoogle ScholarCross RefCross Ref
  8. Adela del Río-Ortega, Félix García, Manuel Resinas, Elmar Weber, Francisco Ruiz, and Antonio Ruiz-Cortés. 2017a. Enriching decision making with data-based thresholds of process-related KPIs. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’17). 193--209.Google ScholarGoogle ScholarCross RefCross Ref
  9. Adela del Río-Ortega, Manuel Resinas, Cristina Cabanillas, and Antonio Ruiz-Cortés. 2013. On the definition and design-time analysis of process performance indicators. Info. Syst. 38, 4 (2013), 470--490. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Adela del Río-Ortega, Manuel Resinas, Amador Durán, Beatriz Bernárdez, Antonio Ruiz-Cortés, and Miguel Toro. 2017b. Visual PPINOT: A graphical notation for process performance indicators. Bus. Info. Syst. Eng. (2017).Google ScholarGoogle Scholar
  11. Adela del Río-Ortega, Manuel Resinas, and Antonio Ruiz-Cortés. 2010. Defining process performance indicators: An ontological approach. In Proceedings of the Confederated International Conferences: CoopIS, IS, DOA, and ODBASE (2010). 555--572.Google ScholarGoogle ScholarCross RefCross Ref
  12. Claudio Di Ciccio, Andrea Marrella, and Alessandro Russo. 2015. Knowledge-intensive processes: Characteristics, requirements and analysis of contemporary approaches. J. Data Semant. 4, 1 (2015), 29--57.Google ScholarGoogle ScholarCross RefCross Ref
  13. Marlon Dumas, Marcello La Rosa, Jan Mendling, and Hajo A. Reijers. 2013. Fundamentals of Business Process Management. Vol. 1. Springer-Verlag, Berlin. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. James D. Evans. 1996. Straightforward Statistics for the Behavioral Sciences. Brooks/Cole Publishing Company.Google ScholarGoogle Scholar
  15. Bernhard Ganter and Gerd Stumme. 2003. Creation and merging of ontology top-levels. In Conceptual Structures for Knowledge Creation and Communication. 131--145.Google ScholarGoogle Scholar
  16. Oscar González, Rubby Casallas, and Dirk Deridder. 2009. MMC-BPM: A domain-specific language for business processes analysis. In Proceedings of the Business Information Systems Conference (BIS’09), Vol. 21. 157--168.Google ScholarGoogle ScholarCross RefCross Ref
  17. Daniela Grigori, Fabio Casati, Malu Castellanos, Umeshwar Dayal, Mehmet Sayal, and Ming-Chien Shan. 2004. Business process intelligence. Comput. Ind. 53, 3 (2004), 321--343. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Giancarlo Guizzardi. 2005. Ontological foundations for structural conceptual models. Ph.D. Dissertation.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Giancarlo Guizzardi, Ricardo de Almeida Falbo, and Renata S. S. Guizzardi. 2008. Grounding software domain ontologies in the unified foundational ontology (UFO): The case of the ODE software process ontology. In Proceedings of the Workshop Requirements Engineering and Software Environments (IDEAS’08). 127--140.Google ScholarGoogle Scholar
  20. Kaisa Henttonen, Aino Kianto, and Paavo Ritala. 2016. Knowledge sharing and individual work performance: An empirical study of a public sector organisation. J. Knowl. Manag. 20, 4 (2016), 749--768.Google ScholarGoogle ScholarCross RefCross Ref
  21. Steffen Herbold, Jens Grabowski, and Stephan Waack. 2011. Calculation and optimization of thresholds for sets of software metrics. Empir. Softw. Eng. 16, 6 (2011), 812--841. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Richard Hull, Agnes Koschmider, Hajo A. Reijers, and William Wong. 2016. Fresh approaches to business process modeling (Dagstuhl seminar 16191). Dagstuhl Reports 6, 5 (2016), 1--30.Google ScholarGoogle Scholar
  23. Öykü Isik, Willem Mertens, and Joachim Van den Bergh. 2013. Practices of knowledge-intensive process management: Quantitative insights. Bus. Process Manag. J. 19, 3 (2013), 515--534.Google ScholarGoogle ScholarCross RefCross Ref
  24. Robert S. Kaplan and David P. Norton. 1992. The balanced scorecard: Measures that drive performance. Harvard Bus. Rev. 70, 1 (1992), 71--79.Google ScholarGoogle Scholar
  25. Birgit Korherr and Beate List. 2007. Extending the EPC and the BPMN with business process goals and performance measures. In Proceedings of the International Conference on Enterprise Information Systems (ICEIS’07). 287--294.Google ScholarGoogle Scholar
  26. Elfriede Krauth, Hans Moonen, Viara Popova, and Martijn C. Schut. 2005. Performance measurement and control in logistics service providing. In Proceedings of the International Conference on Enterprise Information Systems (ICEIS’05). 239--247.Google ScholarGoogle Scholar
  27. Peter Kueng. 2000. Process performance measurement system: A tool to support process-based organizations. Total Qual. Manag. 11, 1 (2000), 67--85.Google ScholarGoogle ScholarCross RefCross Ref
  28. Kun Chang Lee, Sangjae Lee, and In Won Kang. 2005. KMPI: Measuring knowledge management performance. Info. Manag. 42, 3 (2005), 469--482. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Richard Lenz and Manfred Reichert. 2007. IT support for healthcare processes—Premises, challenges, perspectives. Data Knowl. Eng. 61, 1 (2007), 39--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Todd A. Little and Amit V. Deokar. 2011. Role of knowledge asset indicators in understanding knowledge-intensive processes. In Proceedings of the Americas Conference on Information Systems (AMCIS’11).Google ScholarGoogle Scholar
  31. Todd A. Little and Amit V. Deokar. 2016. Understanding knowledge creation in the context of knowledge-intensive business processes. J. Knowl. Manag. 20, 5 (2016), 858--879.Google ScholarGoogle ScholarCross RefCross Ref
  32. Fabrizio M. Maggi, R. P. Jagadeesh Chandra Bose, and Wil M. P. van der Aalst. 2012. Efficient discovery of understandable declarative process models from event logs. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’12). 270--285. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Olivera Marjanovic and Ronald Freeze. 2011. Knowledge-intensive business processes: Theoretical foundations and research challenges. In Proceedings of the Hawaii International Conference on Systems Science. 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Anne P. Massey, Mitzi M. Montoya-Weiss, and Tony M. O’Driscoll. 2002. Performance-centered design of knowledge-intensive processes. J. Manag. Info. Syst. 18, 4 (2002), 37--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Chris McChesney, Sean Covey, and Jim Huling. 2012. The 4 Disciplines of Execution: Achieving Your Wildly Important Goals. Simon and Schuster.Google ScholarGoogle Scholar
  36. Christof Momm, Robert Malec, and Sebastian Abeck. 2007. Towards a model-driven development of monitored processes. In Proceedings of the Internationale Tagung Wirtschaftsinformatik (WI’07). 319--336.Google ScholarGoogle Scholar
  37. Ednilson Veloso Moura, Flávia Maria Santoro, and Fernanda Araujo Baião. 2015. XCuteKIP: Support for knowledge-intensive process activities. In Proceedings of the International Workshop on Groupware (CRIWG’15). 164--180.Google ScholarGoogle ScholarCross RefCross Ref
  38. Andy Neely, Huw Richards, John Mills, Ken Platts, and Mike Bourne. 1997. Designing performance measures: A structured approach. Int. J. Oper. Prod. Manag. 17, 11 (1997), 1131--1152.Google ScholarGoogle ScholarCross RefCross Ref
  39. Natalya F. Noy and Deborah L. McGuinness. 2001. Ontology Development 101: A Guide to Creating Your First Ontology. Technical Report.Google ScholarGoogle Scholar
  40. Object Management Group (OMG). 2014. Case Management Model and Notation, Version 1.0.Google ScholarGoogle Scholar
  41. Oluwafemi Oyemomi, Shaofeng Liu, Irina Neaga, and Ali Alkhuraiji. 2016. How knowledge sharing and business process contribute to organizational performance: Using the fsQCA approach. J. Bus. Res. 69, 11 (2016), 5222--5227.Google ScholarGoogle ScholarCross RefCross Ref
  42. Carlos Pedrinaci, Dave Lambert, Branimir Wetzstein, Tammo van Lessen, Luchesar Cekov, and Marin Dimitrov. 2008. SENTINEL: A semantic business process monitoring tool. In Proceedings of International Workshop on Ontology-supported Business Intelligence 2008. 26--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Viara Popova and Alexei Sharpanskykh. 2010. Modeling organizational performance indicators. Info. Syst. 35, 4 (2010), 505--527. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Alexander Rannacher, Robert Stranzenbach, Flavius Sturm, Susanne Mütze-Niewöhner, and Christopher M. Schlick. 2013. A system dynamics model for the evaluation of the productivity of knowledge-intensive services. iBusiness 5, 3B (2013), 55--58.Google ScholarGoogle Scholar
  45. Anand S. Rao and Michael P. Georgeff. 1995. BDI agents: From theory to practice. In Proceedings of International Conference on Multi-Agent Systems (ICMAS’95). 312--319.Google ScholarGoogle Scholar
  46. Pierre J. Richard, Timothy M. Devinney, George S. Yip, and Gerry Johnson. 2009. Measuring organizational performance: Towards methodological best practice. J. Manag. 35, 3 (2009), 718--804.Google ScholarGoogle ScholarCross RefCross Ref
  47. Cornelia Richter von Hagen, Dietmar Ratz, and Roman Povalej. 2005. Towards self-organizing knowledge-intensive processes. J. Univers. Knowl. Manag. 0, 2 (2005), 148--169.Google ScholarGoogle Scholar
  48. Ann Rosenberg, Greg Chase, Rukhshaan Omar, James Taylor, and Mark von Rosing. 2010. Applying Real-World BPM in an SAP Environment. Galileo Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Jorge Saldivar, Carla Vairetti, Carlos Rodríguez, Daniel Florian, Fabio Casati, and Rosa Alarcón. 2016. Analysis and improvement of business process models using spreadsheets. Info. Syst. 57 (2016), 1--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Juliana Baptista dos Santos França, Joanne Manhães Netto, Juliana do E. S.Carvalho, Flávia Maria Santoro, Fernanda Araujo Baião, and Mariano Pimentel. 2015. KIPO: The knowledge-intensive process ontology. Softw. Syst. Model. 14, 3 (2015), 1127--1157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Gerd Stumme and Alexander Maedche. 2001. FCA-MERGE: Bottom-up merging of ontologies. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’01). Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Flavius Sturm, Alexander Rannacher, Lesya Straub-Bilan, Robert Stranzenbach, and Susanne Mütze-Niewöhner. 2011. Productivity measurement of knowledge-intensive services--towards adaptable service performance evaluation systems. In Proceedings of International Conference of the European Association for Research on Services (RESER’08).Google ScholarGoogle Scholar
  53. Mari Carmen Suárez-Figueroa, Asunción Gómez-Pérez, and Mariano Fernández-López. 2012. The NeOn methodology for ontology engineering. In Proceedings of the Conference on Ontology Engineering in a Networked World. 9--34.Google ScholarGoogle ScholarCross RefCross Ref
  54. Ganesh Vaidyanathan. 2005. A framework for evaluating third-party logistics. Commun. ACM 48, 1 (2005), 89--94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Amy Van Looy and Aygun Shafagatova. 2016. Business process performance measurement: A structured literature review of indicators, measures and metrics. SpringerPlus 5, 1 (2016), 1797.Google ScholarGoogle ScholarCross RefCross Ref
  56. Zhining Wang, Pratyush Nidhi Sharma, and Jinwei Cao. 2016. From knowledge sharing to firm performance: A predictive model comparison. J. Bus. Res. 69, 10 (2016), 4650--4658.Google ScholarGoogle ScholarCross RefCross Ref
  57. Branimir Wetzstein, Zhilei Ma, and Frank Leymann. 2008. Towards measuring key performance indicators of semantic business processes. In Proceedings of the International Conference on Business Information Systems (BIS’08). 227--238.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Measuring Performance in Knowledge-intensive Processes

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Internet Technology
            ACM Transactions on Internet Technology  Volume 19, Issue 1
            Regular Papers, Special Issue on Service Management for IOT and Special Issue on Knowledge-Driven BPM
            February 2019
            321 pages
            ISSN:1533-5399
            EISSN:1557-6051
            DOI:10.1145/3283809
            • Editor:
            • Ling Liu
            Issue’s Table of Contents

            Copyright © 2019 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 6 February 2019
            • Revised: 1 October 2018
            • Accepted: 1 October 2018
            • Received: 1 December 2016
            Published in toit Volume 19, Issue 1

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!