[Paper Review] Measuring and improving information systems agility through the balanced scorecard approach

Yassine Rdiouat, Samir Bahsani+2 more|arXiv (Cornell University)|Sep 13, 2021

Collaboration in agile enterprises86 references4 citations

TL;DR

This paper proposes the IS Agility BSC framework, a balanced scorecard-based model that measures and improves information systems agility through four perspectives—business contribution, user orientation, operation excellence, and innovation and competitiveness—using 14 key success factors and 42 evaluation criteria to support agile organizational performance.

ABSTRACT

Facing an environment increasingly complex, uncertain and changing, even in crisis, organizations are driven to be agile in order to survive. Agility, at the core heart of business strategy, represents the ability to grow in a competitive environment of continuous and unpredictable changes with information systems perceived as one of its main enablers. In other words, to be agile, organizations must be able to rely on agile enterprise information systems/information technology (IT/IS). Since, the agility needs are not the same among stakeholders, the objective of this research is to develop a conceptual model for the achievement and assessment of IT/IS agility from balanced perspectives to support agile organizations. Several researches have indicated that the IT balanced scorecard (BSC) approach is an appropriate technique for evaluating IT performance. This paper provides a balanced-scorecard based framework to evaluate the IS agility through four perspectives: business contribution, user orientation, operation excellence and innovation and competitiveness. The proposed framework, called IS Agility BSC, propose a three layer structure for each of the four perspectives: mission, key success factors, and agility evaluation criteria. According to this conceptual model, enterprise information systems agility is measured according to 14 agility key success factors, over the four BSC Perspectives, using 42 agility evaluation criteria that are identified based on literature survey methodology. This paper explores agility in the broader context of the enterprise information systems. The findings will provide, for both researchers and practitioners, a practical approach for achieving and measuring IS agility performance to support organizations in attempt to become agile as a new condition of surviving in the new business world.

Motivation & Objective

• To address the growing need for organizations to become agile in response to complex, uncertain, and rapidly changing environments.
• To overcome the lack of a comprehensive, multi-perspective framework for assessing information systems agility.
• To develop a conceptual model that aligns IT/IS agility with strategic business goals using the balanced scorecard approach.
• To provide practitioners and researchers with a practical, structured method for evaluating and improving IS agility.
• To ensure stakeholder-specific agility needs are captured through a balanced, multi-dimensional assessment framework.

Proposed method

• Adapting the traditional balanced scorecard (BSC) framework to focus specifically on information systems agility.
• Structuring each of the four BSC perspectives (business contribution, user orientation, operation excellence, innovation and competitiveness) into three layers: mission, key success factors (KSFs), and evaluation criteria.
• Identifying 14 agility-specific key success factors (KSFs) derived from a literature review, distributed across the four perspectives.
• Defining 42 agility evaluation criteria—three per KSF—based on existing research and best practices to enable measurable assessment.
• Using a conceptual modeling approach to integrate strategic agility goals with operational IT performance metrics.
• Ensuring alignment between IS agility and organizational strategy through structured, perspective-based evaluation.

Experimental results

Research questions

• RQ1 How can information systems agility be systematically measured across multiple strategic perspectives?
• RQ2 What key success factors (KSFs) are most critical for enabling IS agility in complex organizational environments?
• RQ3 How can the balanced scorecard framework be adapted to assess agility in enterprise information systems?
• RQ4 What evaluation criteria are most effective for measuring agility performance across different dimensions of IS?
• RQ5 How can the proposed framework support organizations in improving their IS agility in practice?

Key findings

• The IS Agility BSC framework provides a comprehensive, four-perspective model for assessing IS agility, integrating strategic and operational dimensions.
• The framework identifies 14 key success factors (KSFs) that are critical for achieving IS agility, distributed across business contribution, user orientation, operation excellence, and innovation and competitiveness.
• A total of 42 specific evaluation criteria are defined—three per KSF—to enable measurable and actionable assessment of IS agility.
• The framework supports alignment between IT performance and organizational agility by linking IS goals to strategic business outcomes.
• The model offers a practical, research-informed tool for both practitioners and researchers to evaluate and enhance IS agility in real-world settings.
• The conceptual model demonstrates how balanced scorecard principles can be extended to address agility, a critical yet under-measured dimension in IS performance.