[Paper Review] On Course, But Not There Yet: Enterprise Architecture Conformance and Benefits in Systems Development

Ralph Foorthuis, Marlies van Steenbergen+4 more|arXiv (Cornell University)|Aug 23, 2020

Information Technology Governance and Strategy30 references45 citations

TL;DR

Empirically tests EA conformance and benefits through a survey (n=293), identifying key techniques that boost conformance, and showing that many benefits are not fully realized yet.

ABSTRACT

Various claims have been made regarding the benefits that Enterprise Architecture (EA) delivers for both individual systems development projects and the organization as a whole. This paper presents the statistical findings of a survey study (n=293) carried out to empirically test these claims. First, we investigated which techniques are used in practice to stimulate conformance to EA. Secondly, we studied which benefits are actually gained. Thirdly, we verified whether EA creators (e.g. enterprise architects) and EA users (e.g. project members) differ in their perceptions regarding EA. Finally, we investigated which of the applied techniques most effectively increase project conformance to and effectiveness of EA. A multivariate regression analysis demonstrates that three techniques have a major impact on conformance: carrying out compliance assessments, management propagation of EA and providing assistance to projects. Although project conformance plays a central role in reaping various benefits at both the organizational and the project level, it is shown that a number of important benefits have not yet been fully achieved.

Motivation & Objective

• Assess which practices are used to stimulate conformance to Enterprise Architecture (EA) in real projects.
• Evaluate which benefits are actually realized from EA conformance at both project and organizational levels.
• Compare perceptions of EA creators (architects) versus EA users (project members).
• Identify which applied techniques most effectively increase project conformance to and effectiveness of EA.

Proposed method

• Conducted a survey (n=293) across projects to test claims about EA benefits and conformance.
• Measured techniques used to stimulate conformance and correlated them with conformance outcomes.
• Used multivariate regression analysis to assess the impact of techniques on conformance and benefits.
• Compared perceptions between EA creators and EA users.
• Reported which techniques most strongly predict higher conformance and perceived benefits.

Experimental results

Research questions

• RQ1 Which techniques are actually used in practice to stimulate conformance to EA?
• RQ2 Which benefits are realized at organizational and project levels when conformance is achieved?
• RQ3 Do EA creators and EA users differ in their perceptions of EA?
• RQ4 Which applied techniques most effectively increase project conformance to and the effectiveness of EA?

Key findings

• Three techniques have a major impact on conformance: carrying out compliance assessments, management propagation of EA, and providing assistance to projects.
• Project conformance is central to reaping benefits at both organizational and project levels, but several benefits have not been fully achieved yet.
• There's a gap between perceived and realized benefits despite conformance efforts.
• Perceptions of EA effectiveness vary between architects and project members.
• Some benefits remain under-realized even with conformance activities in place.