[논문 리뷰] Visioning Human-Agentic AI Teaming: Continuity, Tension, and Future Research

Bowen Lou, Tian Lu외 2명|arXiv (Cornell University)|2026. 03. 05.

Embodied and Extended Cognition인용 수 2

한 줄 요약

본 논문은 Team Situation Awareness (Team SA)를 개방형 에이전트 AI로 확장하여 연속성 및 긴장이 장기적 정렬에 미치는 영향을 살펴보고 인간–에이전트 AI teaming을 위한 연구 로드맵을 제안한다.

ABSTRACT

Artificial intelligence is undergoing a structural transformation marked by the rise of agentic systems capable of open-ended action trajectories, generative representations and outputs, and evolving objectives. These properties introduce structural uncertainty into human-AI teaming (HAT), including uncertainty about behavior trajectories, epistemic grounding, and the stability of governing logics over time. Under such conditions, alignment cannot be secured through agreement on bounded outputs; it must be continuously sustained as plans unfold and priorities shift. We advance Team Situation Awareness (Team SA) theory, grounded in shared perception, comprehension, and projection, as an integrative anchor for this transition. While Team SA remains analytically foundational, its stabilizing logic presumes that shared awareness, once achieved, will support coordinated action through iterative updating. Agentic AI challenges this presumption. Our argument unfolds in two stages: first, we extend Team SA to reconceptualize both human and AI awareness under open-ended agency, including the sensemaking of projection congruence across heterogeneous systems. Second, we interrogate whether the dynamic processes traditionally assumed to stabilize teaming in relational interaction, cognitive learning, and coordination and control continue to function under adaptive autonomy. By distinguishing continuity from tension, we clarify where foundational insights hold and where structural uncertainty introduces strain, and articulate a forward-looking research agenda for HAT. The central challenge of HAT is not whether humans and AI can agree in the moment, but whether they can remain aligned as futures are continuously generated, revised, enacted, and governed over time.

연구 동기 및 목표

• Define how agentic AI introduces open-ended action, representation, and objective evolution that destabilizes traditional HAT alignment.
• Extend Team SA to reconceptualize human and AI awareness under open-ended agency.
• Assess whether dynamic processes (relational interaction, learning, coordination) stabilize or destabilize teaming with adaptive autonomy.
• Distinguish continuity versus tension to map where existing insights hold and where new questions arise.
• Propose a structured research agenda for future HAT research under agentic AI.

제안 방법

• Theoretically integrate Team SA with open-ended agency to reinterpret Level 1–3 perception, comprehension, and projection for both humans and AI.
• Map how evaluative attitude theories, relational interaction, cognitive learning, explanatory guidance, collective coordination, and operational control align with Team SA levels in HAT.
• Introduce projection congruence as a key metric for cross-system alignment between human and AI expectations over time.
• Develop a continuity–tension framework to identify stable versus destabilizing dynamics in open-ended HAT.
• Offer RQs (RQ1.1–RQ1.4) and first-order questions to guide empirical work on trajectory interpretation, representation coherence, and projection congruence.

실험 결과

연구 질문

• RQ1 RQ1.1 What indicators capture how humans interpret AI-initiated trajectory shifts and implicit commitments?
• RQ2 RQ1.2 How can the coherence and stability of human-constructed task representations be assessed across unfolding intermediate states?
• RQ3 RQ1.3 How can projection congruence be evaluated across branching futures and shifting objective priorities?
• RQ4 RQ1.4 How do task characteristics influence the robustness of these evaluative processes?

주요 결과

• Open-ended agency can invert typical Team SA benefits, causing relational legitimacy to hinge on epistemic fragility and iterative updating to amplify divergence.
• Projection congruence becomes central as humans and AI anticipate futures with shifting objectives and governance priorities.
• AI awareness (perception, comprehension, projection) must be made observable and interoperable to assess cross-system alignment under open-ended agency.
• Continuity persists in static, lower Team SA levels, but tension arises in dynamic layers where trajectories and governing regimes evolve.
• The paper outlines a research agenda to test how stability or instability emerges in relational interaction, learning, and coordination under agentic autonomy.