AI-in-the-human-loop: an operational model for mature human-AI collaboration

Abstract Artificial intelligence (AI) and large language models are powerful tools but come with risks and caveats for proper usage. In this work, we are concerned with effective human-AI collaboration. As in any team, it is key that all team members properly understand the problem at hand, the environment, and each other. In order to scaffold a mature human-AI collaboration, we propose a process framework based on continual, proactive, bidirectional assessment, fostering co-evolution of the human-AI team. While most research has focused on assessing the strengths and limitations of AI agents, we especially focus on the risks associated with a lack of maturity on the human side. The core idea is to have AI agents help humans assess themselves and to anticipate interaction issues. Furthermore, to place the proposal within a broader vision, we advance two novel concepts: (i) “AI-in-the-human-loop”, the idea of AI agents observing and acting within human reasoning and activity, and (ii) “human-under-test”, a software testing analogy that suggests techniques for assessing human maturity by AI agents. A case study of a human-AI team involved in a statistical analysis project is considered, and analysed through an LLM-as-a-judge setup. Preliminary results highlight collaboration maturity issues and the possibility of proactive corrections, showcasing limitations in current collaborative workflows and opportunities for further research.