Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology

Paper · Source
Domain Specialization in LLMs

Source: Agarwal, Moehring, Rajpurkar, Salz, NBER w31422 · 2026-08

Full automation using Artificial Intelligence (AI) predictions may not be optimal if humans have information not available to the AI (contextual information). We study human-AI collaboration using an information experiment with professional radiologists. Results show that providing (i) AI predictions does not improve performance on average, whereas (ii) contextual information does. Radiologists do not realize the gains from AI assistance because of errors in belief updating – they underweight AI predictions and treat their own information and AI predictions as statistically independent. Unless these mistakes can be corrected, the optimal human-AI collaboration design delegates cases either to humans or to AI, but rarely to AI assisted humans.

Synthesis notes that discuss concepts related to this paper 8

Notes in this collection most semantically related to the paper, ranked by cosine similarity.

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

How do clinicians assess, trust, and effectively use medical AI in practice? How does increased AI involvement reshape human agency and accountability? Which task types benefit from human-AI collaboration and how should work divide? How do quality and diversity in synthetic data interact and compete? Do confidence signals reliably indicate whether AI models will succeed or fail? What happens to the meaning and authority of expertise when AI generates knowledge?