ALPaCA: Adapting Llama for Pathology Context Analysis to enable slide-level question answering
Abstract Large Vision Language Models (LVLMs) are increasingly used in computational pathology for image classification, description generation, question answering and interactive diagnostics. However, most pathology LVLMs analyse small regions of interest rather than pyramidal, gigapixel-scale whole-slide images (WSIs), limiting their use for tasks requiring whole-slide assessment across sub-regions and magnification levels. Here, we present ALPaCA (Adapting Llama for Pathology Context Analysis), a slide-level LVLM framework for WSI question answering across diverse cancer types and tissue sites. ALPaCA is trained using 35,913 WSIs with curated descriptions and 341,051 question-answer pairs from TCGA and GTEx. It combines a LongFormer vision-text adaptor with a Gaussian mixture model-based prototyping adaptor and Llama3.1. ALPaCA exceeds 90% accuracy on internal close-ended benchmarks and maintains 77–82% accuracy on independent external cohorts. Expert pathologist evaluation of open-ended responses supports its slide-level reasoning capability. Additionally, ALPaCA can be fine-tuned on organ- or disease-specific datasets, supporting specialised pathology question answering.
Authors
- Mercedes Jimenez‐Liñan (ORCID: https://orcid.org/0000-0002-8164-5462)
- Mireia Crispin‐Ortuzar (ORCID: https://orcid.org/0000-0002-4351-3709)
- Brian Rous (ORCID: https://orcid.org/0000-0002-7619-461X)
- Shangqi Gao (ORCID: https://orcid.org/0000-0003-4567-1636)
- Mengling Feng (ORCID: https://orcid.org/0000-0002-5338-6248)
- Tieliang Gong (ORCID: https://orcid.org/0000-0002-3840-441X)
- Zeyu Gao (ORCID: https://orcid.org/0000-0003-2365-8318)
- Chunbao Wang (ORCID: https://orcid.org/0000-0002-8795-0142)
- Chengzu Li (ORCID: https://orcid.org/0000-0002-1879-7391)
- Faisal Mahmood (ORCID: https://orcid.org/0000-0001-7587-1562)
- William McGough
- Weiheng Su
- Di Zhang (ORCID: https://orcid.org/0000-0002-7921-6781)
- Chen Li (ORCID: https://orcid.org/0000-0002-0079-3106)
- Kai He (ORCID: https://orcid.org/0000-0003-2639-1532)
- Inês Machado (ORCID: https://orcid.org/0000-0003-4160-9624)
- Xiaobo Pang
Institutions
- Harvard University (US)
- National University of Singapore (SG)
- University of Cambridge (GB)
- Cambridge University Hospitals NHS Foundation Trust (GB)
- First Affiliated Hospital of Xi'an Jiaotong University (CN)
- National University Health System (SG)
- Xi'an Jiaotong University (CN)
Publication Details
- Journal
- Nature Communications
- Published
- 2026-08-20
- DOI
- https://doi.org/10.1038/s41467-026-76372-z
- Citations
- 3
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 14.20
Funders
- UK Research and Innovation
- Cancer Research UK
- National Institute for Health and Care Research
- Department of Health and Social Care
- University of Warwick
- University of Birmingham
- National University of Singapore
- Engineering and Physical Sciences Research Council
- NIHR Cambridge Biomedical Research Centre