What You Can Do With CheXthought

CheXthought is a diverse resource. Below are some potential research directions you can explore (not an exhaustive list) for ways you can use our data!

Vision & reasoning

Computer Vision & Reasoning Models

Fine-tune vision language models on expert chain-of-thought instead of final reports alone
Link each finding in the text to the image region it describes
Build visually grounded VQA from expert interpretations
Score intermediate reasoning steps, not only the final diagnosis

Report generation

Radiology Report Generation

Use expert reasoning as an intermediate step between the image and the report
Distill radiologist visual search into compact evidence for the report model
Control report sensitivity, such as thresholds for what gets reported
Train with preference or self-editing using expert reasoning as the reference

Agentic AI

Build agents that zoom, crop and inspect regions before answering
Give agents tools such as retrieval, measurement and segmentation
Retrieval-grounded agents with auditable reasoning steps
Self-correction of unsupported findings

Clinical decisions

Sequential Clinical Decision-Making, Confidence & Calibration

Learn when to gather more evidence and when to commit to a finding
Study how the order of inspection affects the final diagnosis
Check whether model confidence tracks expert disagreement
Compare the confidence of model and expert in the same finding

Uncertainty

Separate ambiguity in the image from uncertainty in the model
Identify images with more than one reasonable reading
Predict a distribution over findings instead of a single label
Conformal prediction sets for findings

Evaluation

Dataset & Benchmark Evaluation

Evaluate intermediate reasoning, not only final answers
Benchmark against multiple readers rather than a single label
Validate LLM-as-judge scores against expert ratings
Measure how much reasoning data a model needs

Failure modes

Model Failure Modes

Catalog misses, false positives and mislocalizations
Find cases where models are right for the wrong reasons
Compare model errors with the errors experts make
Test robustness across image quality and patient subgroups

Visual attention

Visual Attention: Human vs. Model

Compare model attention maps with expert cursor traces
Find regions experts check that models ignore
Compare the search order of models and experts
Test whether attention overlap predicts correct answers

Segmentation

Segmentation & Localization

Use cursor traces as weak supervision for lesion segmentation
Instruction-guided segmentation from expert finding statements
Localize findings without pixel-level masks
Compare predicted regions with the regions experts inspected

Reinforcement learning

Reinforcement Learning & Human Feedback

Design rewards for localization, accuracy and consistency
Learn a visual search policy from expert cursor trajectories
Train reward models on expert reasoning statements
Preference learning between competing interpretations

Interpretability

Explainable AI & Faithfulness

Test whether model explanations reflect the evidence in the image
Chain-of-thought faithfulness: does the stated reasoning reflect how the model reached its answer?
Counterfactual tests: does the prediction change when the evidence changes?
Attribute individual claims to specific image regions

AI safety

Hallucination, Safety & Chain-of-Thought Monitoring

Detect and reduce hallucinated findings in generated reports
Train monitors on expert reasoning to flag unsupported steps
Check each reasoning step against where experts looked
Detect shortcut learning from spurious image features

Human-AI

Human-AI Alignment & Trust

Map where model and expert interpretations differ
Teach models when to defer to a clinician
Study how explanations affect clinician trust and reliance

Longitudinal

Temporal Reasoning & Longitudinal Modeling

Detect clinically meaningful change from a prior study
Classify findings as improving, worsening or stable
Generate grounded descriptions of interval change
Measure consistency of interpretations across exams

Clinical NLP

Clinical NLP & Information Extraction

Study how radiologists express uncertainty in language
Extract findings, anatomy and their relations from free text
Classify statements as supported, contradicted or unresolved
Sentence-level image-text learning for zero-shot classification