借助 NVIDIA Clara Reason 推进放射学研究中的可解释 AI

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image


# Load the Clara Reason model 
model_name = "nvidia/NV-Reason-CXR-3B"
model = AutoModelForImageTextToText.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
).eval().to("cuda")


processor = AutoProcessor.from_pretrained(model_name)

# Load chest x-ray image
image = Image.open("chest_xray.png")

# Prepare input with clinical context
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": image,
            },
            {
                "type": "text",
                "text": "Find abnormalities and support devices."
            }
        ]
    }
]

# Create prompt using chat template
text = processor.apply_chat_template(
    messages,
    add_generation_prompt=True
)

# Process inputs
inputs = processor(
    text=text,
    images=[image],
    return_tensors="pt"
).to(model.device)

# Generate reasoning with chain-of-thought
generated_ids = model.generate(
    **inputs,
    max_new_tokens=2048
)

# Trim the generated ids to remove the input ids
trimmed_generated_ids = [
    out_ids[len(in_ids):]
    for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]


# Decode the response
generated_text = processor.batch_decode(
    trimmed_generated_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)[0]


print("Output:")
print(generated_text)

THINK: We'll begin with the quality assessment of this AP chest x-ray. As you can see, the lung fields are relatively well visualized, although the patient's position and the fact that it's a portable AP view may limit the evaluation to some extent. I think the quality is acceptable for this type of examination.

Now, we are looking at the medical devices and instrumentation. I don't see any medical devices such as central venous catheters, endotracheal tubes, or chest tubes. However, I do notice that the patient is likely in a clinical setting, given the portable nature of the examination.

The central airways appear patent. The trachea is visible and not deviated, and the right and left main bronchi appear unremarkable. I don't see any foreign objects or abnormalities in the central airways.

Moving on to the right lung, we are evaluating the upper lobe first. The aeration of the right upper lobe appears reduced, which could be suggestive of some volume loss or atelectasis. I don't see any distinct masses or nodules, but there are some subtle opacities that could represent infiltrates or edema.

Next, we will evaluate the right middle and lower lobes. The lung volumes appear low, which could be indicative of some atelectasis or volume loss. I see some patchy opacities in the right lower lobe, which may represent pulmonary edema or infiltrates. The costophrenic angle is slightly blunted, which could be significant for a small pleural effusion.

Now, let's examine the left lung. The left upper lobe appears similar to the right, with reduced aeration and some subtle opacities that could represent infiltrates or edema. The left lower lobe also shows some patchy opacities and a slightly blunted costophrenic angle, suggestive of a small pleural effusion.

We are looking at the pleura now, and as I mentioned before, the costophrenic angles are slightly blunted bilaterally, which is concerning for small bilateral pleural effusions. I don't see any evidence of pneumothorax.

The mediastinum appears enlarged, which could be indicative of cardiomegaly. The cardiac silhouette is prominent, and I think this finding may be meaningful for moderate cardiomegaly.

The hila appear unremarkable, without any significant adenopathy or masses. The lung vasculature appears prominent, which could be suggestive of pulmonary edema.

The heart, as I mentioned before, appears enlarged, with a cardiothoracic ratio that is elevated. This finding is concerning for cardiomegaly.

Now, we are looking at the upper abdomen, and I see some normal-appearing bowel gas patterns. There is no free air under the diaphragm, which is important to note.

Finally, we are looking at the bony structures. The clavicles, scapulae, and ribs appear unremarkable, without any focal lesions or fractures. The vertebral bodies are well visualized and appear normal.

In summary, the most significant findings in this patient are the presence of mild pulmonary edema, small bilateral pleural effusions, and moderate cardiomegaly. These findings could represent a range of clinical conditions, including heart failure or other cardiopulmonary diseases. 

ANSWER: Cardiomegaly, Edema, Enlarged Cardiomediastinum, Lung Opacity, Pleural Effusion