mattmdjaga

/

segformer_b2_clothes

like 336

what kind of code, the model card has the code below that shows how to use the model. Or are you looking for other code?

from transformers import SegformerImageProcessor, AutoModelForSemanticSegmentation
from PIL import Image
import requests
import matplotlib.pyplot as plt
import torch.nn as nn

processor = SegformerImageProcessor.from_pretrained("mattmdjaga/segformer_b2_clothes")
model = AutoModelForSemanticSegmentation.from_pretrained("mattmdjaga/segformer_b2_clothes")

url = "https://plus.unsplash.com/premium_photo-1673210886161-bfcc40f54d1f?ixlib=rb-4.0.3&ixid=MnwxMjA3fDB8MHxzZWFyY2h8MXx8cGVyc29uJTIwc3RhbmRpbmd8ZW58MHx8MHx8&w=1000&q=80"

image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(images=image, return_tensors="pt")

outputs = model(**inputs)
logits = outputs.logits.cpu()

upsampled_logits = nn.functional.interpolate(
    logits,
    size=image.size[::-1],
    mode="bilinear",
    align_corners=False,
)

pred_seg = upsampled_logits.argmax(dim=1)[0]
plt.imshow(pred_seg)

Thank you for your answer, I am a Python beginner, I want to ask, is there any complete and runnable code, just like your example, that directly outputs JSON

you can turn the pred_seg into a list then output that as a json with the code below if you use the code above to get "pred_seg":

import json
list_pred = pred_seg.tolist()
output = {"prediction": list_pred}
json.dump(output, open("seg_prediction.json","w"))

thank you for your work

So i think the score is always 1 because it's hard to get a score on segmentation, but the code below will do this for you using the starting code again:

id2label = model.config.id2label
pred_ids = pred_seg.unique()
output = []
for id in pred_ids:
    output.append({
        "score": 1,
        "label": id2label[id.item()],
        "mask": ""
    })

How to obtain the corresponding mask?

well that depends on what you want in your mask, I have no idea what is doing on the model card as that's done automatically.

I want the mask after segmentation. In your examples, you have already completely output the mask for the corresponding area

not sure what you mean, could you specify what's the output that you're looking for and the data type, the current pred_seg output is (H, W) where each pixel has a value 0-17 corresponding to the label in that pixel.

i don't know, it's done by the website not me

Okay, thank you. I thought these examples were implemented by you.