microsoft

/

Phi-3.5-vision-instruct

like 553

Follow

Microsoft 4,759

Issue Summary

I'm encountering a TypeError when trying to process a batch of data using the processor in my code. The error occurs when I attempt to process both text and image data together in a batch.

Code Snippet

texts = []
images = []
for example in examples:
    prompt = "my prompt"
    placeholder = f"<|image_{1}|>\n"
    messages = [
        {"role": "user", "content": placeholder + prompt},
    ]

    text = processor.tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=False
    )
    texts.append(text)
    images.append(example["image"])

batch = processor(texts, images, return_tensors="pt", padding=True)

return _compile(pattern, flags).split(string, maxsplit)
TypeError: expected string or bytes-like object

Detailed Description

The error occurs when trying to create a batch using the processor by passing a list of texts and images. The apply_chat_template function seems to return a result that causes the processor to raise a TypeError related to expecting a string or bytes-like object. It appears the issue is triggered when handling the batch processing of both text and image data together.

Expected Behavior

The processor should correctly handle the provided text and image inputs and return the appropriate tensors without raising an error.

Environment

Python Version: 3.9
OS: ubuntu