multi task
Hello. How can we ask several questions in one prompt? For example: What is document name , total price , number of bank account and bank name ?
The model has only one answer to this question.
This is a great point! Because of the data the model was trained on, it makes sense that it won't generalize like that to many questions. What you can do is:
- Create datasets with the type of multiple questions that you want and fine-tune the model on them.
- Do some engineering with this model to split multiple questions and get multiple answers and re-stich them before presenting them to the user.
In any case, I would start from this model which was fine tuned for DocVQA:
https://huggingface.co/HuggingFaceM4/Florence-2-DocVQA
This is a great point! Because of the data the model was trained on, it makes sense that it won't generalize like that to many questions. What you can do is:
- Create datasets with the type of multiple questions that you want and fine-tune the model on them.
- Do some engineering with this model to split multiple questions and get multiple answers and re-stich them before presenting them to the user.
In any case, I would start from this model which was fine tuned for DocVQA:
https://huggingface.co/HuggingFaceM4/Florence-2-DocVQA
Thank you very much for your detailed answer.