|
--- |
|
base_model: qresearch/doubutsu-2b-pt-756 |
|
library_name: peft |
|
license: apache-2.0 |
|
datasets: |
|
- google/docci |
|
language: |
|
- en |
|
pipeline_tag: image-text-to-text |
|
--- |
|
|
|
# doubutsu-2b-lora-756-docci |
|
|
|
An adapter for [qresearch/doubutsu-2b-pt-756](https://huggingface.co/qresearch/doubutsu-2b-pt-756) on DOCCI for 3 epochs. |
|
|
|
|
|
## Usage |
|
|
|
```python |
|
import torch |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
from PIL import Image |
|
|
|
model_id = "qresearch/doubutsu-2b-pt-756" |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_id, |
|
trust_remote_code=True, |
|
torch_dtype=torch.float16, |
|
).to("cuda") |
|
|
|
tokenizer = AutoTokenizer.from_pretrained( |
|
model_id, |
|
use_fast=True, |
|
) |
|
|
|
model.load_adapter("qresearch/doubutsu-2b-lora-756-docci") |
|
|
|
image = Image.open("IMAGE") |
|
|
|
print( |
|
model.answer_question( |
|
image, "Describe the image", tokenizer, max_new_tokens=128, temperature=0.1 |
|
), |
|
) |
|
``` |
|
|
|
## Examples |
|
|
|
| Image | Examples | |
|
| --- | --- | |
|
| <img src="assets/demo1.png" width="300"/> | **Describe the image.**<br>A top-down view of a sidewalk with the word "ROW" in orange spray paint on the left side. A red fire lane symbol is below the word in the middle. A line is on the right side of the sidewalk, going up vertically. A metal strip is on the left side of the sidewalk, going horizontally. Dirt is on the ground in front of the sidewalk. | |
|
| <img src="assets/demo2.png" width="300"/> | **Describe the image.**<br>A low-angle view of a golden statue of a woman on top of a golden horse and a golden chariot. The woman is facing the right side of the image. She is holding a golden bird in her right hand and a golden shield in her left hand. She is wearing a golden robe with a golden belt around her waist. She is wearing a golden helmet on her head. She is holding a golden staff in her right hand and her left hand is raised up above her head, palm facing forward. The woman is wearing a golden crown on her head. The horse is facing the left side of the image. | |
|
|
|
``` |
|
.x+=:. |
|
z` ^% .uef^" |
|
.u . . <k .u . :d88E |
|
.u@u .d88B :@8c .u .@8Ned8" .u u .d88B :@8c . `888E |
|
.zWF8888bx ="8888f8888r ud8888. .@^%8888" ud8888. us888u. ="8888f8888r .udR88N 888E .z8k |
|
.888 9888 4888>'88" :888'8888. x88: `)8b. :888'8888. .@88 "8888" 4888>'88" <888'888k 888E~?888L |
|
I888 9888 4888> ' d888 '88%" 8888N=*8888 d888 '88%" 9888 9888 4888> ' 9888 'Y" 888E 888E |
|
I888 9888 4888> 8888.+" %8" R88 8888.+" 9888 9888 4888> 9888 888E 888E |
|
I888 9888 .d888L .+ 8888L @8Wou 9% 8888L 9888 9888 .d888L .+ 9888 888E 888E |
|
`888Nx?888 ^"8888*" '8888c. .+ .888888P` '8888c. .+ 9888 9888 ^"8888*" ?8888u../ 888E 888E |
|
"88" '888 "Y" "88888% ` ^"F "88888% "888*""888" "Y" "8888P' m888N= 888> |
|
88E "YP' "YP' ^Y" ^Y' "P' `Y" 888 |
|
98> J88" |
|
'8 @% |
|
` :" |
|
``` |
