Update README.md
Browse files
README.md
CHANGED
@@ -19,19 +19,19 @@ The model only takes images as document-side inputs and produce vectors represen
|
|
19 |
|
20 |
# News
|
21 |
|
22 |
-
- 2024-07-14: We released **huggingface demo**! Try our [online demo](https://huggingface.co/spaces/bokesyo/minicpm-visual-embeeding-v0-demo)!
|
23 |
|
24 |
-
- 2024-07-14: We released a **Gradio demo** of `miniCPM-visual-embedding-v0`, take a look at [pipeline_gradio.py](https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0/blob/main/pipeline_gradio.py). You can run `pipeline_gradio.py` to build a demo on your PC.
|
25 |
|
26 |
-
- 2024-07-13: We released a **command-line based demo** of `miniCPM-visual-embedding-v0` for users to retireve most relavant pages from a given PDF file (could be very long), take a look at [pipeline.py](https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0/blob/main/pipeline.py).
|
27 |
|
28 |
- 2024-06-27: π We released our first visual embedding model checkpoint minicpm-visual-embedding-v0 on [huggingface](https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0).
|
29 |
|
30 |
- 2024-05-08: π We [open-sourced](https://github.com/RhapsodyAILab/minicpm-visual-embedding-v0) our training code (full-parameter tuning with GradCache and DeepSpeed, supports large batch size across multiple GPUs with zero-stage1) and eval code.
|
31 |
|
32 |
-
#
|
33 |
|
34 |
-
Pip install all dependencies:
|
35 |
|
36 |
```
|
37 |
Pillow==10.1.0
|
@@ -43,27 +43,32 @@ sentencepiece==0.1.99
|
|
43 |
numpy==1.26.0
|
44 |
```
|
45 |
|
46 |
-
|
|
|
|
|
47 |
|
48 |
```bash
|
49 |
git lfs install
|
50 |
git clone https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0
|
51 |
```
|
52 |
|
53 |
-
|
54 |
|
55 |
```bash
|
|
|
56 |
huggingface-cli download --resume-download RhapsodyAI/minicpm-visual-embedding-v0 --local-dir minicpm-visual-embedding-v0 --local-dir-use-symlinks False
|
57 |
```
|
58 |
|
59 |
-
|
60 |
|
61 |
```bash
|
62 |
pip install gradio
|
63 |
python pipeline_gradio.py
|
64 |
```
|
65 |
|
66 |
-
|
|
|
|
|
67 |
|
68 |
```python
|
69 |
from transformers import AutoModel
|
@@ -105,11 +110,11 @@ print(scores)
|
|
105 |
|
106 |
# Todos
|
107 |
|
108 |
-
|
109 |
|
110 |
-
|
111 |
|
112 |
-
|
113 |
|
114 |
# Limitations
|
115 |
|
|
|
19 |
|
20 |
# News
|
21 |
|
22 |
+
- 2024-07-14: We released **online huggingface demo**! Try our [online demo](https://huggingface.co/spaces/bokesyo/minicpm-visual-embeeding-v0-demo)!
|
23 |
|
24 |
+
- 2024-07-14: We released a **locally deployable Gradio demo** of `miniCPM-visual-embedding-v0`, take a look at [pipeline_gradio.py](https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0/blob/main/pipeline_gradio.py). You can run `pipeline_gradio.py` to build a demo on your PC.
|
25 |
|
26 |
+
- 2024-07-13: We released a **locally deployable command-line based demo** of `miniCPM-visual-embedding-v0` for users to retireve most relavant pages from a given PDF file (could be very long), take a look at [pipeline.py](https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0/blob/main/pipeline.py).
|
27 |
|
28 |
- 2024-06-27: π We released our first visual embedding model checkpoint minicpm-visual-embedding-v0 on [huggingface](https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0).
|
29 |
|
30 |
- 2024-05-08: π We [open-sourced](https://github.com/RhapsodyAILab/minicpm-visual-embedding-v0) our training code (full-parameter tuning with GradCache and DeepSpeed, supports large batch size across multiple GPUs with zero-stage1) and eval code.
|
31 |
|
32 |
+
# Deploy on your PC
|
33 |
|
34 |
+
1. Pip install all dependencies:
|
35 |
|
36 |
```
|
37 |
Pillow==10.1.0
|
|
|
43 |
numpy==1.26.0
|
44 |
```
|
45 |
|
46 |
+
2. Download the model weights and modeling file, choose one of the following:
|
47 |
+
|
48 |
+
- Download with git clone.
|
49 |
|
50 |
```bash
|
51 |
git lfs install
|
52 |
git clone https://huggingface.co/RhapsodyAI/minicpm-visual-embedding-v0
|
53 |
```
|
54 |
|
55 |
+
- Download with huggingface-hub.
|
56 |
|
57 |
```bash
|
58 |
+
pip install huggingface-hub
|
59 |
huggingface-cli download --resume-download RhapsodyAI/minicpm-visual-embedding-v0 --local-dir minicpm-visual-embedding-v0 --local-dir-use-symlinks False
|
60 |
```
|
61 |
|
62 |
+
3. To deploy a local demo, first check `pipeline_gradio.py`, change `model_path` to your local path and change `device` to your device (for users with Nvidia card, use `cuda`, for users with apple silicon, use `mps`, for users with only x86 cpu, please use `cpu`). then launch the demo:
|
63 |
|
64 |
```bash
|
65 |
pip install gradio
|
66 |
python pipeline_gradio.py
|
67 |
```
|
68 |
|
69 |
+
# For research purpose
|
70 |
+
|
71 |
+
To run the model for research purpose, please refer the following code:
|
72 |
|
73 |
```python
|
74 |
from transformers import AutoModel
|
|
|
110 |
|
111 |
# Todos
|
112 |
|
113 |
+
[x] Release huggingface space demo.
|
114 |
|
115 |
+
[] Release the evaluation results.
|
116 |
|
117 |
+
[] Release technical report.
|
118 |
|
119 |
# Limitations
|
120 |
|