Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
Commit
•
6318758
1
Parent(s):
ebea0c0
update the doc (replace npm -> bun etc)
Browse files
README.md
CHANGED
@@ -73,6 +73,8 @@ As a prerequisite you need to have [git lfs](https://git-lfs.com/) installed (se
|
|
73 |
git lfs install
|
74 |
```
|
75 |
|
|
|
|
|
76 |
Clapper has been tested with Node `20.15.1`.
|
77 |
|
78 |
To make sure you use this version, you can use [NVM](https://github.com/nvm-sh/nvm) to activate it:
|
@@ -84,39 +86,42 @@ nvm use
|
|
84 |
|
85 |
If you find that Clapper is working with a more recent (stable) version of Node, or have a better version management to suggest, please open a ticket.
|
86 |
|
87 |
-
###
|
88 |
|
89 |
Install the dependencies.
|
90 |
|
91 |
`--include=optional` is to make sure optional dependencies are installed (pre-build native modules compatible with your system)
|
92 |
|
93 |
```bash
|
94 |
-
|
95 |
```
|
96 |
|
97 |
-
|
98 |
|
99 |
```bash
|
100 |
-
npm run
|
101 |
```
|
102 |
|
|
|
103 |
|
104 |
-
### Building the app
|
105 |
|
106 |
-
|
107 |
|
108 |
```bash
|
109 |
-
|
110 |
```
|
111 |
|
112 |
### Running the app with Electron
|
113 |
|
|
|
114 |
An Electron build is in progress.
|
115 |
|
116 |
There are still some things to debug and figure out, but if you are a developer you can try it out by starting Clapper through Electron like this:
|
117 |
|
|
|
118 |
```bash
|
119 |
-
|
|
|
120 |
```
|
121 |
|
122 |
If that doesn't work for you (issue with node-gyp, setuptools, distutils etc) you might have to run:
|
@@ -128,7 +133,8 @@ python3 -m pip install --break-system-packages setuptools
|
|
128 |
You can also build Clapper for your operating system by typing:
|
129 |
|
130 |
```bash
|
131 |
-
|
|
|
132 |
```
|
133 |
|
134 |
You might see a DeprecationWarning message written in red at the end but that's only a warning, just open `./out/make` to check if the build worked.
|
@@ -141,7 +147,9 @@ I have setup Prettier and ESLint, they use some basic rules (you can propose new
|
|
141 |
|
142 |
To run all the tests (unit and e2e) please run:
|
143 |
|
144 |
-
|
|
|
|
|
145 |
|
146 |
This is not instantaneous: playwright may seems to do nothing for a while at first.
|
147 |
|
@@ -153,10 +161,18 @@ Note: I've just added Vitest so we only have a few tests for now.
|
|
153 |
|
154 |
|
155 |
To run the test without watching, type:
|
156 |
-
|
|
|
|
|
|
|
|
|
157 |
|
158 |
To run the tests with watching, type:
|
159 |
-
|
|
|
|
|
|
|
|
|
160 |
|
161 |
#### End-to-end tests
|
162 |
|
@@ -165,30 +181,30 @@ Note: I've just added Playwright, but we don't really have tests yet.
|
|
165 |
Please note that due to the app needing to build during 30~60s (depending on the speed of your computer), running those tests may
|
166 |
take some time to execute.
|
167 |
|
168 |
-
`
|
169 |
|
170 |
More Playwright commands:
|
171 |
-
`
|
172 |
Runs the end-to-end tests.
|
173 |
|
174 |
-
`
|
175 |
Starts the interactive UI mode.
|
176 |
|
177 |
-
`
|
178 |
Runs the tests only on Desktop Chrome.
|
179 |
|
180 |
-
`
|
181 |
Runs the tests in a specific file.
|
182 |
|
183 |
-
`
|
184 |
Runs the tests in debug mode.
|
185 |
|
186 |
-
`
|
187 |
Auto generate tests with Codegen.
|
188 |
|
189 |
We suggest that you begin by typing:
|
190 |
|
191 |
-
`
|
192 |
|
193 |
|
194 |
### Architecture
|
|
|
73 |
git lfs install
|
74 |
```
|
75 |
|
76 |
+
You will also need to install [Bun](https://bun.sh/docs/installation)
|
77 |
+
|
78 |
Clapper has been tested with Node `20.15.1`.
|
79 |
|
80 |
To make sure you use this version, you can use [NVM](https://github.com/nvm-sh/nvm) to activate it:
|
|
|
86 |
|
87 |
If you find that Clapper is working with a more recent (stable) version of Node, or have a better version management to suggest, please open a ticket.
|
88 |
|
89 |
+
### Installation
|
90 |
|
91 |
Install the dependencies.
|
92 |
|
93 |
`--include=optional` is to make sure optional dependencies are installed (pre-build native modules compatible with your system)
|
94 |
|
95 |
```bash
|
96 |
+
bun i
|
97 |
```
|
98 |
|
99 |
+
You can run the following command to build the app, making sure all the types are consistant and properly set:
|
100 |
|
101 |
```bash
|
102 |
+
npm run build
|
103 |
```
|
104 |
|
105 |
+
### Running the web app
|
106 |
|
|
|
107 |
|
108 |
+
Then run the actual app, the first time you go to localhost:3000 after typing this command, the app will compile, which can take a minute (like, literally: `Compiled / in 52.6s (6372 modules)`)
|
109 |
|
110 |
```bash
|
111 |
+
bun run dev
|
112 |
```
|
113 |
|
114 |
### Running the app with Electron
|
115 |
|
116 |
+
|
117 |
An Electron build is in progress.
|
118 |
|
119 |
There are still some things to debug and figure out, but if you are a developer you can try it out by starting Clapper through Electron like this:
|
120 |
|
121 |
+
|
122 |
```bash
|
123 |
+
cd packages/app
|
124 |
+
bun run electron:start
|
125 |
```
|
126 |
|
127 |
If that doesn't work for you (issue with node-gyp, setuptools, distutils etc) you might have to run:
|
|
|
133 |
You can also build Clapper for your operating system by typing:
|
134 |
|
135 |
```bash
|
136 |
+
cd packages/app
|
137 |
+
bun run electron:make
|
138 |
```
|
139 |
|
140 |
You might see a DeprecationWarning message written in red at the end but that's only a warning, just open `./out/make` to check if the build worked.
|
|
|
147 |
|
148 |
To run all the tests (unit and e2e) please run:
|
149 |
|
150 |
+
```bash
|
151 |
+
bun run test
|
152 |
+
```
|
153 |
|
154 |
This is not instantaneous: playwright may seems to do nothing for a while at first.
|
155 |
|
|
|
161 |
|
162 |
|
163 |
To run the test without watching, type:
|
164 |
+
|
165 |
+
```bash
|
166 |
+
cd packages/app
|
167 |
+
bun run test:unit:ci
|
168 |
+
```
|
169 |
|
170 |
To run the tests with watching, type:
|
171 |
+
|
172 |
+
```bash
|
173 |
+
cd packages/app
|
174 |
+
bun run test:unit:watch
|
175 |
+
```
|
176 |
|
177 |
#### End-to-end tests
|
178 |
|
|
|
181 |
Please note that due to the app needing to build during 30~60s (depending on the speed of your computer), running those tests may
|
182 |
take some time to execute.
|
183 |
|
184 |
+
`bun run test:e2e`
|
185 |
|
186 |
More Playwright commands:
|
187 |
+
`bunx playwright test`
|
188 |
Runs the end-to-end tests.
|
189 |
|
190 |
+
`bunx playwright test --ui`
|
191 |
Starts the interactive UI mode.
|
192 |
|
193 |
+
`bunx playwright test --project=chromium`
|
194 |
Runs the tests only on Desktop Chrome.
|
195 |
|
196 |
+
`bunx playwright test example`
|
197 |
Runs the tests in a specific file.
|
198 |
|
199 |
+
` bunx playwright test --debug`
|
200 |
Runs the tests in debug mode.
|
201 |
|
202 |
+
`bunx playwright codegen`
|
203 |
Auto generate tests with Codegen.
|
204 |
|
205 |
We suggest that you begin by typing:
|
206 |
|
207 |
+
`bunx playwright test`
|
208 |
|
209 |
|
210 |
### Architecture
|
bun.lockb
CHANGED
Binary files a/bun.lockb and b/bun.lockb differ
|
|
package.json
CHANGED
@@ -21,7 +21,7 @@
|
|
21 |
"dev": "bun run --cwd packages/app dev",
|
22 |
"start": "bun run --cwd packages/app start",
|
23 |
"build": "bun run build:all",
|
24 |
-
"build:all": "bun run build:clap && bun run build:timeline && bun run build:api-client && bun run build:io && bun run build:colors && bun run build:engine && bun run build:broadway && bun run build:services && bun run build:app",
|
25 |
"build:clap": "bun run --cwd packages/clap build",
|
26 |
"build:timeline": "bun run --cwd packages/timeline build",
|
27 |
"build:api-client": "bun run --cwd packages/api-client build",
|
|
|
21 |
"dev": "bun run --cwd packages/app dev",
|
22 |
"start": "bun run --cwd packages/app start",
|
23 |
"build": "bun run build:all",
|
24 |
+
"build:all": "bun run build:clap && bun run build:timeline && bun run build:api-client && bun run build:io && bun run build:colors && bun run build:engine && bun run build:broadway && bun run build:clapper-services && bun run build:app",
|
25 |
"build:clap": "bun run --cwd packages/clap build",
|
26 |
"build:timeline": "bun run --cwd packages/timeline build",
|
27 |
"build:api-client": "bun run --cwd packages/api-client build",
|
packages/app/package.json
CHANGED
@@ -86,7 +86,7 @@
|
|
86 |
"@tailwindcss/container-queries": "^0.1.1",
|
87 |
"@types/dom-speech-recognition": "^0.0.4",
|
88 |
"@types/pngjs": "^6.0.5",
|
89 |
-
"@
|
90 |
"@xyflow/react": "^12.0.3",
|
91 |
"autoprefixer": "10.4.19",
|
92 |
"base64-arraybuffer": "^1.0.2",
|
|
|
86 |
"@tailwindcss/container-queries": "^0.1.1",
|
87 |
"@types/dom-speech-recognition": "^0.0.4",
|
88 |
"@types/pngjs": "^6.0.5",
|
89 |
+
"@huggingface/transformers": "3.0.0-alpha.5",
|
90 |
"@xyflow/react": "^12.0.3",
|
91 |
"autoprefixer": "10.4.19",
|
92 |
"base64-arraybuffer": "^1.0.2",
|
packages/app/public/workers/captioning.worker.js
CHANGED
@@ -1,4 +1,4 @@
|
|
1 |
-
importScripts('https://cdn.jsdelivr.net/npm/@
|
2 |
|
3 |
const { AutoProcessor, AutoTokenizer, Moondream1ForConditionalGeneration, RawImage } = transformers;
|
4 |
|
|
|
1 |
+
importScripts('https://cdn.jsdelivr.net/npm/@huggingface/[email protected]');
|
2 |
|
3 |
const { AutoProcessor, AutoTokenizer, Moondream1ForConditionalGeneration, RawImage } = transformers;
|
4 |
|
packages/app/src/services/autocomplete/extractCaptionFromFrameMoondream.ts
CHANGED
@@ -3,7 +3,8 @@ import {
|
|
3 |
AutoTokenizer,
|
4 |
Moondream1ForConditionalGeneration,
|
5 |
RawImage,
|
6 |
-
|
|
|
7 |
|
8 |
export async function extractCaptionFromFrameMoondream(
|
9 |
imageInBase64DataUri: string
|
@@ -51,12 +52,13 @@ Linux experimental support also requires launching the browser with --enable-fea
|
|
51 |
const vision_inputs = await processor(image)
|
52 |
|
53 |
// Generate response
|
54 |
-
const output = await model.generate({
|
55 |
...text_inputs,
|
56 |
...vision_inputs,
|
57 |
do_sample: false,
|
58 |
max_new_tokens: 177,
|
59 |
-
})
|
|
|
60 |
const decoded = tokenizer.batch_decode(output, { skip_special_tokens: false })
|
61 |
console.log(decoded)
|
62 |
// [
|
|
|
3 |
AutoTokenizer,
|
4 |
Moondream1ForConditionalGeneration,
|
5 |
RawImage,
|
6 |
+
Tensor,
|
7 |
+
} from '@huggingface/transformers'
|
8 |
|
9 |
export async function extractCaptionFromFrameMoondream(
|
10 |
imageInBase64DataUri: string
|
|
|
52 |
const vision_inputs = await processor(image)
|
53 |
|
54 |
// Generate response
|
55 |
+
const output = (await model.generate({
|
56 |
...text_inputs,
|
57 |
...vision_inputs,
|
58 |
do_sample: false,
|
59 |
max_new_tokens: 177,
|
60 |
+
})) as Tensor // is that alright? I'm not sure..
|
61 |
+
|
62 |
const decoded = tokenizer.batch_decode(output, { skip_special_tokens: false })
|
63 |
console.log(decoded)
|
64 |
// [
|
packages/app/src/services/autocomplete/extractCaptionsFromFrames.ts
CHANGED
@@ -3,7 +3,7 @@
|
|
3 |
// AutoTokenizer,
|
4 |
// Florence2ForConditionalGeneration,
|
5 |
// RawImage,
|
6 |
-
// } from '@
|
7 |
|
8 |
export const cache: {
|
9 |
model?: Promise<any>
|
|
|
3 |
// AutoTokenizer,
|
4 |
// Florence2ForConditionalGeneration,
|
5 |
// RawImage,
|
6 |
+
// } from '@huggingface/transformers'
|
7 |
|
8 |
export const cache: {
|
9 |
model?: Promise<any>
|