arcee-ai/mergekit-gui · LoRA Merging

Apr 15

•

Is this tool capable of LoRA merging? I attempted a merge with LoRA's but was met with an error regarding cache location. I am able to run these merges in my custom colab notebook with the following flag:

--lora-merge-cache .

Is there a way to pass this flag in mergekit-gui?

Nitral-AI

Apr 17

Would also like to see this either added or explained as well if possible?

jeiku

Apr 18

[2024-04-18 03:34:53] [INFO] No HF token provided. Your merged model will be uploaded to the https://huggingface.co/mergekit-community organization.
[2024-04-18 03:34:53] [INFO] Merge configuration saved in /tmp/tmpv6rb5riz/merged/config.yaml
[2024-04-18 03:34:53] [INFO] No repo name provided. Generating a random one.
[2024-04-18 03:34:53] [INFO] Creating repo mergekit-community/mergekit-model_stock-mnzarxs
[2024-04-18 03:34:55] [INFO] Repo created: https://huggingface.co/mergekit-community/mergekit-model_stock-mnzarxs
[2024-04-18 03:34:55] [INFO] Running mergekit-yaml config.yaml merge --copy-tokenizer --cuda --low-cpu-memory
[2024-04-18 03:34:58] [INFO] 
[2024-04-18 03:34:58] [INFO] 
[2024-04-18 03:35:11] [INFO] Warmup loader cache:   0%|          | 0/4 [00:00<?, ?it/s][A
[2024-04-18 03:35:11] [INFO] 
[2024-04-18 03:35:11] [INFO] Warmup loader cache:  25%|██▌       | 1/4 [00:13<00:39, 13.30s/it][A
[2024-04-18 03:35:11] [INFO] Warmup loader cache:  25%|██▌       | 1/4 [00:13<00:39, 13.30s/it]
[2024-04-18 03:35:11] [INFO] Traceback (most recent call last):
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/bin/mergekit-yaml", line 8, in <module>
[2024-04-18 03:35:11] [INFO] sys.exit(main())
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
[2024-04-18 03:35:11] [INFO] return self.main(*args, **kwargs)
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/click/core.py", line 1078, in main
[2024-04-18 03:35:11] [INFO] rv = self.invoke(ctx)
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
[2024-04-18 03:35:11] [INFO] return ctx.invoke(self.callback, **ctx.params)
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/click/core.py", line 783, in invoke
[2024-04-18 03:35:11] [INFO] return __callback(*args, **kwargs)
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/mergekit/options.py", line 78, in wrapper
[2024-04-18 03:35:11] [INFO] f(*args, **kwargs)
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/mergekit/scripts/run_yaml.py", line 47, in main
[2024-04-18 03:35:11] [INFO] run_merge(
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/mergekit/merge.py", line 69, in run_merge
[2024-04-18 03:35:11] [INFO] loader_cache.get(model)
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/mergekit/io/tasks.py", line 31, in get
[2024-04-18 03:35:11] [INFO] merged = model.merged(
[2024-04-18 03:35:11] [INFO] File "/home/user/.pyenv/versions/3.10.14/lib/python3.10/site-packages/mergekit/common.py", line 93, in merged
[2024-04-18 03:35:11] [INFO] raise RuntimeError("Need to specify cache dir to merge adapters")
[2024-04-18 03:35:11] [INFO] RuntimeError: Need to specify cache dir to merge adapters
[2024-04-18 03:35:12] [ERROR] Command exited with code 1
[2024-04-18 03:35:12] [ERROR] Merge failed. Deleting repo as no model is uploaded.

jeiku

Apr 18

@chargoddard As far as I can tell, you just need to add --lora-merge-cache . to the startup flags. I have tested this extensively in my own colab notebook, and there should be no ill effect of adding that flag as it is ignored when not required.

chargoddard

Arcee AI org Apr 19

Should be fixed now! Thanks for the report.

chargoddard changed discussion status to closed Apr 19