TheBloke

/

goliath-120b-GPTQ

like 16

Regarding this branch: https://huggingface.co/TheBloke/goliath-120b-GPTQ/tree/gptq-3bit--1g-actorder_True

When I run it with TGI I get:

huggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'TheBloke/goliath-120b-GPTQ:gptq-3bit--1g-actorder_True'.

I've tested in TGI v1.1.1 and the latest version, v1.3.4, so I'm guessing this isn't due to a recent change in requirements.

Full error:

2024-01-04T11:50:01.191601230Z 2024-01-04T11:50:01.191458Z  INFO text_generation_launcher: Args { model_id: "TheBloke/goliath-120b-GPTQ:gptq-3bit--1g-actorder_True", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: Some(Gptq), dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 1024, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, hostname: "75538cdede3a", port: 3000, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/data"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, env: false }
2024-01-04T11:50:01.191631687Z 2024-01-04T11:50:01.191485Z  INFO text_generation_launcher: Sharding model on 2 processes
2024-01-04T11:50:01.191635594Z 2024-01-04T11:50:01.191549Z  INFO download: text_generation_launcher: Starting download process.
2024-01-04T11:50:03.643908063Z 2024-01-04T11:50:03.643633Z ERROR download: text_generation_launcher: Download encountered an error: Traceback (most recent call last):
2024-01-04T11:50:03.643941646Z 
2024-01-04T11:50:03.643945183Z   File "/opt/conda/bin/text-generation-server", line 8, in <module>
2024-01-04T11:50:03.643948229Z     sys.exit(app())
2024-01-04T11:50:03.643950924Z 
2024-01-04T11:50:03.643953038Z   File "/opt/conda/lib/python3.9/site-packages/text_generation_server/cli.py", line 115, in download_weights
2024-01-04T11:50:03.643955933Z     utils.weight_files(model_id, revision, extension)
2024-01-04T11:50:03.643958278Z 
2024-01-04T11:50:03.643960281Z   File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/hub.py", line 96, in weight_files
2024-01-04T11:50:03.643962425Z     filenames = weight_hub_files(model_id, revision, extension)
2024-01-04T11:50:03.643964541Z 
2024-01-04T11:50:03.643966524Z   File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/hub.py", line 25, in weight_hub_files
2024-01-04T11:50:03.643968689Z     info = api.model_info(model_id, revision=revision)
2024-01-04T11:50:03.643970712Z 
2024-01-04T11:50:03.643972716Z   File "/opt/conda/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
2024-01-04T11:50:03.643975101Z     validate_repo_id(arg_value)
2024-01-04T11:50:03.643977445Z 
2024-01-04T11:50:03.643979398Z   File "/opt/conda/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 164, in validate_repo_id
2024-01-04T11:50:03.643981502Z     raise HFValidationError(
2024-01-04T11:50:03.643983606Z 
2024-01-04T11:50:03.643985720Z huggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'TheBloke/goliath-120b-GPTQ:gptq-3bit--1g-actorder_True'.
2024-01-04T11:50:03.643987854Z 
2024-01-04T11:50:03.643989828Z 
2024-01-04T11:50:03.644028811Z Error: DownloadError