license: cc-by-sa-4.0
SLIM-SA-NER-3B-TOOL
slim-sa-ner-3b-tool is a 4_K_M quantized GGUF version of slim-sa-ner-3b, providing a small, fast inference implementation, optimized for multi-model concurrent deployment.
This model combines two of the most popular traditional classifier capabilities (sentiment analysis and named entity recognition) and re-images them as function calls on a small specialized decoder LLM, generating output in the form of a python dictionary with keys corresponding to sentiment and NER identifiers.
The intent of SLIMs is to forge a middle-ground between traditional encoder-based classifiers and open-ended API-based LLMs.
The size of the self-contained GGUF model binary is 1.71 GB, which is small enough to run locally on a CPU, and yet which comparables favorably with the use of two traditional FP32 versions of Roberta-Large for NER (1.42GB) and BERT for Sentiment Analysis (440 MB), while offering greater potential capacity depth with 2.7B parameters, and without the requirement of Pytorch and other external dependencies.
slim-sa-ner-3b is part of the SLIM ("Structured Language Instruction Model") series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.
To pull the model via API:
from huggingface_hub import snapshot_download
snapshot_download("llmware/slim-sa-ner-3b-tool", local_dir="/path/on/your/machine/", local_dir_use_symlinks=False)
Load in your favorite GGUF inference engine, or try with llmware as follows:
from llmware.models import ModelCatalog
# to load the model and make a basic inference
model = ModelCatalog().load_model("slim-sa-ner-3b-tool")
response = model.function_call(text_sample)
# this one line will download the model and run a series of tests
ModelCatalog().tool_test_run("slim-sa-ner-3b-tool", verbose=True)
Note: please review config.json in the repository for prompt wrapping information, details on the model, and full test set.
Model Card Contact
Darren Oberst & llmware team