Edit model card

SetFit Aspect Model with BAAI/bge-small-en-v1.5

This is a SetFit model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses BAAI/bge-small-en-v1.5 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification. In particular, this model is in charge of filtering aspect span candidates.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

This model was trained within the context of a larger system for ABSA, which looks like so:

  1. Use a spaCy model to select possible aspect span candidates.
  2. Use this SetFit model to filter these possible aspect span candidates.
  3. Use a SetFit model to classify the filtered aspect span candidates.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
aspect
  • "younger ones:It might be an entertaining point of discussion with a child 12 or older, but it's too dark for younger ones, unless you're ready to talk about true evil, adult motivations, supernatural forces, and fratricide!"
  • 'Nix:-enjoy the genre of fantasies, of a unknown world, as Nix weaves a wonderful tale of the things that will open your eyes to a different world'
  • 'mystery:The mystery is secondary to the rest of the story and is only really approached in the remaining 30 pages of the book'
no aspect
  • "point:It might be an entertaining point of discussion with a child 12 or older, but it's too dark for younger ones, unless you're ready to talk about true evil, adult motivations, supernatural forces, and fratricide!"
  • "discussion:It might be an entertaining point of discussion with a child 12 or older, but it's too dark for younger ones, unless you're ready to talk about true evil, adult motivations, supernatural forces, and fratricide!"
  • "child:It might be an entertaining point of discussion with a child 12 or older, but it's too dark for younger ones, unless you're ready to talk about true evil, adult motivations, supernatural forces, and fratricide!"

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import AbsaModel

# Download from the 🤗 Hub
model = AbsaModel.from_pretrained(
    "omymble/books-full-bge-aspect",
    "omymble/books-full-bge-polarity",
)
# Run inference
preds = model("The food was great, but the venue is just way too busy.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 2 25.9648 72
Label Training Sample Count
no aspect 572
aspect 167

Training Hyperparameters

  • batch_size: (64, 64)
  • num_epochs: (5, 5)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: True
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0002 1 0.2687 -
0.0090 50 0.2516 -
0.0180 100 0.2619 -
0.0270 150 0.2499 -
0.0360 200 0.2428 -
0.0450 250 0.2443 -
0.0540 300 0.246 -
0.0629 350 0.249 -
0.0719 400 0.2354 -
0.0809 450 0.2347 -
0.0899 500 0.2154 -
0.0989 550 0.2285 -
0.1079 600 0.1812 -
0.1169 650 0.1446 -
0.1259 700 0.165 -
0.1349 750 0.1125 -
0.1439 800 0.0971 -
0.1529 850 0.1059 -
0.1619 900 0.0866 -
0.1709 950 0.0492 -
0.1799 1000 0.0546 0.274
0.1888 1050 0.037 -
0.1978 1100 0.0189 -
0.2068 1150 0.0279 -
0.2158 1200 0.004 -
0.2248 1250 0.0309 -
0.2338 1300 0.0049 -
0.2428 1350 0.0286 -
0.2518 1400 0.0234 -
0.2608 1450 0.0158 -
0.2698 1500 0.0354 -
0.2788 1550 0.0062 -
0.2878 1600 0.0172 -
0.2968 1650 0.0389 -
0.3058 1700 0.0221 -
0.3147 1750 0.0065 -
0.3237 1800 0.0128 -
0.3327 1850 0.0225 -
0.3417 1900 0.0021 -
0.3507 1950 0.0102 -
0.3597 2000 0.012 0.3429
0.3687 2050 0.0249 -
0.3777 2100 0.0054 -
0.3867 2150 0.0014 -
0.3957 2200 0.0014 -
0.4047 2250 0.0143 -
0.4137 2300 0.0078 -
0.4227 2350 0.0195 -
0.4317 2400 0.0006 -
0.4406 2450 0.0014 -
0.4496 2500 0.0083 -
0.4586 2550 0.0141 -
0.4676 2600 0.0046 -
0.4766 2650 0.01 -
0.4856 2700 0.0268 -
0.4946 2750 0.0008 -
0.5036 2800 0.0076 -
0.5126 2850 0.0004 -
0.5216 2900 0.0037 -
0.5306 2950 0.0005 -
0.5396 3000 0.0065 0.3565
0.5486 3050 0.002 -
0.5576 3100 0.0072 -
0.5665 3150 0.0141 -
0.5755 3200 0.0004 -
0.5845 3250 0.0086 -
0.5935 3300 0.0098 -
0.6025 3350 0.0048 -
0.6115 3400 0.0013 -
0.6205 3450 0.007 -
0.6295 3500 0.0059 -
0.6385 3550 0.0174 -
0.6475 3600 0.0003 -
0.6565 3650 0.0004 -
0.6655 3700 0.0032 -
0.6745 3750 0.0004 -
0.6835 3800 0.0035 -
0.6924 3850 0.0019 -
0.7014 3900 0.015 -
0.7104 3950 0.0204 -
0.7194 4000 0.0016 0.3404
0.7284 4050 0.0003 -
0.7374 4100 0.0036 -
0.7464 4150 0.0016 -
0.7554 4200 0.0104 -
0.7644 4250 0.003 -
0.7734 4300 0.0159 -
0.7824 4350 0.0029 -
0.7914 4400 0.0068 -
0.8004 4450 0.0021 -
0.8094 4500 0.006 -
0.8183 4550 0.006 -
0.8273 4600 0.0038 -
0.8363 4650 0.008 -
0.8453 4700 0.0003 -
0.8543 4750 0.0126 -
0.8633 4800 0.0002 -
0.8723 4850 0.0041 -
0.8813 4900 0.0002 -
0.8903 4950 0.0137 -
0.8993 5000 0.0041 0.3363
0.9083 5050 0.0252 -
0.9173 5100 0.0023 -
0.9263 5150 0.0062 -
0.9353 5200 0.0152 -
0.9442 5250 0.0014 -
0.9532 5300 0.0224 -
0.9622 5350 0.0174 -
0.9712 5400 0.0066 -
0.9802 5450 0.0002 -
0.9892 5500 0.0136 -
0.9982 5550 0.0036 -
1.0072 5600 0.0102 -
1.0162 5650 0.011 -
1.0252 5700 0.0035 -
1.0342 5750 0.0002 -
1.0432 5800 0.0002 -
1.0522 5850 0.0044 -
1.0612 5900 0.0125 -
1.0701 5950 0.0061 -
1.0791 6000 0.0165 0.3591
1.0881 6050 0.006 -
1.0971 6100 0.0003 -
1.1061 6150 0.0074 -
1.1151 6200 0.0019 -
1.1241 6250 0.0002 -
1.1331 6300 0.0064 -
1.1421 6350 0.0127 -
1.1511 6400 0.0012 -
1.1601 6450 0.0003 -
1.1691 6500 0.0251 -
1.1781 6550 0.0002 -
1.1871 6600 0.0003 -
1.1960 6650 0.0002 -
1.2050 6700 0.0002 -
1.2140 6750 0.0123 -
1.2230 6800 0.0055 -
1.2320 6850 0.0098 -
1.2410 6900 0.0028 -
1.25 6950 0.0049 -
1.2590 7000 0.0021 0.3537
1.2680 7050 0.0147 -
1.2770 7100 0.003 -
1.2860 7150 0.0002 -
1.2950 7200 0.0049 -
1.3040 7250 0.0033 -
1.3129 7300 0.0002 -
1.3219 7350 0.0065 -
1.3309 7400 0.0043 -
1.3399 7450 0.0107 -
1.3489 7500 0.0184 -
1.3579 7550 0.0116 -
1.3669 7600 0.0041 -
1.3759 7650 0.0001 -
1.3849 7700 0.0001 -
1.3939 7750 0.0074 -
1.4029 7800 0.0002 -
1.4119 7850 0.0087 -
1.4209 7900 0.0014 -
1.4299 7950 0.0045 -
1.4388 8000 0.0018 0.3439
1.4478 8050 0.0039 -
1.4568 8100 0.007 -
1.4658 8150 0.0066 -
1.4748 8200 0.0101 -
1.4838 8250 0.0047 -
1.4928 8300 0.0021 -
1.5018 8350 0.0002 -
1.5108 8400 0.0116 -
1.5198 8450 0.0017 -
1.5288 8500 0.0032 -
1.5378 8550 0.0053 -
1.5468 8600 0.0038 -
1.5558 8650 0.0001 -
1.5647 8700 0.002 -
1.5737 8750 0.0065 -
1.5827 8800 0.0064 -
1.5917 8850 0.0001 -
1.6007 8900 0.0049 -
1.6097 8950 0.0002 -
1.6187 9000 0.0083 0.3486
1.6277 9050 0.0105 -
1.6367 9100 0.0019 -
1.6457 9150 0.0002 -
1.6547 9200 0.0049 -
1.6637 9250 0.0001 -
1.6727 9300 0.0097 -
1.6817 9350 0.0098 -
1.6906 9400 0.0022 -
1.6996 9450 0.0142 -
1.7086 9500 0.0025 -
1.7176 9550 0.0147 -
1.7266 9600 0.0086 -
1.7356 9650 0.0062 -
1.7446 9700 0.0002 -
1.7536 9750 0.0103 -
1.7626 9800 0.0186 -
1.7716 9850 0.0112 -
1.7806 9900 0.0042 -
1.7896 9950 0.0166 -
1.7986 10000 0.0002 0.3571
1.8076 10050 0.0029 -
1.8165 10100 0.0055 -
1.8255 10150 0.0057 -
1.8345 10200 0.0163 -
1.8435 10250 0.0093 -
1.8525 10300 0.0083 -
1.8615 10350 0.0073 -
1.8705 10400 0.0089 -
1.8795 10450 0.0068 -
1.8885 10500 0.0001 -
1.8975 10550 0.0232 -
1.9065 10600 0.0161 -
1.9155 10650 0.0088 -
1.9245 10700 0.0002 -
1.9335 10750 0.0093 -
1.9424 10800 0.0103 -
1.9514 10850 0.002 -
1.9604 10900 0.0113 -
1.9694 10950 0.0055 -
1.9784 11000 0.0148 0.3461
1.9874 11050 0.0001 -
1.9964 11100 0.0017 -
2.0054 11150 0.0001 -
2.0144 11200 0.0204 -
2.0234 11250 0.0032 -
2.0324 11300 0.0029 -
2.0414 11350 0.002 -
2.0504 11400 0.0001 -
2.0594 11450 0.005 -
2.0683 11500 0.0001 -
2.0773 11550 0.0051 -
2.0863 11600 0.0095 -
2.0953 11650 0.0093 -
2.1043 11700 0.0171 -
2.1133 11750 0.0059 -
2.1223 11800 0.0026 -
2.1313 11850 0.0092 -
2.1403 11900 0.0002 -
2.1493 11950 0.0069 -
2.1583 12000 0.006 0.3572
2.1673 12050 0.009 -
2.1763 12100 0.008 -
2.1853 12150 0.0001 -
2.1942 12200 0.0062 -
2.2032 12250 0.0086 -
2.2122 12300 0.0001 -
2.2212 12350 0.0001 -
2.2302 12400 0.0001 -
2.2392 12450 0.0001 -
2.2482 12500 0.0022 -
2.2572 12550 0.0014 -
2.2662 12600 0.0014 -
2.2752 12650 0.009 -
2.2842 12700 0.0001 -
2.2932 12750 0.0081 -
2.3022 12800 0.0127 -
2.3112 12850 0.0001 -
2.3201 12900 0.0028 -
2.3291 12950 0.0016 -
2.3381 13000 0.0051 0.3587
2.3471 13050 0.0044 -
2.3561 13100 0.0133 -
2.3651 13150 0.0043 -
2.3741 13200 0.0001 -
2.3831 13250 0.0017 -
2.3921 13300 0.0095 -
2.4011 13350 0.008 -
2.4101 13400 0.0074 -
2.4191 13450 0.0181 -
2.4281 13500 0.0141 -
2.4371 13550 0.0114 -
2.4460 13600 0.0046 -
2.4550 13650 0.0053 -
2.4640 13700 0.0001 -
2.4730 13750 0.0001 -
2.4820 13800 0.0114 -
2.4910 13850 0.0001 -
2.5 13900 0.0075 -
2.5090 13950 0.0016 -
2.5180 14000 0.0014 0.3376
2.5270 14050 0.0075 -
2.5360 14100 0.0001 -
2.5450 14150 0.0001 -
2.5540 14200 0.0013 -
2.5629 14250 0.0001 -
2.5719 14300 0.0082 -
2.5809 14350 0.0021 -
2.5899 14400 0.0001 -
2.5989 14450 0.0001 -
2.6079 14500 0.0016 -
2.6169 14550 0.0001 -
2.6259 14600 0.0001 -
2.6349 14650 0.0058 -
2.6439 14700 0.0223 -
2.6529 14750 0.0001 -
2.6619 14800 0.0001 -
2.6709 14850 0.0249 -
2.6799 14900 0.008 -
2.6888 14950 0.0071 -
2.6978 15000 0.0237 0.3769
2.7068 15050 0.0001 -
2.7158 15100 0.0016 -
2.7248 15150 0.0031 -
2.7338 15200 0.0063 -
2.7428 15250 0.0001 -
2.7518 15300 0.0127 -
2.7608 15350 0.0001 -
2.7698 15400 0.0114 -
2.7788 15450 0.0106 -
2.7878 15500 0.0086 -
2.7968 15550 0.0083 -
2.8058 15600 0.0001 -
2.8147 15650 0.0001 -
2.8237 15700 0.0035 -
2.8327 15750 0.0095 -
2.8417 15800 0.0041 -
2.8507 15850 0.0001 -
2.8597 15900 0.0001 -
2.8687 15950 0.0001 -
2.8777 16000 0.0001 0.3509
2.8867 16050 0.0001 -
2.8957 16100 0.0124 -
2.9047 16150 0.0083 -
2.9137 16200 0.0017 -
2.9227 16250 0.0001 -
2.9317 16300 0.0042 -
2.9406 16350 0.0058 -
2.9496 16400 0.0001 -
2.9586 16450 0.0001 -
2.9676 16500 0.0021 -
2.9766 16550 0.0025 -
2.9856 16600 0.0068 -
2.9946 16650 0.0099 -
3.0036 16700 0.0015 -
3.0126 16750 0.0086 -
3.0216 16800 0.0162 -
3.0306 16850 0.0001 -
3.0396 16900 0.0181 -
3.0486 16950 0.0083 -
3.0576 17000 0.0045 0.346
3.0665 17050 0.0072 -
3.0755 17100 0.0045 -
3.0845 17150 0.005 -
3.0935 17200 0.003 -
3.1025 17250 0.0069 -
3.1115 17300 0.0001 -
3.1205 17350 0.003 -
3.1295 17400 0.0077 -
3.1385 17450 0.0001 -
3.1475 17500 0.0001 -
3.1565 17550 0.0166 -
3.1655 17600 0.0001 -
3.1745 17650 0.0001 -
3.1835 17700 0.0084 -
3.1924 17750 0.0106 -
3.2014 17800 0.0027 -
3.2104 17850 0.0092 -
3.2194 17900 0.0001 -
3.2284 17950 0.0001 -
3.2374 18000 0.0066 0.3501
3.2464 18050 0.0037 -
3.2554 18100 0.0035 -
3.2644 18150 0.0029 -
3.2734 18200 0.0017 -
3.2824 18250 0.0001 -
3.2914 18300 0.0034 -
3.3004 18350 0.0121 -
3.3094 18400 0.0051 -
3.3183 18450 0.0024 -
3.3273 18500 0.0019 -
3.3363 18550 0.0014 -
3.3453 18600 0.0167 -
3.3543 18650 0.0097 -
3.3633 18700 0.0025 -
3.3723 18750 0.0065 -
3.3813 18800 0.011 -
3.3903 18850 0.0001 -
3.3993 18900 0.0001 -
3.4083 18950 0.0072 -
3.4173 19000 0.0132 0.3511
3.4263 19050 0.0084 -
3.4353 19100 0.0015 -
3.4442 19150 0.0014 -
3.4532 19200 0.011 -
3.4622 19250 0.0083 -
3.4712 19300 0.0073 -
3.4802 19350 0.0024 -
3.4892 19400 0.002 -
3.4982 19450 0.0155 -
3.5072 19500 0.0042 -
3.5162 19550 0.0001 -
3.5252 19600 0.0043 -
3.5342 19650 0.0026 -
3.5432 19700 0.0022 -
3.5522 19750 0.002 -
3.5612 19800 0.0018 -
3.5701 19850 0.0001 -
3.5791 19900 0.0012 -
3.5881 19950 0.002 -
3.5971 20000 0.0089 0.3516
3.6061 20050 0.003 -
3.6151 20100 0.0036 -
3.6241 20150 0.0001 -
3.6331 20200 0.0001 -
3.6421 20250 0.0156 -
3.6511 20300 0.0001 -
3.6601 20350 0.0174 -
3.6691 20400 0.0001 -
3.6781 20450 0.011 -
3.6871 20500 0.0001 -
3.6960 20550 0.0047 -
3.7050 20600 0.0132 -
3.7140 20650 0.007 -
3.7230 20700 0.0001 -
3.7320 20750 0.0025 -
3.7410 20800 0.0049 -
3.75 20850 0.0074 -
3.7590 20900 0.002 -
3.7680 20950 0.0112 -
3.7770 21000 0.0001 0.3483
3.7860 21050 0.0001 -
3.7950 21100 0.0064 -
3.8040 21150 0.0133 -
3.8129 21200 0.0001 -
3.8219 21250 0.0112 -
3.8309 21300 0.0001 -
3.8399 21350 0.0001 -
3.8489 21400 0.0001 -
3.8579 21450 0.0025 -
3.8669 21500 0.0047 -
3.8759 21550 0.0001 -
3.8849 21600 0.0062 -
3.8939 21650 0.0001 -
3.9029 21700 0.0315 -
3.9119 21750 0.002 -
3.9209 21800 0.0034 -
3.9299 21850 0.004 -
3.9388 21900 0.0046 -
3.9478 21950 0.008 -
3.9568 22000 0.0103 0.3474
3.9658 22050 0.0142 -
3.9748 22100 0.0207 -
3.9838 22150 0.0105 -
3.9928 22200 0.0114 -
4.0018 22250 0.002 -
4.0108 22300 0.0121 -
4.0198 22350 0.0001 -
4.0288 22400 0.0058 -
4.0378 22450 0.0045 -
4.0468 22500 0.0001 -
4.0558 22550 0.0086 -
4.0647 22600 0.0121 -
4.0737 22650 0.0045 -
4.0827 22700 0.0001 -
4.0917 22750 0.0046 -
4.1007 22800 0.0076 -
4.1097 22850 0.0001 -
4.1187 22900 0.0154 -
4.1277 22950 0.0108 -
4.1367 23000 0.0058 0.3575
4.1457 23050 0.0088 -
4.1547 23100 0.0019 -
4.1637 23150 0.0055 -
4.1727 23200 0.0299 -
4.1817 23250 0.0085 -
4.1906 23300 0.0016 -
4.1996 23350 0.0001 -
4.2086 23400 0.0001 -
4.2176 23450 0.0072 -
4.2266 23500 0.0092 -
4.2356 23550 0.0001 -
4.2446 23600 0.0064 -
4.2536 23650 0.0065 -
4.2626 23700 0.0001 -
4.2716 23750 0.0017 -
4.2806 23800 0.0083 -
4.2896 23850 0.0001 -
4.2986 23900 0.0039 -
4.3076 23950 0.002 -
4.3165 24000 0.0037 0.357
4.3255 24050 0.0095 -
4.3345 24100 0.002 -
4.3435 24150 0.017 -
4.3525 24200 0.0086 -
4.3615 24250 0.007 -
4.3705 24300 0.0023 -
4.3795 24350 0.0122 -
4.3885 24400 0.0097 -
4.3975 24450 0.0027 -
4.4065 24500 0.0081 -
4.4155 24550 0.0043 -
4.4245 24600 0.0055 -
4.4335 24650 0.0001 -
4.4424 24700 0.0014 -
4.4514 24750 0.0001 -
4.4604 24800 0.0091 -
4.4694 24850 0.0087 -
4.4784 24900 0.0101 -
4.4874 24950 0.0001 -
4.4964 25000 0.013 0.3566
4.5054 25050 0.013 -
4.5144 25100 0.0082 -
4.5234 25150 0.0063 -
4.5324 25200 0.0046 -
4.5414 25250 0.0087 -
4.5504 25300 0.0063 -
4.5594 25350 0.0019 -
4.5683 25400 0.0061 -
4.5773 25450 0.004 -
4.5863 25500 0.0001 -
4.5953 25550 0.0001 -
4.6043 25600 0.0088 -
4.6133 25650 0.0191 -
4.6223 25700 0.0124 -
4.6313 25750 0.0001 -
4.6403 25800 0.0023 -
4.6493 25850 0.0001 -
4.6583 25900 0.0068 -
4.6673 25950 0.0001 -
4.6763 26000 0.0034 0.3563
4.6853 26050 0.0138 -
4.6942 26100 0.0001 -
4.7032 26150 0.0068 -
4.7122 26200 0.0091 -
4.7212 26250 0.0001 -
4.7302 26300 0.0152 -
4.7392 26350 0.0064 -
4.7482 26400 0.0021 -
4.7572 26450 0.0088 -
4.7662 26500 0.0001 -
4.7752 26550 0.0042 -
4.7842 26600 0.0022 -
4.7932 26650 0.0065 -
4.8022 26700 0.0039 -
4.8112 26750 0.0039 -
4.8201 26800 0.0001 -
4.8291 26850 0.0155 -
4.8381 26900 0.0021 -
4.8471 26950 0.0039 -
4.8561 27000 0.002 0.3555
4.8651 27050 0.0092 -
4.8741 27100 0.0001 -
4.8831 27150 0.0081 -
4.8921 27200 0.0081 -
4.9011 27250 0.0037 -
4.9101 27300 0.0104 -
4.9191 27350 0.0022 -
4.9281 27400 0.004 -
4.9371 27450 0.0076 -
4.9460 27500 0.0043 -
4.9550 27550 0.0142 -
4.9640 27600 0.0126 -
4.9730 27650 0.0038 -
4.9820 27700 0.0107 -
4.9910 27750 0.0019 -
5.0 27800 0.0104 -
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 3.0.1
  • spaCy: 3.7.4
  • Transformers: 4.39.0
  • PyTorch: 2.3.1+cu121
  • Datasets: 2.20.0
  • Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
1
Safetensors
Model size
33.4M params
Tensor type
F32
·
Inference Examples
Inference API (serverless) has been turned off for this model.

Model tree for omymble/books-full-bge-aspect

Finetuned
(107)
this model