SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model trained on the stanfordnlp/imdb dataset that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 2 classes
Training Dataset: stanfordnlp/imdb

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
0	"I can't believe that those praising this movie herein aren't thinking of some other film. I was prepared for the possibility that this would be awful, but the script (or lack thereof) makes for a film that's also pointless. On the plus side, the general level of craft on the part of the actors and technical crew is quite competent, but when you've got a sow's ear to work with you can't make a silk purse. Ben G fans should stick with just about any other movie he's been in. Dorothy S fans should stick to Galaxina. Peter B fans should stick to Last Picture Show and Target. Fans of cheap laughs at the expense of those who seem to be asking for it should stick to Peter B's amazingly awful book, Killing of the Unicorn." 'This has to be the worst piece of garbage I've seen in a while. Heath Ledger is a heartthrob? He looked deformed. I wish I'd known that he and Naomi Watts are an item in real life because I spent 2 of the longest hours of my life wondering what she saw in him. Orlando Bloom is a heartthrob? With the scraggly beard and deer-in-the-headlights look about him, I can't say I agree. Rachel Griffiths was her usual fabulous self, but Geoffrey Rush looked as if he couldn't wait to get off the set. I'm supposed to feel sorry for bankrobbers and murderers? This is a far cry from Butch Cassidy, which actually WAS an entertaining film. This was trite, cliche-ridden and boring. We only stayed because we were convinced it would get better. It didn't. The last 10-15 minutes or so were unintentionally hilarious. Heath and his gang are holed up in a frontier hotel, and women and children are dying because of their presence. That's not funny. But it was funny when they walked out of the hotel with the armor on, because all we could think of was the Black Knight from Monty Python and the Holy Grail. I kept waiting for them to say "I'll bite yer leg off!" We were howling with laughter, as were several other warped members of the audience. When we left, pretty much everyone was talking about what a waste of time this film was. I may not have paid cash to see this disaster (sneak preview), but it certainly wasn't free. It cost me 2 hours of my life that I will never get back.' "This movie was awful. The ending was absolutely horrible. There was no plot to the movie whatsoever. The only thing that was decent about the movie was the acting done by Robert DuVall and James Earl Jones. Their performances were excellent! The only problem was that the movie did not do their acting performances any justice. If the script would have come close to capturing a halfway decent story, it would be worth watching. Instead, Robert DuVall's and James Earl Jones' performances are completely wasted on a god awful storyline...or lack thereof. Not only was I left waiting throughout the movie for something to happen to make the movie....well an actual movie...not just utterless dialog between characters for what ended up being absolutely no reason. It was nothing more than common dialog that would have taken place back in that period of time. There was nothing special about any of the characters. The only thing special was how Robert DuVall portrayed a rambling, senile, drunk, old man. Nothing worthy happens during the entire movie including the end. When the movie ended, I sat amazed...amazed that I sat through the entire movie waiting for something of interest to happen to make watching the movie worth while. It never happened! The cast of characters suddenly started rolling making it apparent that the movie really was over and I realized that I had just wasted 2 hours of my life watching a movie with absolutely no plot and no meaning. It wasn't even a story. The entire movie takes place in a day's worth of time. That's it. It was one day in the life (and death) of some Southerners on a plantation. How much of a story can take place in a single day (other than the movie Training Day)? The acting performances by the entire cast were excellent, but they were grossly wasted on such a disappointment of a movie...if you can even call it a movie."
1	"OK its not the best film I've ever seen but at the same time I've been able to sit and watch it TWICE!!! story line was pretty awful and during the first part of the first short story i wondered what the hell i was watching but at the same time it was so awful i loved it cheap laughs all the way. And Jebidia deserves an Oscar for his role in this movie the only thing that let him down was half way through he stopped his silly name calling. overall the film was pretty perfetic but if your after cheap laughs and you see it in pound land go by it." "I very much looked forward to this movie. Its a good family movie; however, if Michael Landon Jr.'s editing team did a better job of editing, the movie would be much better. Too many scenes out of context. I do hope there is another movie from the series, they're all very good. But, if another one is made, I beg them to take better care at editing. This story was all over the place and didn't seem to have a center. Which is unfortunate because the other movies of the series were great. I enjoy the story of Willie and Missy; they're both great role models. Plus, the romantic side of the viewers always enjoy a good love story." "or anyone who was praying for the sight of Al Cliver wrestling a naked, 7ft tall black guy into a full nelson, your film has arrived! Film starlet Laura Crawford (Ursula Buchfellner) is kidnapped by a group who demand the ransom of $6 million to be delivered to their island hideaway. What they don't count on is rugged Vietnam vet Peter Weston (Cliver) being hired by a film producer to save the girl. And what they really didn't count on was a local tribe that likes to offer up young women to their monster cannibal god with bloodshot bug eyes. Pretty much the same filming set up as CANNIBALS, this one fares a bit better when it comes to entertainment value, thanks mostly a hilarious dub track and the impossibly goofy monster with the bulging eyes (Franco confirms they were split ping pong balls on the disc's interview). Franco gets a strong EuroCult supporting cast including Gisela Hahn (CONTAMINATION) and Werner Pochath (whose death is one of the most head-scratching things I ever seen as a guy who is totally not him is shown - in close up - trying to be him). The film features tons of nudity and the gore (Tempra paint variety) is there. The highlight for me was the world's slowly fistfight between Cliver and Antonio de Cabo in the splashing waves. Sadly, ol' Jess pads this one out to an astonishing (and, at times, agonizing) 1 hour and 40 minutes when it should have run 80 minutes tops. For the most part, the Severin DVD looks pretty nice but there are some odd ghosting images going on during some of the darker scenes. Also, one long section of dialog is in Spanish with no subs (they are an option, but only when you listen to the French track). Franco gives a nice 16- minute interview about the film and has much more pleasant things to say about Buchfellner than his CANNIBALS star Sabrina Siani."

Label

Examples

"I can't believe that those praising this movie herein aren't thinking of some other film. I was prepared for the possibility that this would be awful, but the script (or lack thereof) makes for a film that's also pointless. On the plus side, the general level of craft on the part of the actors and technical crew is quite competent, but when you've got a sow's ear to work with you can't make a silk purse. Ben G fans should stick with just about any other movie he's been in. Dorothy S fans should stick to Galaxina. Peter B fans should stick to Last Picture Show and Target. Fans of cheap laughs at the expense of those who seem to be asking for it should stick to Peter B's amazingly awful book, Killing of the Unicorn."
'This has to be the worst piece of garbage I've seen in a while.

Heath Ledger is a heartthrob? He looked deformed. I wish I'd known that he and Naomi Watts are an item in real life because I spent 2 of the longest hours of my life wondering what she saw in him.

Orlando Bloom is a heartthrob? With the scraggly beard and deer-in-the-headlights look about him, I can't say I agree.

Rachel Griffiths was her usual fabulous self, but Geoffrey Rush looked as if he couldn't wait to get off the set.

I'm supposed to feel sorry for bankrobbers and murderers? This is a far cry from Butch Cassidy, which actually WAS an entertaining film. This was trite, cliche-ridden and boring. We only stayed because we were convinced it would get better. It didn't.

The last 10-15 minutes or so were unintentionally hilarious. Heath and his gang are holed up in a frontier hotel, and women and children are dying because of their presence. That's not funny. But it was funny when they walked out of the hotel with the armor on, because all we could think of was the Black Knight from Monty Python and the Holy Grail. I kept waiting for them to say "I'll bite yer leg off!" We were howling with laughter, as were several other warped members of the audience. When we left, pretty much everyone was talking about what a waste of time this film was.

I may not have paid cash to see this disaster (sneak preview), but it certainly wasn't free. It cost me 2 hours of my life that I will never get back.'
"This movie was awful. The ending was absolutely horrible. There was no plot to the movie whatsoever. The only thing that was decent about the movie was the acting done by Robert DuVall and James Earl Jones. Their performances were excellent! The only problem was that the movie did not do their acting performances any justice. If the script would have come close to capturing a halfway decent story, it would be worth watching. Instead, Robert DuVall's and James Earl Jones' performances are completely wasted on a god awful storyline...or lack thereof. Not only was I left waiting throughout the movie for something to happen to make the movie....well an actual movie...not just utterless dialog between characters for what ended up being absolutely no reason. It was nothing more than common dialog that would have taken place back in that period of time. There was nothing special about any of the characters. The only thing special was how Robert DuVall portrayed a rambling, senile, drunk, old man. Nothing worthy happens during the entire movie including the end. When the movie ended, I sat amazed...amazed that I sat through the entire movie waiting for something of interest to happen to make watching the movie worth while. It never happened! The cast of characters suddenly started rolling making it apparent that the movie really was over and I realized that I had just wasted 2 hours of my life watching a movie with absolutely no plot and no meaning. It wasn't even a story. The entire movie takes place in a day's worth of time. That's it. It was one day in the life (and death) of some Southerners on a plantation. How much of a story can take place in a single day (other than the movie Training Day)? The acting performances by the entire cast were excellent, but they were grossly wasted on such a disappointment of a movie...if you can even call it a movie."

"OK its not the best film I've ever seen but at the same time I've been able to sit and watch it TWICE!!! story line was pretty awful and during the first part of the first short story i wondered what the hell i was watching but at the same time it was so awful i loved it cheap laughs all the way.

And Jebidia deserves an Oscar for his role in this movie the only thing that let him down was half way through he stopped his silly name calling.

overall the film was pretty perfetic but if your after cheap laughs and you see it in pound land go by it."
"I very much looked forward to this movie. Its a good family movie; however, if Michael Landon Jr.'s editing team did a better job of editing, the movie would be much better. Too many scenes out of context. I do hope there is another movie from the series, they're all very good. But, if another one is made, I beg them to take better care at editing. This story was all over the place and didn't seem to have a center. Which is unfortunate because the other movies of the series were great. I enjoy the story of Willie and Missy; they're both great role models. Plus, the romantic side of the viewers always enjoy a good love story."
"or anyone who was praying for the sight of Al Cliver wrestling a naked, 7ft tall black guy into a full nelson, your film has arrived! Film starlet Laura Crawford (Ursula Buchfellner) is kidnapped by a group who demand the ransom of $6 million to be delivered to their island hideaway. What they don't count on is rugged Vietnam vet Peter Weston (Cliver) being hired by a film producer to save the girl. And what they really didn't count on was a local tribe that likes to offer up young women to their monster cannibal god with bloodshot bug eyes.

Pretty much the same filming set up as CANNIBALS, this one fares a bit better when it comes to entertainment value, thanks mostly a hilarious dub track and the impossibly goofy monster with the bulging eyes (Franco confirms they were split ping pong balls on the disc's interview). Franco gets a strong EuroCult supporting cast including Gisela Hahn (CONTAMINATION) and Werner Pochath (whose death is one of the most head-scratching things I ever seen as a guy who is totally not him is shown - in close up - trying to be him). The film features tons of nudity and the gore (Tempra paint variety) is there. The highlight for me was the world's slowly fistfight between Cliver and Antonio de Cabo in the splashing waves. Sadly, ol' Jess pads this one out to an astonishing (and, at times, agonizing) 1 hour and 40 minutes when it should have run 80 minutes tops.

For the most part, the Severin DVD looks pretty nice but there are some odd ghosting images going on during some of the darker scenes. Also, one long section of dialog is in Spanish with no subs (they are an option, but only when you listen to the French track). Franco gives a nice 16- minute interview about the film and has much more pleasant things to say about Buchfellner than his CANNIBALS star Sabrina Siani."

Evaluation

Metrics

Label	Accuracy
all	0.8242

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("If only to avoid making this type of film in the future. This film is interesting as an experiment but tells no cogent story.<br /><br />One might feel virtuous for sitting thru it because it touches on so many IMPORTANT issues but it does so without any discernable motive. The viewer comes away with no new perspectives (unless one comes up with one while one's mind wanders, as it will invariably do during this pointless film).<br /><br />One might better spend one's time staring out a window at a tree growing.<br /><br />")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	48	244.4571	888

Label	Training Sample Count
1	7
0	63

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0039	1	0.2493	-
0.1953	50	0.0016	-
0.3906	100	0.0003	-
0.5859	150	0.003	-
0.7812	200	0.0014	-
0.9766	250	0.0002	-
1.0	256	-	0.4699

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.8.19
SetFit: 1.0.3
Sentence Transformers: 3.0.1
Transformers: 4.39.0
PyTorch: 2.4.0
Datasets: 2.20.0
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

kenhktsui
/

anyclassifier_setfit_demo