Edit model card

Sentence similarity model based on SlovakBERT

This is a sentence similarity model based on SlovakBERT. The model was fine-tuned using STSbenchmark [Cer et al 2017] translated to Slovak using M2M100. The model can be used as an universal sentence encoder for Slovak sentences.

Results

The model was evaluated in our paper [Pikuliak et al 2021, Section 4.3]. It achieves 0.7910.791 Spearman correlation on STSbenchmark test set.

Usage

Using this model becomes easy when you have sentence-transformers installed:

pip install -U sentence-transformers

Then you can use the model like this:

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('kinit/slovakbert-sts-stsb')
embeddings = model.encode(sentences)
print(embeddings)

Cite

@inproceedings{pikuliak-etal-2022-slovakbert,
    title = "{S}lovak{BERT}: {S}lovak Masked Language Model",
    author = "Pikuliak, Mat{\'u}{\v{s}}  and
      Grivalsk{\'y}, {\v{S}}tefan  and
      Kon{\^o}pka, Martin  and
      Bl{\v{s}}t{\'a}k, Miroslav  and
      Tamajka, Martin  and
      Bachrat{\'y}, Viktor  and
      Simko, Marian  and
      Bal{\'a}{\v{z}}ik, Pavol  and
      Trnka, Michal  and
      Uhl{\'a}rik, Filip",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2022",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.findings-emnlp.530",
    pages = "7156--7168",
    abstract = "We introduce a new Slovak masked language model called \textit{SlovakBERT}. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.",
}
Downloads last month
72
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train kinit/slovakbert-sts-stsb