Commit
•
f1484b7
1
Parent(s):
9d3f212
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,12 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-sa-4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-sa-4.0
|
| 3 |
+
pipeline_tag: fill-mask
|
| 4 |
+
language: en
|
| 5 |
+
tags:
|
| 6 |
+
- legal
|
| 7 |
+
- long-documents
|
| 8 |
---
|
| 9 |
+
|
| 10 |
+
# Legal Longformer (base)
|
| 11 |
+
|
| 12 |
+
This is a derivative model based on the [LexLM (base)](https://huggingface.co/lexlms/roberta-base-cased) RoBERTa model. All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following [Beltagy et al. (2020)](https://arxiv.org/abs/2004.05150) using a python script similar to this one (https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb).
|