ArtifactAI
commited on
Commit
•
1de5f30
1
Parent(s):
5e23461
Update README.md
Browse files
README.md
CHANGED
@@ -163,7 +163,7 @@ model-index:
|
|
163 |
# Longformer Encoder-Decoder (LED) fine-tuned on Billsum
|
164 |
This model is a fine-tuned version of led-large-16384 on the billsum dataset.
|
165 |
|
166 |
-
As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-
|
167 |
|
168 |
|
169 |
# Use In Transformers
|
|
|
163 |
# Longformer Encoder-Decoder (LED) fine-tuned on Billsum
|
164 |
This model is a fine-tuned version of led-large-16384 on the billsum dataset.
|
165 |
|
166 |
+
As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-large-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.
|
167 |
|
168 |
|
169 |
# Use In Transformers
|