| Based ob the paper: "UmlsBERT: Augmenting Contextual Embeddings with a Clinical Metathesaurus" (https://aclanthology.org/2021.naacl-main.139.pdf). | |
| and the github repo: https://github.com/gmichalo/UmlsBERT | |
| BERT base model. | |
| Trained from scratch on MIMIC dataset, using the UMLS dataset to mask words within the text. | |
| We achived better accuracy on MedNLI dataset. | |
| Bert Model accuracy: 83% |