Text Generation
Transformers
PyTorch
English
gpt2
causal-lm
text-generation-inference
Cerebras-GPT-2.7B / config.json
jaliu's picture
Add files
7903d2d
raw
history blame contribute delete
361 Bytes
{
"model_type": "gpt2",
"attn_pdrop": 0.0,
"scale_attn_weights": true,
"resid_pdrop": 0.0,
"n_inner": 10240,
"n_embd": 2560,
"layer_norm_epsilon": 1e-05,
"n_positions": 2048,
"activation_function": "gelu",
"n_head": 32,
"n_layer": 32,
"tie_word_embeddings": true,
"vocab_size": 50257,
"embd_pdrop": 0.0
}