Skip to content

Commit 697890b

Browse files
committed
update config.py
1 parent 7c69c25 commit 697890b

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

transformer/config.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ def __init__(
8484
attn_dropout: Optional[float] = 0.0,
8585
tied_weights: bool = False,
8686
seq_len: int = 1024,
87+
pos_encoding: str = "RoPE",
8788
rope_base: float = 10000.0,
8889
max_seq_len: int = 4096,
8990
**kwargs: Dict,

0 commit comments

Comments
 (0)