Files

 main

/

notebooks

/

Copy path

Directory actions

More options

Directory actions

More options

Latest commit

le1nux

refactor: replaced max_length with truncation in configs

Jun 10, 2024

c5a9f6f · Jun 10, 2024

History

History

/

notebooks

/

Top

Folders and files

Name		Name	Last commit message	Last commit date
parent directory ..
tokenizer		tokenizer	refactor: local changes	Apr 16, 2024
components.yaml		components.yaml	refactor: replaced max_length with truncation in configs	Jun 10, 2024
redpajama_tokenizer_test.ipynb		redpajama_tokenizer_test.ipynb	refactor: local changes	Apr 16, 2024
View all files