Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix potential pointer math truncation and improve readability #551

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

davidfiala
Copy link

Fix potential pointer math truncation bug. Improve readability of datatypes when binary length or handling is at play.

  • Fix potential pointer error due to integer truncation in read_checkpoint. Although unlikely due to compiler padding, *data + sizeof(Config)/sizeof(float); could can been fractional and thus truncated, leading to an incorrect pointer. Admittedly rare given the pointer padding, but given how people depend on llama2.c as a canonical learning resource, it should be fixed.
  • Simplify reading of weights by opening the file only once. Less LOC, less opens by using just one fd now.
  • Simplify code related to "data = start of file + sizeof(config)" in that same function
  • Add packing pragmas to ensure that the compiler understands we're reading a binary file in mmap'd structs to avoid implicit padding anywhere in the structs. Although offsets of 4x bytes often works out of the box, I'd hate for people to copy/paste this code and in some case get unexpected results/corrupt pointers.
  • Simplify/clarify datatypes for a few places where we're dealing with binary data and binary math such as Config and rng_seed: use uint64_t and [u]int32_t to make it absolutely clear what the binary format is and what length's we're dealing with rather than depending on compiler convention of int and unsigned long long
  • improvement: parse CLI flag seed as a uint64_t rather than int, unlocking the full range of possible seeds rather than just those up to 2^31-1

…dability of datatypes when binary length matters.

- Fix potential pointer error due to integer truncation in read_checkpoint. Although unlikely due to compiler padding, `*data + sizeof(Config)/sizeof(float);` could can been fractional and thus truncated, leading to an incorrect pointer.
- Simplify reading of weights by opening the file only once. Less LOC, less opens by using just one fd now.
- Simplify code related to "data = start of file + sizeof(config)" in
  that same function
- Add packing pragmas to ensure that the compiler understands we're reading a binary file in mmap'd structs to avoid implicit padding anywhere in the structs. Although offsets of 4x bytes often works out of the box, I'd hate for people to copy/paste this code and in some case get unexpected results/corrupt pointers.
- Simplify/clarify datatypes for a few places where we're dealing with binary data and binary math such as Config and rng_seed: use uint64_t and [u]int32_t to make it absolutely clear what the binary format is and what length's we're dealing with rather than depending on compiler convention of `int` and `unsigned long long`
- improvement: parse CLI flag seed as a uint64_t rather than `int`, unlocking the full range of possible seeds rather than just those up to `2^31-1`
…te I didn't have a quantized bin file on hand to test)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant