Please keep this simple

[kroggen]

The main goal should be code readability, and easy understanding, for learning

There are many PRs that add lots of complexity

One option is to use separate branches, maybe 3:

• main -> clean code
• fastest -> to check where this can reach
• intermediate

Or reference other implementations on the README

[karpathy]

@kroggen agree ty. If people want the fastest thing they should take a look at the excellent llama.cpp.

[Foundation42]

I feel that this issue is targeted towards the work I did writing the matrix multiply code, and fixing the cache alignment issue.

It is a little disheartening of course given the amount of work, and there is a comment by @karpathy on #95 saying he doesn't mind the matrix multiplies being more complicated because that is where the work gets done, and now if you want fast use something else.

Yet the march for performance moves on, with exploration of half floats and other data types that are sure to add complexity.

Perhaps it would be good to say in the README what kinds of PR's the maintainers will allow and which not so that other people don't waste time in future.

I still believe there is educational value in seeing the guts of a matrix multiplication, since those are the guts of the whole system.

Maybe the right thing to do would be just to leave it frozen in time like nanoGPT, so it preserves its simplicity, and then do additional versions with more performance or features as a separate thing, idk.

In any case, I quite enjoyed writing the code, so not to worry.

All the best

[karpathy]

Hi @Foundation42 thanks for your thoughts, I adjusted the readme with contributor guidelines.