llm.f90

I'd like to think he took the name from my llm.f90 project https://github.com/rbitr/llm.f90 It was originally based off of Karpathy's llama2.c but I renamed it when I added support for other architectures. Probable a coincidence :)

I wasn't aware of this, I'd seen those URLs before but only in the context of Chinese ones and thought it was Chinese-specific. It's interesting because I just went down an apparent rabbit hole inplementing Byte-level encoding for using language models with unicode. There each byte in a unicode character is mapped to a printable character that goes up to 255 < ord(x) < 511 (I don't remember the highest but the point is each byte is mapped to another printable unicode character. See https://github.com/openai/gpt-2/blob/9b63575ef42771a015060c9... And the actual list of characters: https://github.com/rbitr/llm.f90/blob/dev/phi2/phi2/pretoken...

I'm a Fortran hobbyist. I'm working (unfortunately less frequently now) on a LLM framework in Fortan. Sort of a spin out from work, firmly in hobby territory though: https://github.com/rbitr/llm.f90 There are lots of good free or open source tools, which is one of the things I like about fortran - support from GNU, Intel, etc. I wouldn't be inclined to pay for one.

Also, I should promote the code I wrote for running this. It runs models in ggml format, the one I made available is an older checkpoint though. It's easy to convert the newer one. And it's in Fortran but it should be easy to get gfortran if you don't have it installed. https://github.com/rbitr/llm.f90/tree/optimize16/purefortran

The original mamba code has a lot of speed optimizations and other stuff that make it difficult to immediately get so this will help with learning. I can't help but also plug my own Mamba inference implementation. https://github.com/rbitr/llm.f90/tree/master/ssm For inference one token at a time everything simplifies considerably.

You may also be interested in https://github.com/rbitr/llm.f90/tree/master/ssm it's my inference only implementation of mamba which ends up being much simpler than the training code in the original repo

There is a directory of github projects that use Fortran: https://github.com/Beliavsky/Fortran-code-on-GitHub I'll add that it's well suited to modern AI inference and several projects exist, e.g. https://github.com/nlpodyssey/rwkv.f90 https://github.com/certik/fastGPT https://github.com/rbitr/llm.f90 (disclaimer, mine)

I submitted an earlier version of this a few months ago (as llama2.f90). At that time it had a lot of steps to run and was just a toy, now it's easy to run and is a competitive option for llm inference. See the motivation section for discussion and the `Performance` issue for an ongoing discussion about performance.