SIsimonw·Sep 8, 2025
Oh sorry, I missed that bit.
I got Codex + GPT-5 to modify that Llama chat example to implement the "load from local directory" pattern. It appears to work.
First you'll need to grab the checkout of the local model (~1.3GB):
git lfs install
git clone https://huggingface.co/onnx-community/Llama-3.2-1B-Instruct-q4f16
Then visit this page: https://static.simonwillison.net/static/2025/llama-3.2-webgp... - in Chrome or Firefox Nightly.
Now click "Browse folder" and select the folder you just checked out with Git.
Click the confusing "Upload" confirmation (it doesn't upload anything, just opens those files in the current browser session).
Now click "Load local model" - and you should get a full working chat interface.
Code is here: https://github.com/simonw/transformers.js-examples/commit/cd...
Here's the full Codex session that I used to
Read the rest
build this: https://gist.github.com/simonw/3c46c9e609f6ee77367a760b5ca01...
I ran Codex against the https://github.com/huggingface/transformers.js-examples/tree... folder and prompted:
> Modify this application such that it offers the user a file browse button for selecting their own local copy of the model file instead of loading it over the network. Provide a "download model" option too.
Then later:
> Build the production app and then make it available on localhost somehow
And:
> Uncaught (in promise) Error: Invalid configuration detected: both local and remote models are disabled. Fix by setting `env.allowLocalModels` or `env.allowRemoteModels` to `true`.
And:
> Add a bash script which will build the application such that I can upload a folder called llama-3.2-webgpu to http://static.simonwillison.net/static/2025/llama-3.2-webgpu... and http://static.simonwillison.net/static/2025/llama-3.2-webgpu... will serve the app
(Note that this doesn't allow you to use any model on your machine, but it proves that it's possible.)