mlx-vlm

I think mlx-vlm should support both the Audio and the Omni model: https://github.com/Blaizzy/mlx-vlm/ and there should be native MLX exports for both here: - https://huggingface.co/LiquidAI/LFM3-Audio-2B-MLX-8bit - https://huggingface.co/LiquidAI/LFM3-Omni-2B-MLX-8bit

This: https://github.com/Blaizzy/mlx-lm/tree/pc/add-lg ...and this is what I should probably wait for (not sure why it's in vlm): https://github.com/Blaizzy/mlx-vlm/tree/pc/laguna-s-nvfp4 ...or perhaps I should've just used the gguf with the provided llama.cpp instead of trying to run the nvfp4-mlx from the get go, but where's the chaos in that :) Running deepseek flash on something locally now, this will have to wait a bit. I still stand by my initial quick assessment - looks capable. Some people on r/localllama also reported loops. We'll see in ~10 hours. Hopefully I haven't terribly mislead people.

To clarify, this MIT-licensed app is from the very same dev, 'Prince Canuma', who maintains the popular MLX-VLM library (https://github.com/Blaizzy/mlx-vlm). MLX-VLM is a long-time dependency of the excellent LM Studio and others because it can provide faster inference on Apple devices than llama.cpp. Historically, MLX is a smaller community than CUDA, but has some of the fastest updates upon the release of new models, particularly in models with modalities beyond text-in, text-out (vision, STT, TTS, video gen). See also (https://github.com/Blaizzy/mlx-audio-swift). Would be totally unsurprised if those modalities and models get integrated into this UI. Possibly vibe-coded landing page notwithstanding, the app is mostly written in Swift language. That suggests it will be easy to port this inference stack to iPad and iPhone.

Sorry on the delay - so it installs https://github.com/Blaizzy/mlx-vlm and other components and sets up the commands - you don't need to use it but we thought it might be easier for folks

I've shared a bunch of notes on MLX over the past year, many of them with snippets of code I've used to try out models: https://simonwillison.net/tags/mlx/ I mainly use MLX for LLMs (with https://github.com/ml-explore/mlx-lm and my own https://github.com/simonw/llm-mlx which wraps that), vision LLMs (via https://github.com/Blaizzy/mlx-vlm) and running Whisper (https://github.com/ml-explore/mlx-examples/tree/main/whisper) I haven't tried mlx-audio yet (which can synthesize speech) but it looks interesting too: https://github.com/Blaizzy/mlx-audio The two best people to follow for MLX stuff are Apple's Awni Hannun - https://twitter.com/awnihannun and https://github.com/awni - and community member Prince Canuma who's responsible for both mlx-vlm and mlx-audio: https://twitter.com/Prince_Canuma and https://github.com/Blaizzy

This model is available for MLX now, in various different sizes. I ran https://huggingface.co/mlx-community/Qwen2.5-VL-32B-Instruct... using uv (so no need to install libraries first) and https://github.com/Blaizzy/mlx-vlm like this: uv run --with 'numpy<2' --with mlx-vlm \ python -m mlx_vlm.generate \ --model mlx-community/Qwen2.5-VL-32B-Instruct-4bit \ --max-tokens 1000 \ --temperature 0.0 \ --prompt "Describe this image." \ --image Mpaboundrycdfw-1.png That downloaded an ~18GB model and gave me a VERY impressive result, shown at the bottom here: https://simonwillison.net/2025/Mar/24/qwen25-vl-32b/

Mistral 3.1 vision works with MLX and mlx-vlm - https://github.com/Blaizzy/mlx-vlm/pull/256 https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I...