cactus

for js/ts you have vercel aisdk [0], for c++ you have [1], for flutter/reactnative/kotlin there is [2] [0] https://github.com/vercel/ai [1] https://github.com/ClickHouse/ai-sdk-cpp [2] https://github.com/cactus-compute/cactus

You don't need to guess: https://github.com/cactus-compute/cactus/tree/main/cpp

you can plug in a vector DB and run Cactus embeddings for retrieval. Assuming you're using React Native, here's an example: https://github.com/cactus-compute/cactus/tree/main/react#emb... (Flutter works the same way) What are you building?

Hey HN, Henry and Roman here - we've been building a cross-platform framework for deploying LLMs, VLMs, Embedding Models and TTS models locally on smartphones. Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more. Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks and Google AI Edge respectively. However, both are platform-specific and only support specific models from the company. To this end, Cactus: - Is available in Flutter, React-Native & Kotlin Multi-platform for cross-platform developers, since most apps

Interesting to think about the concept of local-first in the age of AI. Wanting to be able to run AI fully privately, and offline is the reason we created Cactus: https://github.com/cactus-compute/cactus Fully open-source, cross-platform & blazing-fast; lets you plug in private AI into any app on your phone.