onnxruntime

//github.com/microsoft/PowerToys) - The DB is sqlite but the system is probably just kernel-memory which is a local .NET application: https://github.com/microsoft/kernel-memory - The SLM is Phi-3 which is open and designed primarily to run locally https://azure.microsoft.com/en-us/blog/introducing-phi-3-red... - The actual underlying tech stack is DirectML (https://github.com/microsoft/DirectML) and ONNX (https://github.com/microsoft/onnxruntime). ---- So the data is intended to be encrypted at rest along with the rest of the OS, it's all run locally (which isn't a handwaivy thing, the tech is all very much capable of running locally) and if you don't have hardware capable of running it, it shouldn't be enabled in the first place. My confusion with all of this is why Recall didn't start out as a PowerToys feature. It sounds like the exact type of internal "look at this cool little toy I built" thing that generally makes it into PowerToys but I'm assuming some exec ran with the opportunity and said "this is awesome, let's ship it with the OS and make it a highliner feature for our AI push" which is how we got here.

a nightmare to work with thoes drivers and jit compilers. Hopefully Vcc can take compute shader more seriously. [1]: https://www.circle-lang.org/ [2]: https://shader-slang.com/ [3]: https://github.com/microsoft/onnxruntime/tree/dev/opencl

e sense to support anything besides WebGPU at this point. WASM + SIMD is around 15-20x slower on my machine[1]. Although WebGL is more widely supported today, it doesn't have the compute features needed for efficient modern ML (transformers etc) and will likely be a deprecated backend for other frameworks when WebGPU comes online. tensorflow rust has a struct.Tensor: https://tensorflow.github.io/rust/tensorflow/struct.Tensor.h... "ONNX Runtime merges WebGPU backend" https://github.com/microsoft/onnxruntime https://news.ycombinator.com/item?id=35696031 ... TIL about wonnx: https://github.com/webonnx/wonnx#in-the-browser-using-webgpu... microsoft/onnxruntime: https://github.com/microsoft/onnxruntime Apache/arrow has language-portable Tensors for cpp: https://arrow.apache.org/docs/cpp/api/tensor.html and rust: https://docs.rs/arrow/latest/arrow/tensor/struct.Tensor.html and Python: https://arrow.apache.org/docs/python/api/tables.html#tensors https://arrow.apache.org/docs/python/generated/pyarrow.Tenso... Fwiw it looks like the llama.cpp Tensor is from ggml, for which there are CUDA and OpenCL implementations (but not yet ROCm, or a WebGPU shim for use with emscripten transpilation to WASM): https://github.com/ggerganov/llama.cpp/blob/master/ggml.h Are the recommendable ways to cast e.g. arrow Tensors to pytorch/tensorflow? FWIU, Rust has a better compilation to WASM; and that's probably faster than already-compiled-to-JS/ES TensorFlow + WebGPU. What's a fair benchmark?

ase is published approximately every quarter, and the upcoming roadmap can be found here. ONNX Runtime v1.14.0 was Feb 10th