UCall

ined on balanced multilingual datasets, with equal exposure to languages from English, Chinese, and Hindi to Arabic, Hebrew, and Armenian. UForm now supports 21 languages, is so tiny that you can run it in the browser, and outputs small 256-dimensional embeddings. Perfect for rapid image and video searches. It's already available on Hugging-Face as "unum-cloud/uform-vl-multilingual-v2". UCall [3]: It started as a FastAPI alternative focusing on JSON-RPC (instead of REST protocols), offering 70x the bandwidth and 1/50th the latency. It was good but not enough, so we've added REST and TLS support, broadening its appeal. I've merged that code, and it is yet to be tested. Early benchmarks suggest that we still hit the same 150'000-250'000 requests/s on a single CPU core in Python by reusing HTTPS connections. StringZilla [4]: This project lets you sift through multi-gigabyte or terabyte strings with minimal use of RAM and maximal use of SIMD and SWAR techniques. All these projects are engineered for scalability and efficiency, even on tight budgets. Our demo, for instance, works on hundreds of gigabytes of images using just a few gigabytes of RAM and no GPUs for AI inference. That is a toy example with a small, noisy dataset, and I look forward to showing a much larger setup. Interestingly, even this tiny setup illustrates issues common to UForm and much larger OpenAI CLIP models - the quality of Multi-Modal alignment [5]. It also shows how different/accurate the search results are across different languages. Synthetic benchmarks suggest massive improvements for some low-resource languages (like Armenian and Hebrew) and more popular ones (like Hindi and Arabic) [6]. Still, when we look at visual demos like this, I can see a long road ahead for us and the broader industry, making LLMs Multi-Modal in 2024 :) All of the projects and the demo code are available under an Apache license, so feel free to use them in your commercial projects :) PS: The demo looks much nicer with just Unsplash dataset of 25'000 images, but it's less representative of modern AI datasets, too small, and may not be the best way to honestly show our current weaknesses. The second dataset - Conceptual Captions - is much noisier, and quite ugly. [1]: https://github.com/unum-cloud/usearch [2]: https://github.com/unum-cloud/uform [3]: https://github.com/unum-cloud/ucall [4]: https://github.com/ashvardanian/stringzilla [5]: https://ashvardanian.com/posts/searching-stable-marriages/#e... [6]: https://www.unum.cloud/blog/2023-08-17-uform-graphcore

s, like multi-shot requests, also look intriguing, but we couldn't see a flawless way to integrate them into UJRPC. Maybe next time. Like a parent with two kids, we tell everyone we love Kernel Bypass and SIMD equally. So I decided to combine the two, potentially implementing one of the fastest implementations of the most straightforward RPC protocol - JSON-RPC. ~~Healthy and Fun~~ Efficient and Simple, what can be better? By now, you may already guess at least one of the dependencies - `simdjson` by Daniel Lemiere, that has become the industry standard. io_uring is generally very fast, even with a single core. Adding more polling threads may only increase congestion. We needed to continue using no more than one thread, but parsing messages may involve more work than just invoking a JSON parser. JSON-RPC is transport agnostic. The incoming requests can be sent over HTTP, pre-pended by rows of headers. Those would have to be POSTs and generally contain Content-Length and Content-Type. There is a SIMD-accelerated library for that as well. It is called `picohttpparser`, uses SSE, and is maintained by H2O. The story doesn't end there. JSON is limited. Passing binary strings is a nightmare. The most common approach is to encode them with base-64. So we took the Turbo-Base64 from the PowTurbo project to decode those binary strings. The core implementation of UJRPC is under 2000 lines of C++. Knowing that those lines connect 3 great libraries with the newest and coolest parts of Linux is enough to put a smile on my face. Most people are more rational, so here is another reason to be cheerful. - FastAPI throughput: 3'184 rps. - Python gRPC throughput: 9'849 rps. - UJRPC throughput: -- Python server with io_uring: 43'000 rps. -- C server with POSIX: 79'000 rps. -- C server with io_uring: 231'000 rps. Granted, this is yet to be your batteries-included server. It can't balance the load, manage threads, spell S in HTTPS, or call parents when you misbehave in school. But at least part of it you shouldn't expect from a web server. After following the standardization process of executors in C++ for the last N+1 years, we adapted the "bring your runtime" and "bring your thread-pool" policies. HTTPS support, however, is our next primary objective. --- Of course, it is a pre-production project and must have a lot of bugs. Don't hesitate to report them. We have huge plans for this tiny package and will potentially make it the default transport of UKV: https://github.com/unum-cloud/ukv