TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.

Postgres with GPUs for ML/AI apps.

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

A blazing fast inference solution for text embeddings models

Framework to prove inference of ML models blazingly fast

Korvus is a search SDK that unifies the entire RAG pipeline in a single database query. Built on top of Postgres with bindings for Python, JavaScript, Rust and C.

playwright for windows computer use

🧠 Motorhead is a memory and information retrieval server for LLMs.

Fast, streaming indexing, query, and agentic LLM applications in Rust

An Open-Source Machine Learning Framework in Rust Δ

⚡ Edgen: Local, private GenAI server alternative to OpenAI. No GPU required. Run AI models locally: LLMs (Llama2, Mistral, Mixtral...), Speech-to-text (whisper) and many others.

Powerful Powerful Machine Learning library with GPU, CPU and WASM backends

AI no jimaku gumi (AIの字幕組), a subtitle maker for video using AI.

High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included

User friendly CLI tool for AI tasks. Stop thinking about LLMs and prompts, start getting results!

A Deep Learning and preprocessing framework in Rust with support for CPU and GPU.

A natively parallel dataloader for Python, written in Rust. Serving data at GB/s speeds, while covering aspect ratio bucketing, crop and resize for image ML workloads.

OpenAI's Gym written in pure Rust for blazingly fast performance

SciRS2 - Scientific Computing and AI in Rust

The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework in Rust