Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Refine high-quality datasets and visual AI models

A light-weight, flexible, and expressive statistical data testing library

Easy Data Preparation with latest LLMs-based Operators and Pipelines.

Machine learning with dataframes

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Easy to use Python library of customized functions for cleaning and analyzing data.

The toolkit to test, validate, and evaluate your models and surface, curate, and prioritize the most valuable data for labeling.

🗣️ A book and repo to get you started programming voice computing applications in Python (10 chapters and 200+ scripts).

Deal with bad samples in your dataset dynamically, use Transforms as Filters, and more!

Continuously updated paper list on advancements in Data Agents. Companion repo to our paper "A Survey of Data Agents: Emerging Paradigm or Overstated Hype?"

LLM-based text extraction from unstructured data like PDFs, Words and HTMLs. Transform and cluster the text into your desired format. Less information loss, more interpretation, and faster R&D!

Pydantic extension for annotating autocorrecting fields.

🗺️ Data Cleaning and Textual Data Visualization 🗺️

🤖 An automated machine learning framework for audio, text, image, video, or .CSV files (50+ featurizers and 15+ model trainers). Python 3.6 required.

A powerful multi-format file parsing, data cleaning, and AI annotation toolkit.

pyDVL is a library of stable implementations of algorithms for data valuation and influence function computation

A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for cloud data warehouse and Metabase to serve the needs of data visualizations such as analytical dashboards.

OpenDataVal: a Unified Benchmark for Data Valuation in Python (NeurIPS 2023)

🚢 Data Toolkit for Sailor Language Models