Here are articles that we've written to show you how to work with Kura.

Benchmarking Kura

Kura is an open-source topic modeling library that automatically discovers and summarizes high-level themes from large collections of conversational data. By combining AI-powered summarization with advanced clustering techniques, Kura transforms thousands of raw conversations into actionable insights about what your users are actually discussing.

We benchmarked Kura across three critical dimensions: processing performance, storage efficiency, and clustering quality. Our results show that Kura delivers production-ready performance with:

  1. Fast, predictable processing: 6,000 conversations analyzed with GPT-4o-mini in under 7 minutes and just around $2 in token costs (using 20 concurrent tasks)
  2. Storage is not an issue: 440x compression ratios mean even 100,000 conversations require only 20MB of storage - storage overhead is negligible for production workloads
  3. Accurate topic discovery: Over 85% cluster alignment when validated against similar conversation topics

In this article, we'll walk through our benchmark methodology, detailed findings and how you can apply these results to your own use cases.

Dataset Used

For this benchmark, we used the lmsys/mt_bench_human_judgments dataset from Hugging Face.

This dataset contains 3,000+ rows of human preferences between two model responses to identical questions, with each question sampled multiple times across different model pairs.

We generated two conversations per row, creating a 6,000+ conversation evaluation dataset that tests clustering quality with identical inputs and varying responses.

Note

If you're interested in the full dataset, we've uploaded the processed dataset we used here to hugging face. We also have the full benchmarking scripts and datasets generated at here.

Kura v0.5.0 Released

We're excited to announce the release of Kura v0.5.0! This release brings significant improvements to documentation, introduces a new procedural API for maximum flexibility, and includes numerous enhancements to make Kura even better for analyzing conversation data.

What's New in v0.5.0

New Procedural API (v1)

The headline feature of this release is the introduction of a functional, procedural API that gives you fine-grained control over the analysis pipeline:

from kura.summarisation import summarise_conversations
from kura.cluster import generate_base_clusters_from_conversation_summaries
from kura.meta_cluster import reduce_clusters_from_base_clusters
from kura.dimensionality import reduce_dimensionality_from_clusters

# Run each step independently
summaries = await summarise_conversations(conversations, model=summary_model)
clusters = await generate_base_clusters_from_conversation_summaries(summaries, model=cluster_model)
meta_clusters = await reduce_clusters_from_base_clusters(clusters, model=meta_cluster_model)
projected = await reduce_dimensionality_from_clusters(meta_clusters, model=dim_reduction_model)

This new API offers: - Complete control over each pipeline step - Easy integration with heterogeneous models (OpenAI, vLLM, Hugging Face) - Functional programming style with no hidden state - Keyword-only arguments for clarity

New Documentation Release

We're excited to announce a comprehensive overhaul of the Kura documentation! The new documentation is designed to help users get started quickly and make the most of Kura's powerful features for analyzing conversation data.

What's New

Our documentation has been completely reorganized and expanded to provide a better experience:

  • Clear Structure: New organization with dedicated sections for Getting Started, Core Concepts, and API Reference
  • Comprehensive Installation Guide: Detailed instructions for different installation methods, including both uv and pip
  • In-depth Tutorials: Step-by-step guides in our Getting Started section
  • Enriched API Reference: Better organized and more detailed API documentation
  • Core Concepts Explained: Detailed explanations of Kura's architecture and components
  • Improved Code Examples: Concise, practical examples throughout

Evaluating Kura's Clustering ability using Synthetic Analysis

Over the weekend, I spent some time to evaluate Kura's clustering ability using synthetic data. When tested against synthetically generated technical conversations, Kura is able to identify base clusters that align with our original category distribution with over 95% accuracy and also discover more nuanced groupings that align with real-world technical divisions and use cases.

In this article, we'll walk through the process of how we generated a diverse dataset of ~190 user conversations and then evaluated Kura's clustering ability against this dataset. These findings demonstrate that language model-assisted clustering can identify natural conversation patterns while validating synthetic data generation approaches.

Generating Synthetic Data

I carefully constructed a dataset of 190 user conversations by using a multi-step process. You can access the dataset of these conversations on hugging face here. To do so, we introduced controlle variation at each level through a systematic approach that involved 3 steps.