This invention is a multi-agent retrieval-augmented large language model (LLM) architecture that classifies free-form text comments into precise operational categories and subcategories, providing an auditable reasoning for its decision. Its most significant advantage is a no-code graphical user interface (GUI) that allows non-technical users to quickly repurpose the framework for any classification domain by simply supplying new data and rules, moving beyond its initial transportation focus.
Description
The core technology is a multi-agent retrieval-augmented LLM pipeline designed to overcome the inconsistencies and delays of manual text review. This system functions in three parts: first, a knowledge database stores historical comments and approved decision labels ; second, a retrieval component uses cosine-similarity search to inject the most relevant historical examples and policy snippets into the model context ; and third, reasoning agents (LLM) constrained by a defined label taxonomy consume this retrieved evidence to produce a primary category/subcategory prediction, a secondary prediction, and an explicit, two-sentence, auditable reasoning process.
The system uses a novel three-stage multi-agent pipeline that replaces traditional one-shot classifiers, which boosts accuracy, reduces cost, and cuts label confusion. It is made domain-general through a no-code configuration GUI, which compiles user-defined classification sets, a knowledge database, and retrieval parameters into agent-specific prompts and configurations, allowing non-technical staff to deploy a ready-to-use classifier for new use cases without writing code. Furthermore, a full audit trail records all neighbors, scores, ranks, and agent outputs for every decision, delivering end-to-end traceability and supporting continuous performance improvement.
Applications
- Public Sector Comment Triage: Classifying high-volume free-form public comments (e.g., about transit, potholes, or traffic signals) for transportation agencies.
- Facility/Building Management: Automating the classification and routing of facility tickets and maintenance requests.
- Permitting & Licensing: Categorizing and routing public comments related to permitting applications.
- General Customer/Public Feedback: Classifying feedback in any industry based on domain-specific taxonomy and policy rules.
- Document Management & Triage: Categorizing and routing internal documents or communications into operational or archival categories.
Advantages
- High Accuracy and Consistency: Uses a three-stage multi-agent pipeline with role-specialized agents to boost accuracy and reduce latency and label confusion compared to one-shot classifiers.
- Rapid, No-Code Deployment: The no-code GUI enables non-technical users to rapidly configure and deploy a new classifier for any domain by simply uploading data and defining taxonomy and policies.
- Evidence-Grounded and Auditable Decisions: Provides an explicit, readable reasoning process and a full audit trail for every classification, ensuring transparency and supporting human oversight.
- Enhanced Decision Stability: Utilizes rank-weighted similarity with label-support aggregation, which sums evidence across neighboring examples to produce more stable decisions, even with close scores or imbalanced classes.
- Dual-Prediction Review Support: Surfaces both a primary and secondary (next most likely) prediction with a concise justification, which helps in reviewing multi-issue comments and exposes credible alternatives to reviewers.
Invention Readiness
A prototype exists for the multi-agent retrieval-augmented LLM system for automated text classification, and its performance has been validated. The software is generalizable and includes a no-code configuration GUI to repurpose the framework for any classification domain by supplying new taxonomy definitions, policy rules, and a knowledge corpus. The system has been validated on transportation problems for classifying comments into categories and subcategories. Further studies could involve fine-tuning the model on other datasets, as envisioned by the currently disabled "Train Model" feature in the application launcher.
IP Status
Copyright
