AI tech company Crowdworks announced on the 30th that it had won a contract for Meritz Fire & Marine Insurance's AI agent performance evaluation project.
This project aims to evaluate the performance of Meritz Fire & Marine Insurance's AI-powered sales support service for insurance designers in a real-world work environment and support service advancement by improving quality and stability. This AI service will learn insurance terms, coverage, and terminology to support designers' insurance design work.
For this project, Crowdworks will focus on evaluating AI agent responses. It plans to build an expert-based evaluation dataset to comprehensively verify response accuracy, task success rates, and reliability. The evaluation process will utilize Crowdworks' proprietary AI evaluation and verification solution, Alpy Evaluation. This solution can evaluate performance across a variety of areas, including LLM, RAG, and AI agents, and also includes features to prevent the creation of harmful content and bias.
Specifically focused on building customized evaluation datasets for the insurance industry, data experts with insurance industry experience design Q&A data based on actual designer consultation scenarios.
Kim Woo-seung, CEO of Crowdworks, said, “When evaluating AI agent performance, the sophistication of the evaluation question design is more important than the logic technology,” and added, “We will improve the quality management level of AI services in the financial sector by combining insurance domain-specific data with an automated evaluation system.”
- See more related articles
You must be logged in to post a comment.