HS-ABSA: An Annotated Dataset for Aspect-Based Sentiment Analysis of Home Services Customer Reviews

Abstract

With the progression of new technologies, the explosive growth of user-generated content on online platforms has significantly fuelled interest in Aspect-Based Sentiment Analysis (ABSA) for both academic research and commercial applications. While existing ABSA research often utilizes publicly available product-oriented datasets such as those for laptops and cameras, there remains a scarcity of high-quality, publicly available datasets tailored for service-oriented domains. This gap highlights the need to develop new domain-specific resources that can effectively support the evaluation of ABSA models in real-world customer service contexts. In this paper, we present HS-ABSA, a new benchmark human-annotated dataset specifically curated for aspect-based sentiment analysis of home services customer feedback. The dataset was constructed by scraping reviews from the Google Play Store for a mobile app. The scraped reviews were carefully analysed, filtered, and manually annotated. The annotation process focused on opinion terms, aspect categories, and sentiment polarity. Inter-annotator agreement was measured using Kappa’s score. A total of 4,164 reviews were manually annotated. The inter-annotator agreement was calculated using Kohen’s kappa, and scores of 91.92% for opinion terms, 84.00% for aspect categories, and 94.38% for sentiment polarity were obtained. This newly curated ABSA Dataset for the service industry will provide a robust foundation for evaluating ABSA models.