AI Segmentation & Sentiment Analysis
PRODUCTION FEATURE
IMPACT - HIGH
ADOPTION - HIGH
Pain Point
Clients with hundreds of reviews lacked visibility into what customers were talking about majorly.
Managers only had star ratings.
Difficult to think of corrective actions to be taken from reviews.
Source of Discovery
Client Interactions
User Personas
Regional Managers, Area Managers, Store Managers
Why we Prioritized it
Recurring theme from client interviews.
High impact on client-side outcomes.
01. Rule based keyword tagging was inefficient.
02. Hard to scale the rules across industries with varying segments.
03. NLP is a proven solution for segment & sentiment analysis.
Why AI was the best solution.
MVP & Pilot
Piloted with 2 Enterprise clients.
Built MVP extracting segments and assigning sentiment (negative, neutral, positive)
.png)
API Cost Challenges
MVP used OpenAI API [GPT-4o mini] - accuracy was 76.9% but API costs were unsustainable.
MODEL EVALUATION
Deployed another open-source NLP model from Hugging Face.
Evaluated performance between GPT-4o mini output and the open-source.
Total Pilot reviews
~7000
​
Sample size
~200
Sampling
​
Distributed across locations.
​
Sentiment mix to avoid bias
​​​​Target class
Negative
i.e. the objective is to identify negative sentiment correctly
Confusion matrix
Evaluation Scores
*Actual figures are masked to retain confidentiality
Tuned sentiment classification thresholds (-0.2 / +0.2) to reduce misclassifications and increase client trust.
Trade-off & Decision
Chose open-source for scale.
Focused on improvements on false negatives.
Outcomes
in Production
Client Outcomes
Average reputation score increased from Fair to Good.
​
Segment-level visibility enabling data backed decisions for the users.
Product Outcomes
Drove ~70% enterprise adoption of specific Analytics features.
Service Engagement increased with ~50% of the clients.
Current Challenges
Industry level
Segments
To check for segmentation accuracy as per the domain, manual sample monitoring is required.
Explainability on Sentiment
Especially for sensitive industries like Healthcare, manual evaluation with sampling is required.
Continuous Improvement
Through User feedback feature (like/dislike for sentiment & segment).
User input by updating sentiment & segment.
Manual evaluation.
AI REVIEW RESPONDER
MEDIUM IMPACT | HIGH ENGAGEMENT
PROBLEM SOLVED
​​
Numerous reviews went unanswered, lowering Response Rate & TAT.
OUTCOMES​​
​
TAT reduced from days to hours.
Response Rate increased ~50% to ~85%
