This feature is designed to enhance the review experience by providing concise, AI-generated summaries of all published reviews, along with keywords. This feature allows users to quickly grasp the overall sentiment of customer feedback without having to read through each individual review and offers customers a quick, clear overview of the most important takeaways from customer reviews.
01
Reading through hundreds of reviews can be overwhelming for both potential customers and merchants. This can lead to information overload, where important feedback is either missed or not given enough attention. For merchants managing large product catalogs, it’s also time-consuming to assess review patterns across different products, making it difficult to identify common issues or strengths at a glance.
02
The AI Reviews Summary feature addresses these challenges by automatically generating brief, easy-to-read summaries of all reviews, highlighting key themes and keywords.
03
Research
From internal data and benchmarking competitors we understood it was an important feature to have on our widgets list to continue to be competitive and improve both merchants and reviewers experience. By adding this feature to our list we would improve user engagement and reduce information overload while also improving clarity and relevance.
04
Idea conception
On the images above we can see an example of the Amazon review summary feature and Yotpo examples. Our approach was more similar to the Amazon one for a few reasons: We wanted to keep the build time to a minimum and develop the feature in the most optimised way, on the Yotpo approach users have to click on a button to generate the summary which generates a sentiment for each keyword. This approach is more costly and takes more dev time to implement without offering major benefits comparing to the Amazon approach that is out-of-the-box.
05
Pricing structure
As the business strategy was not to raise the price of the paid plan, we had to be careful while developing this feature as all the AI costs had to be absorbed by the current price. This helped us made the decision that merchants are the ones that need to generate the summary and not the reviewers. This approach reduces the flux of input and output tokens used. For example Yotpo offers this feature on a custom quote basis, which we wanted to run away from.
06
Testing the LLM's
The LLM we choose was Claude 3 Sonnet, after testing multiple prompts using other models we got to a conclusion that this was the best in terms of output, accuracy and price. The output of the prompt had to be on a Json format as it offers several advantages like readability, UI rendering and flexibility.
07
How it works
The AI model automatically reads through all the reviews and pull out the most important points. It looks for key themes and common keywords, then creates a short, easy-to-read summary that captures the overall sentiment. Plus, the merchants can keep updating the summaries every 30 days as new reviews come in, so the information is always fresh and relevant.
08
The challenges
The reviews widget settings page was the natural place to add the AI review summary feature, but we faced a challenge: the settings page was already packed with options. We needed to design the AI summary settings in a way that it would fit seamlessly without overwhelming merchants. To do this, we focused on a Lean UX approach. This helped both, the merchants and our developers. Our goal was to make it as easy as possible for merchants to activate the AI feature, so they could get up and running quickly without any hassle. We also wanted to lighten the development load, so we could focus on delivering more features.
09
Merchants can effortlessly generate and activate the AI review summary with just one click. Once they decide to generate the summary, the system automatically processes the selected range of shop reviews and creates a concise summary in real time. There’s no need for any complex setup or additional manual input, merchants simply click a button, and the summary is published on their Reviews Carousel widget. This approach allows them to focus their time on what truly matters, growing their business.
