Are All Reviews Equal?

When choosing a restaurant on Kakao Map, the first thing most people look at is the star rating. If a place has a 4.8 rating and 500 reviews, most diners decide to visit without much hesitation.

Naver abolished restaurant star ratings in 2023. And with Kakao introducing payment verification and on-site photo verification, it is clear that both platforms recognize the limits of star ratings.

So how accurate are Kakao Map ratings in reality? We tested their reliability by conducting a full analysis of 1.75 million reviews from 84,736 restaurants in Seoul and Gyeonggi.

For this analysis, we introduced the concept of a 'Gold reviewer.' These are reviewers on Kakao Map who have written at least 50 reviews and whose average rating falls between 2.5 and 4.2—balanced evaluators who are neither overly generous nor excessively harsh. Using these Gold reviewers as the benchmark, we assigned reliability weights to all reviewers and calculated a weighted positive rate. A score of 75% or higher was classified as great restaurant, 50% or higher as decent, and below 30% as caution.

Applying this standard to 8,248 restaurants with a Kakao rating of 4.8 or higher, 98.9% were classified as great restaurants. Only 1.1% were rated decent, and just 0.02% fell into average or below. All 2,995 restaurants with a perfect 5.0 rating were classified as great restaurants.

If you choose a restaurant with a Kakao rating of 4.5 or higher, the probability of it being judged average or below is 0.39%. At 4.8, that drops to 0.02%. By the numbers alone, Kakao star ratings appear to be highly accurate.

So does that mean there is no deception in star ratings at all? A deeper look at the data tells a different story. The problem was not the star rating itself, but the quality of the reviewers who make up that rating.

In general, people assume that the more reviews a place has, the more accurate its rating must be. Intuitively, 500 reviews should be more trustworthy than 30. But the data shows a different pattern.

We compared the share of cases where the gap between the Kakao rating and the weighted analysis score exceeded 0.5 points—in other words, the 'rating error rate'—by grouping restaurants not by total review count, but by the number of Gold reviewers.

Among restaurants with not a single Gold reviewer, Kakao ratings were off by 0.5 points or more in 1 out of 3 cases (33.6%). By contrast, among restaurants with 20 or more Gold reviewers, that figure fell to just 1.4%. This quantitatively shows that what determines rating accuracy is not the quantity of reviews, but the quality of the reviewers.

In fact, even restaurants with 500 reviews showed low rating reliability if they had only one or two Gold reviewers, while places with just 50 reviews but 10 or more Gold reviewers had markedly lower error rates.

In this analysis, the level of rating inflation was divided into three stages. If the share of non-discriminating reviews exceeded 40%, or if the gap between the Kakao rating and the weighted score exceeded 0.5 points, the restaurant was labeled 'caution.' At 20% / 0.3 points, it was labeled 'suspicious.' Everything else was classified as 'clean.'

What stands out is the data for restaurants in the 'clean' category.

For restaurants in the 'clean' category, Kakao ratings were found to be 0.39 points lower than the weighted analysis score. In other words, the tougher, more discerning reviewers were actually more generous than Kakao's simple average. A 'clean' restaurant with a Kakao rating of 3.9 could in reality be closer to a 4.3.

By contrast, in the 'caution' category, Kakao was overestimating by 0.48 points, and the share judged average + caution reached 55%. So even if two restaurants both have a Kakao rating of 4.3, their actual quality can be completely different depending on their bubble grade.

There were 772 restaurants with a Kakao rating of 4.0 or higher that were nevertheless judged average or below. When we break down the gap between their Kakao rating and weighted analysis score by verdict, a consistent pattern emerges.

Restaurants judged to be great showed a gap of just −0.07 points between the Kakao rating and the weighted score—essentially the same level. Even the decent category was within the margin of error at +0.08 points.

By contrast, restaurants judged 'average' showed Kakao overestimating by 1.1 points, while those judged 'caution' showed an overestimation of 1.32 points. In other words, a restaurant with a Kakao rating of 4.2 could in reality be closer to 2.9. What these places had in common was a near absence of Gold reviewers, averaging only 1.7 to 2.8.

Looking at specific cases makes the pattern even clearer.

In the case of 양*** 삼성역점 (Yang*** Samseong Station Branch), the Kakao rating was 4.5, but the weighted analysis score was 1.38—a huge gap of 3.12 points. It had only one Gold reviewer. 판** 수내직영점 (Pan** Sunae Direct Branch) had plenty of data with 573 reviews, but only two Gold reviewers. Even when the number of reviews is high, the meaning of the star rating becomes diluted if there are no trustworthy reviewers behind it.

The common pattern among the 772 restaurants judged average or below is clear: an average of 2.4 Gold reviewers, and a Kakao overestimation of +1.1 points. It reveals a structural problem—the absence of balanced evaluators capable of validating the rating.

We found that Kakao star ratings themselves are not inherently deceptive. The problem lies in a structure where all reviews are reflected equally in the rating. A first-time user with three reviews and an experienced reviewer with more than 200 reviews are contributing to the average with exactly the same weight.

Naver's decision to abolish star ratings, and Kakao's introduction of payment verification and on-site photo verification, appear to stem from the same recognition. The core limitation is not the rating itself, but the absence of a mechanism to evaluate the quality of the reviewers who make up that rating.

The conclusion from analyzing 1.75 million reviews is clear. A Kakao rating of 4.8 was almost always accurate. The distortion did not come from the rating number itself, but from restaurants with high ratings but no Gold reviewers behind them.

For consumers, what matters is not just the rating number itself, but also the makeup of the reviewers who produced it. Instead of simply counting reviews, checking how many experienced and balanced reviewers are included may be the most practical way to avoid being misled by star ratings.