臺大管理論叢 NTU Management Review VOL.28 NO.3

Use of Implicit User Feedback to Support Semantics-Based Personalized Document Recommendation 98 Table 5 Average Rating of Top Five Recommended Documents Recommendation Treatment Average Rating Standard Deviation Stage 1 Random-based 4.4087 0.7323 Keyword-based 4.8364 0.3106 ECE 5.1000 0.7802 ICE 5.1818 0.7974 Stage 2 Random-based 4.7130 0.5683 Keyword-based 5.1364 0.4424 ECE 5.2538 0.8266 ICE 5.3545 0.7169 4.3.1 The Comparison of Four Treatments We employed mixed model ANOVA to measure rating differences among the four treatments (Table 6). The result shows that both recommendation technique and stage have significant effects. A post hoc test was conducted to determine the differences among the four treatments. As Table 7 shows, the average rating of recommended documents within the random-based recommendation technique group is significantly lower than that of the other three groups. This finding indicates that the other three recommendation techniques (ICE, ECE, and keyword-based) can successfully recommend documents that are related to users’ interests. On the other hand, the average rating of Stage 2 is higher than that of Stage 1. This result indicates that subjects had greater interest in the documents recommended in Stage 2. This study also conducted a t-test to compare subjects’ average perception of recommendation quality between Stages 1 and 2. The results show that the average perception of recommendation quality in Stage 2 is higher than that of Stage 1 ( p < 0.001). This implies that recommendation quality increases as users continue to use the recommender system. Thus, the learning effect is significant for both the recommendation system and the subject.