臺大管理論叢 NTU Management Review VOL.28 NO.3

99 NTU Management Review Vol. 28 No. 3 Dec. 2018 Table 6 Analysis Results of Mixed Model ANOVA (Four Treatments) Source Sum of Square df Mean Square F Sig. Recommendation Technique 13.6706 3 4.5569 6.014 0.0009 Stage 2.5069 1 2.5069 15.872 0.0001 Recommendation Technique x Stage 0.2289 3 0.0763 0.483 0.6948 Subjects 67.4369 89 0.7577 Error 14.0567 89 0.1579 Total 97.9000 185 Table 7 Post-Hoc Test between Recommendation Techniques (I-J) J I Random-based Keyword-based ECE Keyword-based 0.4255* (0.1836) ECE 0.6161*** (0.1762) 0.1906 (0.1783) ICE 0.7073*** (0.1836) 0.2818 (0.1856) 0.0913 (0.1783) Note: 1. The value in parentheses is the standard deviation. 2. * p < 0.05, *** p < 0.001 4.3.2 The Comparison of Three Recommendation Techniques Since all three personalized recommendation techniques (ICE, ECE, and keyword- based) recommended documents based on the subjects’ top three interest ratings in Stage 0, the recommendation system’s learning effect was controlled by setting the average interest rating score of these three documents as the covariate variable (Top 3 rating). Mixed model ANCOVA (Analysis of Covariance) was conducted to test for rating differences among these three techniques and recommendation stage. As Table 8 shows, the covariate variable is significant ( p < 0.001), and the effect of the recommendation technique is significant but the effect of the stage is not. The results of post hoc testing indicate that the average score of the keyword-based group is significantly lower than that of the ICE group ( p < 0.05). However, there is no significant difference in the average scores of the ECE and ICE treatments. There is also no significant interaction effect between recommendation stage and recommendation technique.