Keywords

1 Introduction

Social network sites (SNS, e.g. Facebook, Twitter, Instagram, Pinterest, Snapchat, YouTube, etc.) are specific kind of Web 2.0 applications that are transforming how people connect and communicate with each other and have become a significant phenomenon in human interaction [5]. This is because SNS enable users to become visible and facilitate building connections between individuals [5]. Today, SNS are being used in many areas of human activity. To begin with, social media and SNS offer different advantages to various organizations including increased sale, facilitated word-of-mouth communication, sharing business information and providing social support to consumers [10]. Many companies are therefore investing heavily in electronic word-of-mouth (eWOM) with respect to SNS. Tien et al. [28] found that perceived persuasiveness, perceived informativeness and source expertise have an impact on the usefulness of eWOM in the context of SNS. Furthermore, SNS have become important platforms for governments and have influenced how they communicate with citizens [27]. In educational ecosystem, instructors are employing SNS as a mean for connecting with students thus creating digital classroom [9] which stimulates and encourages informal learning [3]. According to study carried out by Lovari and Giglietto [16] in 2012, nearly half of the universities in Italy had an official presence on at least one social media. Kumar and Nanda [12] proposed the framework for integrating specific channels of social media into different processes at higher education institutions. They argue that social media are improving communication between students and other relevant stakeholders being involved in educational process and in the same time are assisting in promotional and developmental activities of higher education institutions such as attracting new students, facilitating their life at campus and maintaining communication with them after they leave the campus.

The quality of SNS is one of main component which significantly affects their adoption by different stakeholders of interactive learning ecosystem and thus their success. Chan et al. [5] emphasize that early studies were mostly focused on the adoption and initial use of SNS but nowadays they are concentrated on identifying motivational factors (e.g. meeting new people, entertainment, maintaining relationships, learning about social events, sharing media, status updating, etc.) that encourage the aforementioned stakeholders to use them. Success of SNS can be generalized into two aspects: (i) the user experience (UX) that the platform offers and (ii) the quality of the content [26]. According to Orehovački et al. [21], there are two major facets of quality in use when Web 2.0 applications are taken into account: (i) usability which refers to the product centered evaluation of pragmatic attributes by means of objective measuring instruments and (ii) user experience (UX) which denotes the use of subjective measuring instruments for the assessment of hedonic attributes.

Objective of the work presented in this paper is to examine hedonic quality of four SNS and quality of content published on them. The remainder of the paper is structured as follows. Theoretical background to our study is provided in the next section. Research methodology is discussed in the third section. Study findings are reported in the fourth section. Conclusions are drawn in the last section.

2 Related Work

The large number of people in the world are using SNS to meet and help each other, to exchange opinions, or to have fun [6]. It is therefore of great importance that facets of user experience are examined before and after development of SNS [6]. In their study, Ou et al. [23] confirmed the significant role of information quality, system quality and networking quality in determining success of SNS (e.g. Facebook and Twitter). Kim and Kim [11] found that respondents who are more gratified with entertainment and need for recognition aspects are those who are using Facebook while respondents who are gratified with the browsing aspect of gratifications and those who have positive attitudes toward levels of openness are those who are using Instagram. According to Sawalha et al. [27], performance expectancy, social influence, effort expectancy, personal innovativeness and enjoyment significantly affect the continuous use of e-government pages on Facebook. Leong et al. [14] discovered that perceived task-technology fit is the great predictor of perceived usefulness and users’ intention to adopt mobile SNS for pedagogical purposes. Results of the study conducted by Chan et al. [5] indicate that perceived critical mass, social presence, and social norms are influential and major factors that determine continuance intention of SNS. Makkonen and Siakas [17] identified understanding of content sharing as a prerequisite for planning the learning/teaching activities based on social media. Pöyry et al. [24] discovered that there is a difference between consumers’ hedonic (to participate in the community) and utilitarian (to browse the community page) motivations for using SNS (e.g. travel agency’s Facebook page). Based on the results of their study, Orehovački and Babić [19] concluded that recoverability, loyalty, reliability, attitude towards use, response time, customizability, adaptability, pleasure, understandability, navigability, aesthetics, error prevention, recoverability, reliability, interactivity and memorability are the most relevant determinants of perceived mobile quality in the context of social Web applications designed for collaborative writing. Aladwani [1] found that content quality in the context of using social media affects customers’ continued interest, active confidence, and feedback openness. Ali et al. [2] discovered that information quality motivates usage intention but does not affects students’ academic use of social media. Rodríguez-Ardura and Meseguer-Artola [25] uncovered perceived playfulness as the essential driver of Facebookers’ experience and perceived usefulness as the major predictor of adopting social media for learning and academic purposes. The results of the study conducted by Nedra et al. [18] have shown that perceived pleasure, social identity (cognitive, affective and evaluative) and perceived ease of use have the positive impact on the intention to use Instagram whereas perceived usefulness have not shown significant influence in that respect. The outcomes of the study conducted by Lee and Kim [13] revealed that system quality, service quality, and hedonic value have an impact on flow experience, information and service quality, and hedonic value influence the relationship quality of SNS whereas the flow experience and relationship quality while using SNS affect the continuance usage intention of SNS. Wu and Chen [29] found that social influence and information quality contribute to users’ continuance intentions to use Facebook in educational context. Finally, the results of the systematic literature review on research related to SNS in the field of information systems carried out by Cao et al. [4] uncovered lack of studies concerning human-computer interaction context.

All the aforementioned indicates that current studies are predominantly focused on exploring pragmatic aspects of quality and adoption of SNS while studies related to both hedonic and content quality are rather scarce. Therefore, we inititated a study focused on exploring hedonic and content dimensions of quality with respect to SNS. Details on the proposed methodology are provided in the following section.

3 Methodology

Research Framework.

The quality model designed for the purpose of this study is composed of two distinctive dimensions: hedonic quality and content quality. Hedonic quality (HDQ) refers to six attributes meant for measuring facets of user experience with respect to interaction with SNS [19, 21]. Aesthetics (AES) denotes the degree to which the SNS has visually appealing user interface. Uniqueness (UNQ) signifies the extent to which the SNS is distinctive among applications with the same purpose. Playfulness (PLY) represents the degree to which the interaction with the SNS is focused and stimulates their curiosity. Pleasure (PLS) indicates the extent to which the employment of SNS arouses users’ emotional responses. Satisfaction (STF) denotes the degree to which users are content with using the SNS. Loyalty (LOY) signifies the extent to which users have the intention to continue to use the SNS and recommend it to others. Content quality (CNQ) refers to five attributes designed for measuring the degree to which the content that the SNS provides is suitable for a specific goal in a defined context [20, 22]. Accuracy (ACU) indicates the extent to which content offered by SNS is correct, valid and free of errors. Credibility (CDB) denotes the degree to which content provided by SNS is unbiased, trustworthy and verifiable. Coverage (CVG) implicates the extent to which content published on SNS is appropriate, complete and compactly represented. Timeliness (TLS) represent the degree to which content offered by SNS can be supplemented, modified, and updated. Added value (ADV) signifies the extent to which content published on SNS is advantageous for users.

Procedure.

The study was conducted in a controlled lab conditions and was composed of evaluating hedonic quality and content quality of four social networks (Facebook, Instagram, Twitter, and YouTube). Upon arriving to the lab, the participants were welcomed and briefly informed about the quality evaluation study. After completing the questionnaire, study subjects were debriefed and thanked for their participation. The duration of the study was twenty minutes.

Apparatus.

The study adopted a within-subjects design comparing four SNS. Data was collected by means of the questionnaire which was administrated online by means of the Google Forms questionnaire builder. The questionnaire was composed of 7 items related to participants’ demography, 30 items meant for exploring hedonic quality and 16 items designed for examining content quality in the context of SNS. Responses to the questionnaire items were modulated on a five point Likert scale (1- strongly disagree, 5 – strongly agree). Each attribute was measured with between three and six items. For the purpose of data analysis, quality attributes and categories were operationalized as composite subjective measures. Values for quality attributes were estimated as a sum of responses to items that are assigned to them. The same holds for perceived quality (PCQ) whose value was estimated as sum of all items meant for measuring facets of hedonic quality (HDQ) and content quality (CNQ). Overall quality (OVQ) was assessed directly by a six-point item (0 - cannot evaluate, 1 - insufficient, 5 - excellent). The internal consistency of quality attributes was tested with Cronbach’s Alpha [8] coefficient values. After the reliability of questionnaire was determined, it was necessary to examine its validity. However, since there were no previously validated measuring instruments for evaluating perceived hedonic quality and content quality of SNS, it was not possible to conduct predicted or concurrent evaluation to obtain a quantitative measure of validity. As an alternative to measuring the validity of a measuring instrument, Lewis [15] suggested the inspection of variables that systematically influence the questionnaire. In this study those variables were four SNS. The sensitivity of measuring instrument, in terms of examining differences between evaluated SNS, was explored by means of the Friedman’s ANOVA. The rationale behind the choice to employ this nonparametric equivalent to the one-way ANOVA with repeated measures draws on the outcomes of Shapiro-Wilk Tests which revealed that at least one of the variables in a pairwise comparison violates the assumption of normality in data (p < .05). In that respect, all the reported results are expressed as the median values. To identify where the differences actually occur, we ran separate Wilcoxon Signed-Rank Tests on the different combinations of SNS being assessed. With an aim to declare a result significant and avoid a Type I error, we applied a Bonferroni correction on the results obtained from the Wilcoxon Signed-Rank Tests. The Bonferroni correction was calculated by dividing significance level of .05 by number of comparisons. The effect size (r) is an objective measure of the importance of effect. It was estimated by dividing Z-value by square root of number of observations. According to Cohen [7], the values of .10, .30, and .50 denote small, medium, and large effect size, respectively.

4 Results

Participants.

A total of 322 respondents took part in the study. They ranged in age from 18 to 49 years (M = 21.23, SD = 4.318). The sample was composed of 66.15% male and 33.85% female students. At the time when the study was conducted, 59.32% of study subjects were enrolled to one of the undergraduate study programs at Polytechnic of Rijeka (POLYRI) whereas remaining 40.68% were students at the Faculty of Informatics, Juraj Dobrila University of Pula (FIPU). Majority of respondents (85.09%) were full-time students whereas 63.98% of them were in their first year of study. They had been using regularly almost all SNS that were the subject of an evaluation. More specifically, 57.77% and 62.73% of study participants had used Facebook and Instagram, respectively, at least 4 h a week, 9.93% of them had used Twitter at least up to one hour a week while 59.32% of respondents had used YouTube at least 11 h a week. When the ofteness of employing those SNS was considered, it appeared that 52.17%, 57.45%, and 64.97% of students had used Facebook, Instagram, and YouTube, respectively, at least three times a day while 10.85% of them had used Twitter at least once a week. The aforementioned findings indicate that the most popular social networking site among students is YouTube followed by Instagram, Facebook, and Twitter.

Measuring Instrument Reliability.

The Cronbach’s alpha values presented in Table 1 ranged from .639 (for attribute added value in the case of YouTube) to .955 (for attribute loyalty with respect to Instagram), thus indicating a high reliability of the scale in the context of exploratory study for all four evaluated SNS.

Table 1. Internal reliability of the scale for social networking sites (SNS)
Full size table

Findings.

The outcomes of data analysis indicate that evaluated SNS differ significantly (χ2(3) = 199.863, p < .001) with respect to the accuracy of content they are providing. Post hoc analysis with Wilcoxon Signed-Rank Tests was conducted with a Bonferroni correction applied, resulting in a significance level set at p < .008. Medium in size differences were discovered between YouTube and Twitter (Z = −10.654, p = .000, r = −.42) as well as between YouTube and Facebook (Z = −8.005, p = .000, r = −.32) while differences between Instagram and Twitter (Z = −7.167, p = .000, r = −.28), YouTube and Instagram (Z = −6.911, p = .000, r = −.27), Facebook and Twitter (Z = −5.615, p = .000, r = −.22), and Instagram and Facebook (Z = −3.166, p = .002, r = −.13) appeared to be small in size. Friedman’s ANOVA revealed a significant difference (χ2(3) = 177.775, p < .001) among four SNS in the context of content credibility. As a follow up for this finding, a post hoc analysis with the significance level set at p < .008 was applied. The difference in terms of the extent to which particular social networking site contains unbiased content was found between YouTube and Twitter (Z = −10.047, p = .000, r = −.40), YouTube and Facebook (Z = −8.437, p = .000, r = −.33), and between YouTube and Instagram (Z = −7.798, p = .000, r = −.31) was medium in size. In addition, post hoc analysis uncovered small in size difference between Instagram and Twitter (Z = −5.695, p = .000, r = −.22), Facebook and Twitter (Z = −4.005, p = .000, r = −.16), and between Instagram and Facebook (Z = −2.679, p = .007, r = −.11). The degree to which content is perceived by study participants as complete is significantly (χ2(3) = 279.407, p < .001) affected by the social networking site they are using. Wilcoxon tests were used to follow-up this finding. A Bonferroni correction was applied and all effects are reported at a .01 level of significance. There was no significant difference between Instagram and Facebook (Z = −.870, p = .380). However, medium in size difference was found between YouTube and Twitter (Z = −12.443, p = .000, r = −.49), Instagram and Twitter (Z = −9.442, p = .000, r = −.37), Facebook and Twitter (Z = −8.733, p = .000, r = −.34), and between YouTube and Facebook (Z = −7.776, p = .000, r = −.31) while difference between YouTube and Instagram (Z = −7.228, p = .000, r = −.29) was small in size. A statistically significant difference (χ2(3) = 136.430, p < .001) regarding content timeliness was found among four SNS that were involved in the study. The follow-up tests with significance level set at .01 revealed medium in size differences between Facebook and Twitter (Z = −10.206, p = .000, r = −.40) and between YouTube and Facebook (Z = −7.861, p = .000, r = −.31) as well as small in size differences between Instagram and Facebook (Z = −7.179, p = .000, r = −.28), Instagram and Twitter (Z = −5.558, p = .000, r = −.22), and between YouTube and Twitter (Z = −4.739, p = .000, r = −.19). However, it appeared that YouTube and Instagram do not differ significantly (Z = −1.415, p = .157) when the degree to which content they provide can be modified is concerned. Data analysis also uncovered a significant difference (χ2(3) = 363.350, p < .001) among evaluated SNS with respect to the extent to which users believe that content SNS are offering is beneficial for them. To follow-up this finding, Wilcoxon tests with Bonferroni correction were applied. The large in size difference was found between YouTube and Twitter (Z = −13.378, p = .000, r = −.53), medium in size differences were identified between YouTube and Instagram (Z = −11.062, p = .000, r = −.44), Facebook and Twitter (Z = −9.864, p = .000, r = −.39), YouTube and Facebook (Z = −9.699, p = .000, r = −.38), and between Instagram and Twitter (Z = −7.884, p = .000, r = −.31) while difference between Instagram and Facebook (Z = −4.543, p = .000, r = −.18) appeared to be small in size. The reported results were at p < .008 significance level. All the aforementioned contributed to significant difference (χ2(3) = 292.334, p < .001) among evaluated SNS when their content quality is taken into account. More specifically, the large in size difference was determined between YouTube and Twitter (Z = −13.855, p = .000, r = −.55), differences between Facebook and Twitter (Z = −10.094, p = .000, r = −.40), YouTube and Instagram (Z = −9.619, p = .000, r = −.38), Instagram and Twitter (Z = −9.241, p = .000, r = −.36), and between YouTube and Facebook (Z = −8.564, p = .000, r = −34) appeared to be medium in size whereas difference between Instagram and Facebook (Z = −1.881, p = .060) in the context of content quality was not significant. All reported findings were at p < .01 level of significance.

There was a significant difference among all SNS that took part in the study with respect to the level to which users perceive their interface as visually attractive (χ2(3) = 343.151, p < .001). With a significance level set at p < .008, post hoc analysis with Bonferroni correction yielded large in size difference between YouTube and Twitter (Z = −12.804, p = .000, r = −.51), medium in size differences between Instagram and Twitter (Z = −11.352, p = .000, r = −.45), Facebook and Twitter (Z = −9.411, p = .000, r = −.37), and between YouTube and Facebook (Z = −7.675, p = .000, r = −.30), and small in size difference between Instagram and Facebook (Z = −4.693, p = .000, r = −.19) and between YouTube and Instagram (Z = −3.215, p = .001, r = −.13). When the degree to which particular social networking site is perceived as exceptional was considered, Friedman’s ANOVA revealed significant difference (χ2(3) = 308.551, p < .001) among all four SNS that were examined in the study. Results of post hoc analysis are divided in three different groups. Large in size difference was determined between YouTube and Twitter (Z = −13.073, p = .000, r = −.52). Medium in size differences were discovered between Instagram and Twitter (Z = −9.821, p = .000, r = −.39), YouTube and Facebook (Z = −9.349, p = .000, r = −.37), Facebook and Twitter (Z = −9.146, p = .000, r = −.36), and between YouTube and Instagram (Z = −8.550, p = .000, r = −.34). Difference between Instagram and Facebook (Z = −2.031, p = .042) was not significant. All reported results were at p < .01 significance level. The results of data analysis are also implying significant difference (χ2(3) = 472.711, p < .001) among examined SNS with regard to the extent to which users are immersed when interacting with them. To follow up on this finding, Wilcoxon Signed-Rank Tests with a Bonferroni correction were conducted. With a significance level set at p < .008, large in size differences were identified between YouTube and Twitter (Z = −14.921, p = .000, r = −.59) and between Instagram and Twitter (Z = −12.677, p = .000, r = −.50), medium in size differences were found between YouTube and Facebook (Z = −12.315, p = .000, r = −.49), Facebook and Twitter (Z = −10.897, p = .000, r = −.43), and between YouTube and Instagram (Z = −9.600, p = .000, r = −.38), while small in size difference was determined between Instagram and Facebook (Z = −5.625, p = .000, r = −.22). Friedman’s ANOVA revealed a significant difference (χ2(3) = 472.711, p < .001) among four SNS in terms of the degree to which study participants are enjoying when employing them. As a follow up to this finding, a post hoc analysis with the significance level set at p < .008 was applied. Identified difference between YouTube and Twitter (Z = −14.453, p = .000, r = −.57) was large in size, differences between Instagram and Twitter (Z = −12.428, p = .000, r = −.49), Facebook and Twitter (Z = −11.845, p = .000, r = −.47), YouTube and Facebook (Z = −10.547, p = .000, r = −.42), and between YouTube and Instagram (Z = −8.069, p = .000, r = −.32) were medium in size whereas difference between Instagram and Facebook (Z = −3.543, p = .000, r = −.14) was small in size. A significant difference (χ2(3) = 476.800, p < .001) among examined SNS with respect to the degree to which they have met users’ expectations was also discovered. Bonferroni pairwise comparisons with a significance level set at p < .008 uncovered large in size difference between YouTube and Twitter (Z = −14.800, p = .000, r = −.58), medium in size differences between Instagram and Twitter (Z = −12.203, p = .000, r = −.48), Facebook and Twitter (Z = −11.749, p = .000, r = −.46), YouTube and Facebook (Z = −11.547, p = .000, r = −.46), and between YouTube and Instagram (Z = −8.908, p = .000, r = −.35), and small in size difference between Instagram and Facebook (Z = −3.847, p = .000, r = −.15). A significant value of chi square (χ2(3) = 526.792, p < .001) indicates the existence of differences among examined SNS with regard to the degree to which study participants are their loyal consumers. A post hoc procedure with the significance level set at p < .01 revealed large in size difference between YouTube and Twitter (Z = −15.002, p = .000, r = −.59) as well as medium in size differences between Facebook and Twitter (Z = −12.477, p = .000, r = −.49), Instagram and Twitter (Z = −12.350, p = .000, r = −.49), YouTube and Facebook (Z = −11.080, p = .000, r = −.44), and between YouTube and Instagram (Z = −9.651, p = .000, r = −.38). However, no significant difference was found in that respect between Instagram and Facebook (Z = −.762, p = .446). The reported findings resulted in significant difference (χ2(3) = 519.561, p < .001) among evaluated SNS with respect to their hedonic quality. Wilcoxon tests were used to follow-up this finding. A Bonferroni correction was applied and all effects are reported at a p < .008 level of significance. Five large in size (between YouTube and Twitter (Z = −15.322, p = .000, r = −.60), Instagram and Twitter (Z = −13.215, p = .000, r = −.52), YouTube and Facebook (Z = −13.152, p = .000, r = −.52), Facebook and Twitter (Z = −12.649, p = .000, r = −.50) and between YouTube and Instagram (Z = − 10.261, p = .000, r = −.40)) and one small in size (between Instagram and Facebook (Z = −4.773, p = .000, r = −.19)) difference were determined in that respect.

A significant difference (χ2(3) = 543.219, p < .001) among examined SNS was found with respect to single-item subjective measure designed for evaluating overall quality. Large in size differences were discovered between YouTube and Twitter (Z = −15.021, p = .000, r = −.59), YouTube and Facebook (Z = −13.371, p = .000, r = −.53), Facebook and Twitter (Z = −13.038, p = .000, r = −.51), and between Instagram and Twitter (Z = −12.759, p = .000, r = −.50), difference between YouTube and Instagram (Z = −9.636, p = .000, r = −.38) was medium in size, while difference between Instagram and Facebook (Z = −4.202, p = .000, r = −.17) appeared to be small in size.

Drawing on findings related to hedonic quality and content quality, significant difference (χ2(3) = 512.102, p < .001) among SNS that were involved in the study in terms of their perceived quality was determined. More specifically, large in size difference was identified between YouTube and Twitter (Z = −15.383, p = .000, r = −.61), Instagram and Twitter (Z = −12.980, p = .000, r = −.51), YouTube and Facebook (Z = − 12.909, p = .000, r = −.51), and between Facebook and Twitter (Z = −12.647, p = .000, r = −.50), medium in size difference was uncovered between YouTube and Instagram (Z = −10.774, p = .000, r = −.43), while difference between Instagram and Facebook (Z = −3.736, p = .000, r = −.15) was small in size. All reported findings are summarized in Table 2.

Table 2. Outcomes of data analysis (note that a higher score means a better result)
Full size table

5 Conclusion

The aim of the work presented of this paper was to examine the validity of the measuring instrument designed for evaluating hedonic and content related dimensions of quality in the context of social networking sites (SNS). For that purpose, an empirical study was conducted during which psychometric features of introduced questionnaire were confirmed. More specifically, the Cronbach’s Alpha exceeded the .70 cut-off value for all quality attributes in the case of all four SNS involved in the study (with an exception of added value attribute in the case of YouTube) thus indicating the high level of inter-item consistency reliability. When the validity of the questionnaire was tackled, study findings suggested that all proposed quality attributes can be employed for determining significant differences among SNS. Taking into account attributes aimed for examining hedonic quality and content quality separately, they managed to discover 12.12% large in size, 56.06% medium in size, and 26.76% small in size differences among four SNS included in the study. The highest deal (33.33%) of large in size differences was discovered by means of the attribute designed for measuring perceived playfulness. Majority of medium in size differences was determined with two attributes associated with content quality (coverage and added value) and four attributes assigned to hedonic quality (uniqueness, pleasure, satisfaction, and loyalty). The highest amount of small in size differences was discovered with an attribute meant for measuring content accuracy. At least one non-significant difference among examined SNS was found with two attributes designed for measuring content quality (coverage and timeliness) and two attributes aimed for evaluating hedonic facets of quality (uniqueness and loyalty). While differences yielded by content quality as a composite measure were mainly medium in size, differences identified by hedonic quality in the same respect were mostly large in size. Both overall quality and perceived quality determined the same amount of large (66.66%), medium (16.67%), and small (16.67%) in size differences. All the aforementioned indicates that practitioners can use proposed questionnaire as an instrument for measuring quality of existing SNS as well as set of guidelines when designing new SNS. Considering that the study carried out for the purpose of this paper is empirical one, it has several limitations. The first one is related to homogeneity of participants. Although students can be perceived as representative users of SNS since they can be used in educational ecosystem, heterogeneous group of study respondents could have provided different answers to questionnaire items related to particular attributes of content quality and hedonic quality. The second limitation deals with the interpretation of reported findings because they are generalizable only to SNS included in our study. Taking the above into account, in order to draw generalizable sound conclusions and to examine the robustness of reported findings, further studies should be conducted.