EMBnet.journal

Validation of the Greek version of the Bergen Social Media Addiction Scale in Undergraduate Students

Anastasios Dadiotis1, Flora Bacopoulou2✉, Ioulia Kokka1, Dimitrios Vlachakis2,3,4, George P. Chrousos2, Christina Darviri1, Petros Roussos5

1 Postgraduate Course of Science of Stress and Health Promotion, School of Medicine, National and Kapodistrian University of Athens, Greece

2 University Research Institute of Maternal and Child Health & Precision Medicine and UNESCO Chair on Adolescent Health Care, National and Kapodistrian University of Athens, Aghia Sophia Children’s Hospital, Athens, Greece

3 Laboratory of Genetics, Department of Biotechnology, School of Applied Biology and Biotechnology, Agricultural University of Athens, Athens, Greece

4 Lab of Molecular Endocrinology, Center of Clinical, Experimental Surgery and Translational Research, Biomedical Research Foundation of the Academy of Athens, Athens, Greece

5 Department of Psychology, School of Philosophy, National and Kapodistrian University of Athens, Athens, Greece

Competing interests: AD none; FB none; IK none; DV none; GPC none; CD none; PR none

Introduction

Social Networking Sites (SNSs) are defined as online services that enable individuals to build a public or semi-public profile and give them the opportunity to create a network of contacts to be linked to (Boyd and Ellison, 2007). According to recent data, 3.81 billion of the global population maintain an account in at least one SNS, while worldwide internet users spend 144 minutes per day in SNSs (Global Digital Population, 2020). In Greece, 74.9% of internet users connect to make use of SNSs.

Internet addiction is not included in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-V) as a recognised disorder. However, one of the problematic aspects of internet, the internet gaming disorder (IGD), is described as a condition that requires further study (Petry and O’Brien, 2013; Yau et al., 2012). Several researchers have suggested that excessive use of SNSs should be characterised as addictive behaviour that falls into the broader category of internet addiction (Kuss and Griffiths, 2011; Yau et al., 2012). According to Kuss and Griffiths (2011), under the biopsychosocial model’s scope, SNSs addiction exhibits the common symptoms found in substance abuse. In summary, these symptoms include salience (behavioral, cognitive and emotional obsession with the use of SNSs), mood modification (use of SNSs leads to a favorable change in emotion), tolerance (need for increasing use of SNSs), withdrawal symptoms (unpleasant physical and emotional symptoms when the use of SNSs is limited or discontinued), conflict (intrapersonal and interpersonal problems due to the use of SNSs) and relapse (return to excessive use of SNSs after a period of abstinence) (Kuss and Griffiths, 2011).

Apart from the growing body of research about SNSs addiction, various psychometric instruments have been developed to assess this behaviour, such as the Bergen Facebook Addiction Scale (BFAS) (Andreassen et al., 2012) or the Facebook Intrusion Questionnaire (Elphinston and Noller, 2011). However, these tools focus only on Facebook, just one of the many SNSs (Griffiths, 2012). To resolve this issue, Andreassen et al. (2016) developed the Bergen Social Media Addiction Scale (BSMAS), an adaptation of the aforementioned BFAS and assessed SNSs addiction generally rather than one specific networking platform. Various studies have examined the psychometric properties of BSMAS. The scale has been validated in Italian (Monacis et al., 2017), Iranian (Lin et al., 2017), Hungarian (Bányai et al., 2017) and Chinese (Chen et al., 2020; Yam et al., 2019). According to the literature, addiction to SNSs has been positively correlated with depression (Shensa et al., 2017), stress (Vanman et al., 2018), anxiety (Atroszko et al., 2018), loneliness (Primack et al., 2017) and negatively with self-esteem (Hawi and Samaha, 2017). In addition, various demographic factors such as sex and age, appear to be associated with excessive use of SNSs (Andreassen et al., 2016). SNS addiction has also been linked to the duration of use (Koc and Gulyagci, 2013).

According to Hammen’s stress generation hypothesis (1991), depressed individuals or individuals at risk of developing depression are influenced by their personal characteristics, beliefs and behavioural expectations in ways that will lead to stressful events in their lives. Therefore, individuals vulnerable to depression are more likely to experience more stressful events than less vulnerable individuals. The stress generation hypothesis has acquired a rather strong empirical documentation (Liu and Alloy, 2010; Starr et al., 2012).

The main objective of this study was to evaluate the psychometric properties of the Greek version of the BSMAS. In addition, the relationships between SNSs addiction with mental health variables (stress, depression, anxiety), personality variables (self-esteem), biological variables (age, gender) and behavioural variables (SNSs use time and SNSs number) were investigated. Specifically, the mediating role of stress in the relationship between depression and SNSs addiction was also examined.

Materials, Methodologies and Techniques

Participants and Procedure

Undergraduate students of the University of Athens, able to read and write in Greek, were eligible for participation in the study. After being fully informed of the research objectives and giving their written consent, participants were asked to complete a series of questionnaires anonymously. The completion time was approximately 15 minutes.

Translation Procedure

The BSMAS was translated to Greek and then back-translated by independent translators. The back-translation was then compared with the original scale, and adjustments were made as necessary. The final version was administered to eight people who were debriefed after completing the inventory to check for clarity and comprehensibility of the items.

Measures

Questions on time spent and number of SNSs accounts. To evaluate the use of SNSs, questions regarding the hours spent on SNSs per day and the number of platforms in which the participants maintained an account were included. The question on time spent on SNSs was used to examine the criterion validity of the BSMAS. It has been argued that correlation coefficients of 0.30 to 0.40 are considered high (Alexopoulos, 2011).

Bergen Social Media Addiction Scale (BSMAS). The BSMAS consists of six items representing key addiction elements (salience, mood modification, tolerance, withdrawal symptoms, conflict and relapse). Each item is answered on a five-point Likert scale ranging from 1 (very rarely) to 5 (very often). Regarding the structure of BSMAS, a unifactorial model has been identified (Andreassen et al., 2016; Monacis et al., 2017).

Depression Anxiety Stress Scale – 21 items (DASS-21). The DASS-21 consists of three 7-item scales that assess depression, anxiety, and stress. Every item is rated on a four-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time). The Greek version of DASS-21 can be used as a reliable and valid instrument in the Greek non-clinical population (Pezirkianidis et al., 2018). The internal consistency of DASS-21 is acceptable (α= 0.85 for stress, 0.78 for anxiety and 0.87 for depression).

Rosenberg Self-esteem Scale (RSES). It consists of ten items that are answered on a four-point scale from strongly agree to strongly disagree. Five of the items have positively worded statements, and five have negatively worded ones. The scale measures self-worth by measuring both positive and negative feelings about the respondent’s self. Scores between 15 and 25 are considered average. It has been translated and validated in the Greek language (Galanou et al., 2013).

UCLA Loneliness Scale (UCLA). Emotions of perceived loneliness and social isolation were evaluated using UCLA, which is a 20-item instrument. Answers are rated on a four-point Likert scale (Anderson and Malikiosi-Loizos, 1992). The internal consistency of UCLA is acceptable (α = 0.89).

Statistical Analysis

To assess the construct validity of BSMAS, confirmatory factor analysis (CFA) was performed using the maximum likelihood (ML) method. To assess the good-fitting of the model χ2 and its degrees of freedom (df) were used. For χ2 values associated with p>0.5 were considered good-fitting models, although it has to be mentioned that the p-value of this test is sensitive to large sample sizes. Besides, the root mean square error of approximation (RMSEA) with its 90% confidence intervals (CI), the standardised root mean square residuals (SRMR) and the comparative fit index (CFI) were used. For RMSEA values < 0.06 is acceptable while for SRMR values < 0.08. For CFI, values > 0.90 were considered s indicators of good fit (Brown, 2006). The scale reliability was examined using Cronbach’s α. Values > 0.7 were considered acceptable. Also, inter-item correlations were used to investigate the extent to which the items are evaluating the same content. An average inter-item correlation between 0.20 and 0.40 is considered ideal for the items to be homogenous enough while they contain sufficient unique variance (Piedmont, 2014). Finally, the standard error of measurement (SEM) was used to assess how the observed scores wavered from the measurement error. A value ≤ SD/2 is considered acceptable (Wuang et al., 2012). Bivariate correlations were used to investigate the associations between the variables of the study. A multiple regression analysis was employed to examine the predictive associations between the various variables of the study and SNSs addiction. Finally, a mediation analysis was performed to investigate the relationship between depression (independent variable), stress (mediator) and addiction to SNSs (depended variable). The mediation effect was assessed by the bootstrapping procedure (10,000 samples), which provides accelerated confidence intervals (95% CI) (Tabachnick et al., 2019). Statistical significance level was set at p≤0.05. For the statistical analyses SPSS 26 and JAMOVI 1.2.5 were used.

Results

Descriptive statistics

The sample consisted of 325 undergraduate students (mean age 21.6 years, SD=5.26) of the University of Athens, among which 266 (81.8%) women and 59 (18.2%) men. Descriptive statistics of the study sample are presented in Table 1. Regarding the time participants spent daily on SNSs, 50%, 30%, and approximately 13% of the participants spent 1-3, 3-5 and over 5 hours, respectively.

Reliability analysis

The internal consistency of BSMAS was α=0.75. The average inter-item correlation was relatively high (r = 0.33). Finally, the SEM (2.18) was equal to SD/2, which was acceptable according to SEM ≤ SD/2.

Correlation and regression analysis

BSMAS correlated statistically significant with all the study variables. The results are presented in Table 2. The results of the regression analysis are presented in Table 3. The model [F(8; 316)=16.4; p<0.001] suggests that 24% (R2=0.236) of the BSMAS variance can be explained through four statistically significant predictors. As shown in Figure 1, stress completely mediated the relationship between depression and addiction to SNSs (total effect, c: p<0.001; direct effect, c’: p=0.103). The indirect effect (a*b) was statistically significant, b=0.127; p=0.007; 95% Ci [0.0401-0.2222].

Discussion

To our knowledge, this study is the first that evaluated exclusively the SNSs addiction and how it is related to biological, social and psychological variables in a Greek sample of adult University students.

The results of this study confirmed the unifactorial solution as all the indicators of good-fit of the model were excellent and according to previous studies which examined the psychometric properties of BSMAS (Andreassen et al., 2016; Chen et al., 2020; Lin et al., 2017; Monacis et al., 2017; Yam et al., 2019). Besides, to improve the RMSEA index, a covariance path between the error terms in items 1 and 2 was added to the model. This finding is consistent with the studies of Monacis et al. ( 2017) and Yam et al. (2019). According to Monacis et al. (2017), this covariance can be justified by the theory. More specifically, in item 1, which examines the criterion of salience, the demonstrated distress in the behaviour may be related to the increased time of activity required to achieve satisfaction, which is also included in the criterion of tolerance, as expressed in item 2 (Monacis et al., 2017). On the other hand, Bányai et al. and Lin et al. did not report similar findings (Bányai et al., 2017; Lin et al., 2017). This could be explained by the fact that the samples of these studies consisted only of adolescents, while in the present study the sample concerned young adults. It has been suggested that differences in cognitive abilities between adolescents and young adults exist, which may lead to a different interpretation of the BSMAS item. At the same time, intercultural differences may also play a role (Lin et al., 2017).

All correlations were found to be statistically significant regarding the relationships between SNSs addiction and demographic, social and psychological variables. More specifically, positive correlations were found between SNSs addiction and stress, depression, anxiety and loneliness, while negative correlations were observed between SNSs addiction and self-esteem and age. These results are in line with the findings of previous studies (Andreassen et al., 2016; Atroszko et al., 2018; Hawi and Samaha, 2017; Koc and Gulyagci, 2013; Primack et al., 2017; Shensa et al., 2017; Vanman et al., 2018); the size of the correlation was similar to that found by others, which was between r=0.05 and 0.35 (Brailovskaia and Margraf, 2018). The present study supported the biopsychosocial model with empirical data, as it highlighted that biological (age, gender) and psychological (stress, depression, etc.) factors are associated with addiction to SNSs.

Gender, time spent on SNS and number of SNS accounts were found as predictive factors of addiction in this study. Regarding gender, it was found that women are more likely to be addicted to SNSs. This finding was consistent with the results of other validations of the scale (Andreassen et al., 2016; Hawi and Rupert, 2018; Monacis et al., 2017). It has been suggested that this may reflect a preference of women in activities that are characterised by social interaction and cooperation, while men show a preference for aggressive and competitive activities (Andreassen et al., 2016). As previous validations of the scale, this study found that time spent on SNSs was a significant predictor of addiction (Chen et al., 2020; Leong et al., 2019; Lin et al., 2017). In the study of Jafarkarini et al. (2016), time spent on SNSs was the only statistically significant predictor of addiction to SNSs. Also, the value of the correlation between the daily use of SNSs and addiction to SNSs (r=0.36; p<0.001) was considered high. It was found that the higher the number of accounts a user maintains, the more likely it is to become addicted to SNSs. This finding was consistent with the results of Pontes et al. (2018), who found that high-risk individuals maintained more accounts than the individuals which were at lower risk for developing SNSs addiction. In addition, this finding highlighted the importance of the transition from BFAS to BSMAS.

This study’s results, which are in accordance with other studies, highlighted stress as a predictive factor of SNS addiction (Brailovskaia et al., 2018). Besides, according to the mediation analysis, stress was found to fully mediate the relationship between depression and SNSs addiction, which is consistent with Hammen’s (1991) stress generation hypothesis, stating that individuals vulnerable to depression experience more stressful events than non-vulnerable individuals. Subsequently, SNSs function as a stress-coping mechanism for their users. It has been found that vulnerable to depression individuals are more likely to use SNS to relieve stress than non-vulnerable to depression individuals (Brailovskaia et al., 2019). This coping strategy could be helpful in the short term but could increase the risk for addiction to SNSs (Brailovskaia et al., 2018). Furthermore, the increasing use of SNSs leads to various problematic behaviours (e.g. the user ignores intimate personal relationships), which activate a vicious cycle and negatively affect mood.

However, this study has some limitations. A test-retest examination was not held. Secondly, the sample mainly consisted of undergraduate students and primarily women, limiting the generalisation of the findings. Future studies should move in the direction of broader samples with respect to sociodemographic characteristics. Finally, the research data concerned self-report questionnaires, which are vulnerable to reporting bias.

In summary, the present study results indicated that the Greek version of the Bergen Social Media Addiction Scale is a valid and reliable instrument to assess SNSs addiction. Furthermore, it was found that biological (age, gender), psychological (stress, depression, anxiety, loneliness), and personality (self-esteem) variables are associated with addiction to SNSs. Gender, time spent on SNSs, number of accounts an individual maintains in SNSs and stress found to be predictive factors of SNSs addiction.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References

1. Adbe1a27-e2d2-5529-2f50-6872239bbff7.pdf. (n.d.). Retrieved July 15, 2020, from https://www.statistics.gr/documents/20181/adbe1a27-e2d2-5529-2f50-6872239bbff7

2. Alexopoulos D, (2011) Psychometry 2nd edition. Pedio.

3. Anderson L and Malikiosi-Loizos M (1992) Reliability data for a Greek translation of the Revised UCLA Loneliness Scale: Comparisons with data from the USA. Psychological Reports 71, 665–666. http://dx.doi.org/10.2466/PR0.71.6.665-666

4. Andreassen CS, Billieux J, Griffiths MD, Kuss DJ, Demetrovic Z, et al. (2016) The relationship between addictive use of social media and video games and symptoms of psychiatric disorders: A large-scale cross-sectional study. Psychology of Addictive Behaviors 30(2), 252–262. http://dx.doi.org/10.1037/adb0000160

5. Andreassen CS, Torsheim T, Brunborg GS, Pallesen S (2012) Development of a Facebook Addiction Scale. Psychological Reports 110(2), 501–517. http://dx.doi.org/10.2466/02.09.18.PR0.110.2.501-517

6. Atroszko PA, Balcerowska JM, Bereznowski P, Biernatowska A, Pallesen S et al. (2018) Facebook addiction among Polish undergraduate students: Validity of measurement and relationship with personality and well-being. Computers in Human Behavior 85, 329–338. http://dx.doi.org/10.1016/j.chb.2018.04.001

7. Bányai F, Zsila Á, Király O, Maraz A, Elekes Z et al. (2017) Problematic Social Media Use: Results from a Large-Scale Nationally Representative Adolescent Sample. PLOS ONE 12(1), e0169839. http://dx.doi.org/10.1371/journal.pone.0169839

8. Boyd DM and Ellison NB (2007) Social Network Sites: definition, history, and scholarship. Journal of Computer-Mediated Communication 13(1), 210–230. http://dx.doi.org/10.1111/j.1083-6101.2007.00393.x

9. Brailovskaia J and Margraf J (2018) What does media use reveal about personality and mental health? An exploratory investigation among German students. PLOS ONE 13(1), e0191810 http://dx.doi.org/10.1371/journal.pone.0191810

10. Brailovskaia J, Teismann T, Margraf J (2018) Physical activity mediates the association between daily stress and Facebook Addiction Disorder (FAD) – A longitudinal approach among German students. Computers in Human Behavior 86, 199–204. http://dx.doi.org/10.1016/j.chb.2018.04.045

11. Brailovskaia J, Velten J, Margaf J (2019) Relationship between daily stress, depression symptoms, and Facebook addiction disorder in Germany and in the United States. Cyberpsychology, Behavior, and Social Networking 22(9), 610–614. http://dx.doi.org/10.1089/cyber.2019.0165

12. Brown TA (2006). Confirmatory factor analysis for applied research. Guilford Press, New York.

13. Chen IH, Strong C, Lin YC, Tsai MC, Leung H et al. (2020) Time invariance of three ultra-brief internet-related instruments: Smartphone Application-Based Addiction Scale (SABAS), Bergen Social Media Addiction Scale (BSMAS), and the nine-item Internet Gaming Disorder Scale- Short Form (IGDS-SF9). Addictive Behaviors 101, 105960. http://dx.doi.org/10.1016/j.addbeh.2019.04.018

14. Elphinston RA and Noller P (2011) Time to face It! Facebook intrusion and the implications for romantic jealousy and relationship satisfaction. Cyberpsychology, Behavior, and Social Networking 14(11), 631–635. http://dx.doi.org/10.1089/cyber.2010.0318

15. Galanou C, Galanakis M, Alexopoulos E, Darviri C (2014) Rosenberg Self-Esteem Scale Greek Validation on Student Sample. Psychology 05, 819–827. http://dx.doi.org/10.4236/psych.2014.58093

16. Global digital population 2020 https://www.statista.com/statistics/617136/digital-population-worldwide/ (accessed 25 May 2020)

17. Griffiths MD (2012) Facebook addiction: concerns, criticism, and recommendations—A response to Andreassen and colleagues. Psychological Reports 110(2), 518–520. http://dx.doi.org/10.2466/01.07.18.PR0.110.2.518-520

18. Hammen C (1991) Generation of stress in the course of unipolar depression. Journal of Abnormal Psychology 100(4), 555–561. http://dx.doi.org/10.1037/0021-843X.100.4.555

19. Hawi N, Rupert M (2018) Identifying commonalities and differences in personality characteristics of Internet and social media addiction profiles: Traits, self-esteem, and self-construal. Behaviour & Information Technology 38, 1–10. http://dx.doi.org/10.1080/0144929X.2018.1515984

20. Hawi NS, Samaha M (2017) The relations among social media addiction, self-esteem, and life satisfaction in university students. Social Science Computer Review 35(5), 576–586. Scopus http://dx.doi.org/10.1177/0894439316660340

21. Koc M, Gulyagci S (2013) Facebook Addiction Among Turkish College Students: The Role of Psychological Health, Demographic, and Usage Characteristics. Cyberpsychology, Behavior, and Social Networking 16(4), 279–284. https://dx.doi.org/10.1089/cyber.2012.0249

22. Kuss DJ, Griffiths MD (2011) Online Social Networking and Addiction—A Review of the Psychological Literature. International Journal of Environmental Research and Public Health 8(9), 3528–3552. http://dx.doi.org/10.3390/ijerph8093528

23. Leong LY, Hew TS, Ooi KB, Lee VH, Hew JJ (2019) A hybrid SEM-neural network analysis of social media addiction. Expert Systems with Applications 133, 296–316. http://dx.doi.org/10.1016/j.eswa.2019.05.024

24. Lin CY, Broström A, Nilsen P, Griffiths MD, Pakpour AH (2017) Psychometric validation of the Persian Bergen Social Media Addiction Scale using classic test theory and Rasch models. Journal of Behavioral Addictions 6(4), 620–629. http://dx.doi.org/10.1556/2006.6.2017.071

25. Liu RT, Alloy LB (2010). Stress generation in depression: A systematic review of the empirical literature and recommendations for future study. Clinical Psychology Review 30(5), 582–593. http://dx.doi.org/10.1016/j.cpr.2010.04.010

26. Monacis L, Palo V, Griffiths MD, Sinatra M (2017) Social networking addiction, attachment style, and validation of the Italian version of the Bergen Social Media Addiction Scale. Journal of Behavioral Addictions 6(2), 178–186. http://dx.doi.org/10.1556/2006.6.2017.023

27. OMICS International. Social networking addiction: Emerging themes and issues | Retrieved June 18, 2020, from https://www.omicsonline.org/social-networking-addiction-emerging-themes-and-issues-2155-6105.1000e118.php?aid=22152

28. Petry NM, O’Brien CP (2013) Internet gaming disorder and the DSM-5. Addiction (Abingdon, England) 108(7), 1186–1187. http://dx.doi.org/10.1111/add.12162

29. Pezirkianidis C, Karakasidou E, Lakioti A, Stalikas A, Galanakis M (2018) Psychometric properties of the Depression, Anxiety, Stress Scales-21 (DASS-21) in a greek sample. Psychology 09, 2933–2950. http://dx.doi.org/10.4236/psych.2018.915170

30. Piedmont RL (2014) Inter-item Correlations. In: MichalosAC (ed) Encyclopedia of Quality of Life and Well-Being Research. Springer Netherlands, Dordrecht, Dordrecht,pp. 3303–3304

31. Pontes HM, Taylor M, Stavropoulos V (2018) Beyond “Facebook Addiction”: The role of cognitive-related factors and psychiatric distress in social networking site addiction. Cyberpsychology, Behavior and Social Networking 21(4), 240–247. http://dx.doi.org/10.1089/cyber.2017.0609

32. Primack B. A, Shensa A, Sidani JE, Whaite EO, Lin LY et al. (2017) Social media use and perceived social isolation among young adults in the U.S. American Journal of Preventive Medicine 53(1), 1–8. http://dx.doi.org/10.1016/j.amepre.2017.01.010

33. Shensa A, Escobar-Viera CG, Sidani JE, Bowman ND, Marshal MP et al. (2017) Problematic social media use and depressive symptoms among U.S. young adults: A nationally-representative study. Social Science & Medicine 182, 150–157. http://dx.doi.org/10.1016/j.socscimed.2017.03.061

34. Starr LR, Hammen C, Brennan PA, Najman J M (2012) Serotonin transporter gene as a predictor of stress generation in depression. Journal of Abnormal Psychology 121(4), 810–818. http://dx.doi.org/10.1037/a0027952

35. Tabachnick BG, Fidell LS, Ullman JB (2019) Using multivariate statistics 7th edition. Pearson, NY.

36. Jafarkarimi H, Sim A, Saadatdoost R, Hee JM (2016) Facebook addiction among Malaysian students. International Journal of Information and Education Technology 6(6), 465–469. http://dx.doi.org/10.7763/IJIET.2016.V6.733

37. Vanman EJ, Baker R, Tobin SJ (2018) The burden of online friends: The effects of giving up Facebook on stress and well-being. The Journal of Social Psychology 158(4), 496–507. http://dx.doi.org/10.1080/00224545.2018.1453467

38. Wuang YP, Su CY, Huang MH (2012) Psychometric comparisons of three measures for assessing motor functions in preschoolers with intellectual disabilities: Motor function measures for intellectual disabilities. Journal of Intellectual Disability Research 56(6), 567–578. http://dx.doi.org/10.1111/j.1365-2788.2011.01491.x

39. Yam CW, Pakpour AH, Griffiths MD, Yau WY, Lo CL et al. (2019) Psychometric Testing of Three Chinese Online-Related Addictive Behavior Instruments among Hong Kong University Students. Psychiatric Quarterly 90(1), 117–128. http://dx.doi.org/10.1007/s11126-018-9610-7

40. Yau YHC, Crowley MJ, Mayes LC, Potenza MN (2012) Are Internet use and video-game-playing addictive behaviors? Biological, clinical and public health implications for youths and adults. Minerva Psichiatrica 53(3), 153–170. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3840433/