PROBLEMATIC INTERNET USE AMONG OLDER ADOLESCENTS
To investigate study quality and reported prevalence among the emergent area of problematic internet use (PIU) research conducted in populations of US adolescents and college students.
We searched PubMed, PsychInfo and Web of Knowledge from inception to July 2010.
Using a keyword search, we evaluated English-language PIU studies with populations of US adolescents and college students.
Main outcome measures
Using a quality review tool based on the STROBE statement, two reviewers independently extracted data items including study setting, subject population, instrument used and reported prevalence.
Search results yielded 658 manuscripts. We identified 18 research studies that met inclusion criteria. Quality assessment of studies ranged between 14 and 29 total points out of a possible 42 points, the average score was 23 (SD 5.1). Among these 18 studies, 8 reported prevalence estimates of US college student PIU, prevalence rates ranged from 0 to 26.3%. An additional 10 studies did not report prevalence.
The evaluation of PIU remains incomplete and is hampered by methodological inconsistencies. The wide range of conceptual approaches may have impacted the reported prevalence rates. Despite the newness of this area of study, most studies in our review were published over 3 years ago. Opportunities exist to pursue future studies adhering to recognized quality guidelines, as well as applying consistency in theoretical approach and validated instruments.
Internet use is nearly ubiquitous among adolescents and young adults; current US data suggests that 93% of adolescents and adults between the ages of 12 and 29 years go online.1 Given these high rates of internet use, internet addiction, often described as “problematic internet use which is uncontrollable and damaging,” is a growing concern.2, 3 Several studies in the US and abroad, and numerous anecdotal media reports, suggest possible links between overuse of the internet by adolescents and young adults and negative health consequences such as depression, ADHD, excessive daytime sleepiness, problematic alcohol use, or injury.4–8 Internet addiction has also been associated with negative academic consequences such as missed classes, lower grades and even academic dismissal.9–11 Currently, internet addiction is proposed as a disorder in need of further study for the appendix of the Diagnostic and Statistical Manual V (DSM-V).12
Efforts towards developing diagnostic criteria for internet addiction or problematic internet use (PIU) began in the 1990s. Two initial approaches to PIU were based upon existing DSM-IV disorders: substance abuse/dependency and pathologic gambling.13, 14 This early work was accompanied by the introduction of three conceptual approaches. First, PIU was more broadly described as a general behavioral addiction. 15,16 Second, a cognitive-behavioral model of PIU drew attention to the impact of an individual’s thoughts on their development of problematic behaviors, and separated PIU into “generalized” PIU, or multidimensional overuse of the internet, and “specific” PIU.17 Specific PIU was defined as dependence on a specific function of the internet. Third, a model proposed that PIU should be more widely classified as a impulse control disorder with criteria defined as: a) maladaptive preoccupation with internet use characterized by either irresistible use, or use that is excessive and longer than planned; b) clinically significant distress or impairment; and, c) an absence of other, explaining, Axis I disorders.18 These differences in the conceptual approach towards PIU have influenced the various instruments that have been developed to evaluate PIU.
At present, there are at least 13 instruments designed to measure PIU. Several were adapted from the DSM-IV substance abuse and dependency criteria, such as the Internet Addiction Disorder Diagnostic Criteria19and the Internet-Related Addictive Behavior Inventory.20 Others are based on the DSM-IV criteria for pathological gambling, including the Young Diagnostic Questionnaire 14 and Young Internet Addiction Test (IAT)21 (the latter being an expansion of the former), the Chen Internet Addiction Scale,22 and the Problematic Internet Usage Questionnaire.23 Other instruments are based on the PIU behavioral addiction model, such as the Compulsive Internet Use Scale24 or the Griffith Addiction Components Criteria.25Additional instruments are based on the Davis cognitive-behavioral model of PIU, including the Online Cognition Scale (OCS)26 and the Generalized Problematic Internet Use Scale (GPIUS).27
Given the high rates of internet use among adolescents and young adults globally, it may not be surprising that research on PIU in this population has received intense international attention. Prevalence estimates of PIU vary widely. In studies focused on adolescents, European prevalence estimates are reported as between 1–9%,28–32 Middle Eastern prevalence estimates are between 1–12% 33–35 and Asian prevalence estimates are reported between 2–18%. 36–43 Similarly, the prevalence for international college students has been reported as between 6–35%.44–47 It is unclear whether the wide range of prevalence estimates reported is related to cultural differences between regions or countries, or due to different approaches in the definition and assessment of PIU.
Despite the timeliness and importance of this topic, to our knowledge, a systematic review of the existing literature on PIU among US adolescent and college students examining both study quality and reported prevalence is lacking. As research findings often lead to diagnostic criteria and clinical practice, the quality of such studies is of the utmost importance. Our goals are to examine: 1) the quality of studies in this area, and 2) the prevalence rate for problematic internet use among US adolescents and college students. By conducting this systematic review we provide an understanding of the current approaches to PIU and a framework upon which future research endeavors can be built.
In consultation with a health sciences librarian, a systematic review was performed of three major databases incorporating medical and social science literature. PubMed, PsychInfo and Web of Knowledge were searched from inception to July 2010. As no Medical Subject Headings (MeSH) terms were found to fit our topic of interest, we identified key word search terms starting with the terms “internet addiction” and “problematic internet use” and building additional terms by identifying keywords associated with those searches or within articles found in those searches. A final list of search terms included the following keywords or keyword combinations: internet addiction, compulsive internet use, problematic internet use, pathological internet use, internet dependence, and excessive internet use. To identify additional articles that addressed problematic internet use we searched the bibliographies of included studies.
Given the current consideration of internet addiction for inclusion in the DSM V, we chose to focus our review on studies which investigated internet use as a source of addiction or dependency. We did not investigate related concerns, such as inappropriate use of the internet for sharing sexual explicit material or cyberbullying. Thus, we included English-language studies that (1) involved a US population, (2) focused on adolescents or college student participants, and (3) assessed internet addiction symptoms empirically through the use of a scale or set criteria. We excluded non-US papers, studies that focused on adults, studies that did not assess PIU specifically, non-empirical work such as case studies or commentaries and unpublished literature. Searches were initially screened for inclusion using titles of articles and abstracts when available, when inclusion criteria were not clear from the title and abstract the full text was evaluated. Full text of articles that met inclusion criteria were retrieved and systematically assessed by two investigators.
|Items for Review||Scoring Categories|
|0 points||1 point||2 points|
|1. Recruitment timeframe||Timeframe not reported||Year or month/season reported||Year and month/season reported|
|2. Study setting||Setting not described||Described university setting or classroom vs. online setting||Described setting of two or more: university, classroom vs. online, or specific course(s)|
|3. PIU assessment validity||No prior work||Assessment alpha reported, or assessment previously piloted||Validation study on assessment published previously|
|4. Criteria for classifying PIU||Criteria not defined||Some discussion of criteria||Score cutoffs clearly defined|
|5. Assessment response scale||Scale not defined||Scale type defined (e.g. likert, binary)||Type of scale defined including exact values (e.g. 6-point likert scale,”never”to “always”)|
|6. Variables||Variables not defined||--||Variables clearly defined|
|7. Participants inclusion/exclusion criteria defined||Criteria not defined||Criteria defined without rationale||Criteria defined with rationale|
|8. Recruitment strategy||Strategy not reported||--||Strategy reported|
|9. Response rate||Response rate not reported||--||Response rate reported|
|10. Representative sampling strategy||Sampling strategy was not representative||Strategy approximated an established representative method||Strategy included an established representative method|
|11. Study size||Explanation not reported||--||Explanation clearly reported|
|12. Statistical methods||Methods not described||--||Methods described including specific tests|
|13. Participant numbers, including: potentially eligible, eligible, examined for eligibility, and confirmed eligible||Numbers not reported||Eligibility numbers partially reported||Reported all eligibility numbers|
|14. Age of Participants||Age not reported||Mean or limits reported||Mean and range reported|
|15. Participants by gender||Gender not reported||--||Participants gender reported|
|16. Participants by ethnicity||Ethnicity not reported||--||Participants ethnicity reported|
|17. Number of participants with missing data||Data not reported||Reported reference to missing data||Total number of participants removed due to missing data reported|
|18. Number of participants meeting PIU criteria||Number not reported||--||Number of participants meeting PIU criteria reported|
|19. Average PIU score overall and by item||Scores not reported||Scores partially reported||Reported all overall and item scores|
|PIU Specific items|
|20. Definition of PIU||PIU not defined||PIU defined||PIU defined with supporting background or citations|
|21. Participants’ Internet Use Habits||Data not measured||--||Internet use measured and reported (e.g. average hours per day or week)|
|DSM Substance Use Criteria||Scherer, 1997||29(69)||531||M = 24.46||University of Texas at Austin||Mailing to random sample||Scherer Internet Dependence Scale||Responding positively to 3+ items out of 10||No||Paper/pencil||13%|
|Fortson, Scotti, Chen, Malone & Del Ben, 2007||26 (69)||411||R = 18–56|
M = 20.4
SD = 3.2
|Large, southeastern, regional university||Classroom- undergraduate introduction psychology||Fortson et al. Internet Dependence Scale||Reporting 3+ symptoms as measured by a set of liberal and conservative criteria||No||Paper/pencil & online||1.2–26.3%|
|Anderson, 2001||25 (60)||1078||--||7 diverse US colleges and 1 Irish college||Classroom- excluded freshman||Anderson Internet Dependence Scale||Responding positively to 3+ items out of 7||No||Paper/pencil||9.8%|
|Lavin & Yuen, 2004||22 (52)||283||≥ 18||Small, private, western New York university||Campus-wide e-mail||Lavin and Yuen Internet Dependence Scale||Scoring 4+ out of 5 points on 3+ items out of 7||No||Online||15.20%|
|DSM Pathological Gambling Criteria||Iacovelli & Valenti, 2009||22 (52)||--||--||Hofstra University||Designated, campus-wide recruiting session||IAT||Scoring 40+ out of 100||Yes||Paper/pencil||25%|
|Kim & Haridakis, 2009||--||203||M = 21.5|
SD = 5.3
R = --
|Large, Midwestern university||Classroom- undergraduate liberal education course||Kim and Haridakis Internet Addiction Scale||--||Partially||Paper/pencil||--|
|Kim & Davis, 2009*||--||S1: 315|
|M = 22.3, 21.4|
SD = 5.8, 3.2
R= --, 16–42
|Large, southeastern, state university||Classroom- undergraduates from a variety of majors||Kim and Davis Problematic Internet Use Scales||--||Yes||Paper/pencil||--|
|Davis Cognitive- Behavioral Model||Caplan, 2002||--||386||M = 20|
SD = 2.2
R = 18–57
|Caplan, 2003||--||386||M = 20|
SD = 2.2
R = 18–57
|--||Classroom- communications course, and word of mouth||Adapted GPIUS||--||Partially||--||--|
|Caplan, 2005||--||251||M = 19.8|
SD = 1.4
R = 18–32
|--||Classroom- undergraduates, various majors in an intro communication course||Adapted GPIUS||--||Partially||--||--|
|Caplan, 2007||--||343||M = 19.4|
SD = 1.4
R = 18–28
|Kim, LaRose & Peng, 2009||--||635||--||2 large, Midwestern universities||Invitation to an online survey||Adapted GPIUS||--||Partially||Online||--|
|Jia & Jia,||--||267||--||Public||Classroom-||Abbreviated||--||Yes||--||--|
A total of 8 studies that provided descriptive data and reported prevalence were assessed using the QRT. Quality assessment of studies ranged between 14 and 29 total points out of a possible 42 points, the average score was 23 (SD 5.1). The majority of these studies received less than two-thirds of the available 42 total quality points (Table 3). Individual QRT categories that occurred least frequently across all studies included: explanations for the selected sample size (0 out of possible 16 total points), response rate reporting (2 out of 16 total points); study timing reporting (3 out of 16 total points), rates of missing data (3 out of 16 total points). The item that measured use of a piloted or validated instrument scored only 5 out of a possible 16 total points. Only 3 studies reported ethnicity (5 points out of possible 16 total points). Only 1 study documented rates of missing data (2 out of possible 16 total points).
|Item||Studies by Quality Review Scoring (reference)|
|1. Recruitment timeframe reported||26, 55, 56, 57, 59, 66||54||58|
|2. Study setting described||--||26, 57||54, 55, 56, 58, 59, 66|
|3. Use of a piloted or validated assessment||54, 57, 58, 59, 66||26, 55||56|
|4. Problematic criteria clearly defined||59||66||26, 54, 55, 56, 57, 58|
|5. Response scale clearly defined for PIU items||59||58||26, 54, 55, 56, 57, 66|
|6. Variables defined||--||59||26, 54, 55, 56, 57, 58, 66|
|7. Participants inclusion/exclusion criteria defined||--||55, 56, 57, 58, 59, 66||26, 54|
|8. Recruitment strategy reported||--||--||26, 54, 55, 56, 57, 58, 59, 66|
|9. Response rate reported [(No. participating/No. invited) × 100]||26, 54, 55, 56, 57, 59, 66||--||58|
|10. Representative sampling strategy used||26?, 55, 56, 59, 66||54||57, 58|
|11. Explanation for study size||26, 54, 55, 56, 57, 58, 59, 66||--||--|
|12. Statistical methods described||--||--||26, 54, 55, 56, 57, 58, 59, 66|
|13. Participant numbers reported||26, 55, 56, 57, 58, 59, 66||54||--|
|14. Age reported||54, 59, 66||26, 57, 56*, 58||55|
|15. Gender reported||--||56*, 57?||26, 54, 55, 58, 59, 66|
|16. Ethnicity reported||26, 54, 57, 59, 66||56*||55, 58|
|17. Number of participants with missing data reported||26, 56, 57, 58, 59, 66||54||55|
|18. Number of participants meeting criteria reported||--||59||26, 54, 55, 56, 57, 58, 66|
|19. Average score overall and by item reported||26, 54, 57, 59, 66||58||55, 56|
|PIU specific items||--||--||--|
|20. Clear definition of PIU reported||56||54, 55, 57, 58, 59, 66||26|
|21. Internet use habits of participants reported||56||--||26, 54, 55, 57, 58, 59, 66|
Overall Mean Score: 22.85
Individual QRT categories that occurred most frequently across all studies included describing the recruitment strategy (16 out of 16 points) and describing statistical methods used (16 out of 16 points).
Prevalence of PIU
Overall, the range of prevalence of PIU in examined studies was between 0% and 26.3%. The reported prevalence of PIU must be considered in the context of the conceptual approach identified in that study (i.e. substance use, pathological gambling).
Four studies evaluated PIU based on DSM IV criteria for substance use. Three of these studies defined “internet dependency” as a participant answering affirmatively to between 3 and 4 items out of 7 to 10 total items; these studies found that prevalence ranged from 9.8 to 15.2%.49, 52, 54 The fourth study used both a “liberal” and “conservative” set of criteria to determine criteria for both internet abuse and dependency. This study found a range of 1.2 to 26.3% prevalence for dependency within a single sample.50 A single study used the IAT, based on DSM IV criteria for pathologic gambling.23 This study defined internet addiction as scoring over 40 total points and found a prevalence of 25%.51
Three studies used independently generated instruments without a specifically described conceptual model and found prevalence to between no participants meeting criteria and 12.6%.53, 55, 62 Among these, one study conducted assessments in two populations. No estimate was given for overall prevalence for the first sample, although reference was made to participants meeting criteria, while no participants met the criteria for PIU in the second sample.55
Studies that did not report PIU prevalence rates
Among the 10 studies that did not report prevalence estimates, the majority were focused on developing a conceptual model of PIU or validating an instrument scale. These studies used a range of instruments, some of which were independently developed, as well as the IAT, the OCS, and the GPIUS. Of these ten studies, three introduced and validated new instruments,27, 63, 64 two adapted previously validated instruments,23, 60 and five modified previously validated instruments, which included the use of additional items.56–58, 60, 61
Overall our findings suggest a paucity of empirical studies addressing PIU among populations of US adolescent and college student populations. Despite initially finding over 600 search hits on the topic of PIU, only 18 articles were identified that met inclusion criteria, less than half of these reported a prevalence estimate. We found no studies specifically targeting adolescent populations.
Among these studies, the overall quality scores were very low. Many of the QRT items that received particularly low scores, such as using a validated instrument and reporting missing data, have significant impact on the internal validity of the findings. Further, other areas that received low scores, such as reporting response rates and participant characteristics, critically impact the external validity of these studies. Future studies of PIU could consider using the STROBE criteria or our QRT to enhance the quality of the study and thus the validity of the findings.
The studies examined in this review reported prevalence rates ranging from no participants meeting criteria to up to a quarter of participants meeting criteria for PIU. There are several possible reasons that this range of reported prevalence rates is so wide. First, many of these instruments applied vastly different conceptual approaches based on addictions such as substance use or gambling, or other cognitive, behavioral or impulse-control models. The lack of consensus in conceptual approach to PIU may be a key reason for the variability amongst these studies approach and findings. Second, perhaps related to the lack of consensus on the appropriate conceptual approach to PIU, the majority of studies in this review used independently created instruments whose conceptual framework is incompletely evaluated. This then leads to additional challenges because the psychometric properties of these new instruments are often incompletely evaluated. Third, instruments used to evaluate PIU applied varying response mechanisms: some used Likert scales which allow for reporting the degree and severity of symptoms or consequences, and others used binary “yes/no” responses which may not fully capture the frequency or severity of a problematic behavior. Fourth, the cut-offs for criteria defining the when a participant met criteria for PIU varied among the instruments used to assess PIU. As studies did not correlate their cut-points to actual negative consequences such as behavioral or achievement problems, it is difficult to know whether participants who were labeled as having PIU were actually experiencing any offline consequences.
Last, over half of the studies reporting prevalence estimates were conducted over five years ago during a time where wide-scale internet use was still varied and growing. Immense changes in both internet access and use have occurred over the last decade.1 Thus, it is reasonable to assume that not only the extent of, but also the populations most at risk for, internet addiction may have changed from what was evident in the past. More recent work is required to determine not only a current estimate of prevalence based on a standardized approach, but also what characteristics may put an individual at increased risk in our current technology-saturated culture. Findings which are informed by current internet use standards and trends may also help to shape the development and definition of a diagnosis for a clinical disorder.
The findings in this review may be limited as we did not search the gray literature (evaluation of theses, dissertations or unpublished work). However, many of the studies examined in our review had methodological flaws limiting external validity, such as failure to report response rates, thus the gap between unpublished and published literature may be small. Further, given the newness of this field and the wide range of prevalence rates reported in studies, including studies that reported a prevalence rate of 0%, it is likely that publication bias may also be small. Our goal in this study was to evaluate US studies, thus, generalization beyond the US is not warranted.
Despite these limitations, our study findings illustrate the critical need for additional rigorous study of PIU. However, in order to fully understand and estimate the impact of this new disorder, we must first have consistency and consensus in the approach to its assessment. Among the instruments identified in this study, the IAT was the only validated instrument used in a study that reported prevalence rates. Another validated and frequently used instrument was the OCS, although this scale was not used in studies reporting prevalence data. Thus, these instruments may be a useful starting point for future study. As both of these measures were initially developed over 8 years ago, re-evaluating their construct structure and establishing face validity in the context of today’s internet-rich environment and within this target population will be an important initial step. Administering multiple instruments in the context of a single study to determine overlap and concurrent validity may be useful in the pursuit of developing a comprehensive instrument to assess PIU. Following this, further rigorous studies using a validated instrument and incorporating recognized quality criteria may be conducted to confirm prevalence data. Finally, among studies that reported time spent on the internet, all relied upon participant self-report for cumulative internet use. Future studies that provide more accurate means of measuring internet use are needed.
Further, of note, no US studies identified in this review included samples focused on the adolescent population, and studies of college students were generally limited to a single university and modest sample sizes. Future large-scale studies within these at-risk populations are urgently needed to confirm and enhance generalizability. Several European and Asian countries have included assessments of internet addiction within national assessments of adolescent and college student health.10, 28, 65, 66 Adopting similar methods within the US may allow for accurate identification and estimated scope of this problem on a national level.
If internet use has potential to lead to addiction, this means that up to 93% of US adolescents and young adults are exposed to this risk, dwarfing exposure rates for any other behavioral or substance-based addiction.1 Before we can fully understand this important phenomenon, we must first have consistency and consensus in the approach to its assessment. Only after these studies have firmly established current prevalence and considered risk factors, can we make informed considerations on what diagnostic criteria should be recommended for inclusion within the DSM or how to evaluate the successes of any proposed treatment programs.
Support for this project was provided by award K12HD055894 from NICHD. The authors would like to thank Heidi Marleau for her assistance with this project.