Volume 5, Issue 2
  • ISSN 2452-0063
  • E-ISSN: 2452-0071
Buy:$35.00 + Taxes



This article studies the efficiency of different samples for content analysis of news in media effects studies by comparing the agenda-setting effect of a classic sample with the effect of a sample drawn based on audiences’ self-reported media habits. Contrary to the belief that exposure to sampled media content is necessary for observation of media effects, samples drawn based on overall readership/viewership of the media are more efficient than samples based on audiences’ actual consumption habits. A traditional media sample yields a stronger agenda-setting effect compared to a sample drawn based on self-reported media habits. But correlations between the two media samples are also strong. The findings suggest that a broad intermedia agenda-setting process makes it possible for researchers to draw a traditional sample that is representative of the issues salient to audiences regardless of their level of exposure to the sampled media. In other words, even in a demassified media environment, traditional samples are still the best option for media effects researchers.


Article metrics loading...

Loading full text...

Full text loading...


  1. Althaus, S. L., & Tewksbury, D.
    (2002) Agenda setting and the “new” news: Patterns of issue importance among readers of the paper and online versions of the New York Times. Communication Research, 29(2), 180–207.   10.1177/0093650202029002004
    https://doi.org/10.1177/0093650202029002004 [Google Scholar]
  2. Ariens, C.
    (2018, March27). Evening News Ratings: Week of March 19 [News]. Ad Week. www.adweek.com/tvnewser/evening-news-ratings-week-of-march-19-2/360317
    [Google Scholar]
  3. Atwater, T., Salwen, M. B., & Anderson, R. B.
    (1985) Media Agenda-Setting with Environmental Issues. Journalism Quarterly, 62(2), 393–397.   10.1177/107769908506200227
    https://doi.org/10.1177/107769908506200227 [Google Scholar]
  4. Bennett, W. L., & Iyengar, S.
    (2008) A new era of minimal effects? The changing foundations of political communication. Journal of Communication, 58(4), 707–731.   10.1111/j.1460‑2466.2008.00410.x
    https://doi.org/10.1111/j.1460-2466.2008.00410.x [Google Scholar]
  5. Berinsky, A. J., Huber, G. A., & Lenz, G. S.
    (2012) Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk. Political Analysis, 20(03), 351–368.   10.1093/pan/mpr057
    https://doi.org/10.1093/pan/mpr057 [Google Scholar]
  6. Blondheim, M.
    (1994) News over the wires: The telegraph and the flow of public information in America, 1844–1897. Harvard University Press.
    [Google Scholar]
  7. Breed, W.
    (1955) Newspaper ‘opinion leaders’ and processes of standardization. Journalism Quarterly, 32(3), 277–328.   10.1177/107769905503200302
    https://doi.org/10.1177/107769905503200302 [Google Scholar]
  8. Brosius, H.-B., & Kepplinger, H. M.
    (1992) Linear and nonlinear models of agenda-setting in television. Journal of Broadcasting & Electronic Media, 36(1), 5–23.   10.1080/08838159209364151
    https://doi.org/10.1080/08838159209364151 [Google Scholar]
  9. Brosius, H.-B., & Weimann, G.
    (1996) Who sets the agenda: Agenda-setting as a two-step flow. Communication Research, 23(5), 561–580.   10.1177/009365096023005002
    https://doi.org/10.1177/009365096023005002 [Google Scholar]
  10. Camaj, L., & Weaver, D. H.
    (2013) Need for orientation and attribute agenda-setting during a U.S. election campaign. International Journal of Communication, 7, 1442–1463.
    [Google Scholar]
  11. Chaffee, S. H., & Metzger, M. J.
    (2001) The end of mass communication?Mass Communication and Society, 4(4), 365–379.   10.1207/S15327825MCS0404_3
    https://doi.org/10.1207/S15327825MCS0404_3 [Google Scholar]
  12. Chmielewski, M., & Kucker, S. C.
    (2020) An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results. Social Psychological and Personality Science, 11(4), 464–473.   10.1177/1948550619875149
    https://doi.org/10.1177/1948550619875149 [Google Scholar]
  13. Coleman, R., & Banning, S.
    (2006) Network TV news’ affective framing of the presidential candidates: Evidence for a second-level agenda-setting effect through visual framing. Journalism & Mass Communication Quarterly, 83(2), 313–328.   10.1177/107769900608300206
    https://doi.org/10.1177/107769900608300206 [Google Scholar]
  14. Coleman, R., & McCombs, M. E.
    (2007) The young and agenda-less? Exploring age-related differences in agenda setting on the youngest generation, baby boomers, and the civic generation. Journalism & Mass Communication Quarterly, 84(3), 495–508.   10.1177/107769900708400306
    https://doi.org/10.1177/107769900708400306 [Google Scholar]
  15. Conway, B. A.
    (2013) Addressing the “medical malady”: Second-level agenda setting and public approval of “Obamacare.” International Journal of Public Opinion Research, 25(4), 535–546.   10.1093/ijpor/eds041
    https://doi.org/10.1093/ijpor/eds041 [Google Scholar]
  16. Conway, B. A., Kenski, K., & Wang, D.
    (2015) The rise of Twitter in the political campaign: Searching for intermedia agenda-setting effects in the presidential primary. Journal of Computer-Mediated Communication, 20(4), 363–380.   10.1111/jcc4.12124
    https://doi.org/10.1111/jcc4.12124 [Google Scholar]
  17. Cowart, H.
    (2020) What to think about: The applicability of agenda-settings in a social media context. The Agenda Setting Journal, 4(2), 195–218.   10.1075/asj.19001.cow
    https://doi.org/10.1075/asj.19001.cow [Google Scholar]
  18. Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M.
    (2013) Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research. PLoS ONE, 8(3), e57410.   10.1371/journal.pone.0057410
    https://doi.org/10.1371/journal.pone.0057410 [Google Scholar]
  19. Dearing, J. W., & Rogers, E. M.
    (1996) Agenda-setting. Sage.
    [Google Scholar]
  20. Dillman Carpentier, F. R.
    (2014) Agenda setting and priming effects based on information presentation: Revisiting accessibility as a mechanism explaining agenda setting and priming. Mass Communication and Society, 17(4), 531–552.   10.1080/15205436.2013.816744
    https://doi.org/10.1080/15205436.2013.816744 [Google Scholar]
  21. Djerf-Pierre, M., & Shehata, A.
    (2017) Still an agenda setter: Traditional news media and public opinion during the transition from low to high choice media environments. Journal of Communication, 67(5), 733–757.   10.1111/jcom.12327
    https://doi.org/10.1111/jcom.12327 [Google Scholar]
  22. Funkhouser, G. R.
    (1973) The issues of the sixties: An exploratory study in the dynamics of public opinion. Public Opinion Quarterly, 37(1), 62.   10.1086/268060
    https://doi.org/10.1086/268060 [Google Scholar]
  23. Golan, G., & Wanta, W.
    (2001) Second-level agenda setting in the New Hampshire primary: A comparison of coverage in three newspapers and public perceptions of candidates. Journalism & Mass Communication Quarterly, 78(2), 247–259.   10.1177/107769900107800203
    https://doi.org/10.1177/107769900107800203 [Google Scholar]
  24. Gold, D., & Simmons, J. L.
    (1965) News selection patterns among Iowa dailies. Public Opinion Quarterly, 29(3), 425–430. 10.1086/267342
    https://doi.org/10.1086/267342 [Google Scholar]
  25. Hester, J. B., & Dougall, E.
    (2007) The efficiency of constructed week sampling for content analysis of online news. Journalism & Mass Communication Quarterly, 84(4), 811–824.   10.1177/107769900708400410
    https://doi.org/10.1177/107769900708400410 [Google Scholar]
  26. Hester, J. B., & Gibson, R.
    (2003) The economy and second-level agenda setting: A time-series analysis of economic news and public opinion about the economy. Journalism & Mass Communication Quarterly, 80(1), 73–90.   10.1177/107769900308000106
    https://doi.org/10.1177/107769900308000106 [Google Scholar]
  27. (2007) The agenda-setting function of national versus local media: A time-series analysis for the issue of same-sex marriage. Mass Communication and Society, 10(3), 299–317.   10.1080/15205430701407272
    https://doi.org/10.1080/15205430701407272 [Google Scholar]
  28. Horton, J. J., Rand, D. G., & Zeckhauser, R. J.
    (2011) The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14(3), 399–425.   10.1007/s10683‑011‑9273‑9
    https://doi.org/10.1007/s10683-011-9273-9 [Google Scholar]
  29. Horvit, B., Gade, P., & Lance, E. A.
    (2013) News wire greatest predictor of papers’ international news. Newspaper Research Journal, 34(1), 89–103.   10.1177/073953291303400108
    https://doi.org/10.1177/073953291303400108 [Google Scholar]
  30. Iyengar, S., & Kinder, D. R.
    (1987) News that matters: Television and American opinion. University of Chicago Press.
    [Google Scholar]
  31. Johnson, T. J.
    (Ed.) (2014) Agenda setting in a 2.0 world. Routledge.
    [Google Scholar]
  32. Kim, S. H., Scheufele, D. A., & Shanahan, J.
    (2002) Think about it this way: Attribute agenda-setting function of the press and the public’s evaluation of a local issue. Journalism & mass communication quarterly 79(1), 7–25. 10.1177/107769900207900102
    https://doi.org/10.1177/107769900207900102 [Google Scholar]
  33. Kim, Y., Kim, Y., & Zhou, S.
    (2017) Theoretical and methodological trends of agenda-setting theory: A thematic analysis of the last four decades of research. The Agenda Setting Journal, 1(1), 5–22.   10.1075/asj.1.1.03kim
    https://doi.org/10.1075/asj.1.1.03kim [Google Scholar]
  34. Kosicki, G. M.
    (1993) Problems and opportunities in agenda-setting research. Journal of Communication, 43(2), 100–127. 10.1111/j.1460‑2466.1993.tb01265.x
    https://doi.org/10.1111/j.1460-2466.1993.tb01265.x [Google Scholar]
  35. Ku, G., Kaid, L. L., & Pfau, M.
    (2003) The impact of web site campaigning on traditional news media and public information processing. Journalism & Mass Communication Quarterly, 80(3), 528–547.   10.1177/107769900308000304
    https://doi.org/10.1177/107769900308000304 [Google Scholar]
  36. Lacy, S., Riffe, D., Stoddard, S., Martin, H., & Chang, K.-K.
    (2001) Sample size for newspaper content analysis in multi-year studies. Journalism & Mass Communication Quarterly, 78(4), 836–845.   10.1177/107769900107800414
    https://doi.org/10.1177/107769900107800414 [Google Scholar]
  37. Lee, B., Lancendorfer, K. M., & Lee, K. J.
    (2005) Agenda-setting and the internet: The intermedia influence of internet bulletin boards on newspaper coverage of the 2000 general election in South Korea. Asian Journal of Communication, 15(1), 57–71.   10.1080/0129298042000329793
    https://doi.org/10.1080/0129298042000329793 [Google Scholar]
  38. Leetaru, K. H.
    (2015) Mining libraries: Lessons learned from 20 years of massive computing on the world’s information. Information Services & Use, 35(1–2), 31–50.   10.3233/ISU‑150767
    https://doi.org/10.3233/ISU-150767 [Google Scholar]
  39. Legendre, P.
    (2005) Species associations: The Kendall coefficient of concordance revisited. Journal of Agricultural, Biological, and Environmental Statistics, 10(2), 226–245.   10.1198/108571105X46642
    https://doi.org/10.1198/108571105X46642 [Google Scholar]
  40. Lim, J.
    (2006) A cross-lagged analysis of agenda setting among online news media. Journalism & Mass Communication Quarterly, 83(2), 298–312.   10.1177/107769900608300205
    https://doi.org/10.1177/107769900608300205 [Google Scholar]
  41. (2011) First-level and second-level intermedia agenda-setting among major news websites. Asian Journal of Communication, 21(2), 167–185.   10.1080/01292986.2010.539300
    https://doi.org/10.1080/01292986.2010.539300 [Google Scholar]
  42. Lopez-Escobar, E., Llamas, J. P., & McCombs, M. E.
    (1998) Agenda setting and community consensus: First and second level effects. International Journal of Public Opinion Research, 10(4), 335–348.   10.1093/ijpor/10.4.335
    https://doi.org/10.1093/ijpor/10.4.335 [Google Scholar]
  43. McCombs, M. E.
    (2005) A Look at Agenda-setting: Past, present and future. Journalism Studies, 6(4), 543–557.   10.1080/14616700500250438
    https://doi.org/10.1080/14616700500250438 [Google Scholar]
  44. (2014) Setting the agenda: The mass media and public opinion (2nd ed.). Polity.
    [Google Scholar]
  45. McCombs, M. E., Llamas, J. P., Lopez-Escobar, E., & Rey, F.
    (1997) Candidate images in Spanish elections: Second-level agenda-setting effects. Journalism & Mass Communication Quarterly, 74(4), 703–717.   10.1177/107769909707400404
    https://doi.org/10.1177/107769909707400404 [Google Scholar]
  46. McCombs, M. E., & Shaw, D. L.
    (1972) The agenda-setting function of mass media. Public Opinion Quarterly, 36(2), 176.   10.1086/267990
    https://doi.org/10.1086/267990 [Google Scholar]
  47. (1993) The evolution of agenda-setting research: Twenty-five years in the marketplace of ideas. Journal of Communication, 43(2), 58–67.   10.1111/j.1460‑2466.1993.tb01262.x
    https://doi.org/10.1111/j.1460-2466.1993.tb01262.x [Google Scholar]
  48. Megwa, E. R., & Brenner, D. J.
    (1988) Toward a paradigm of media agenda-setting effect: Agenda-setting as a process. Howard Journal of Communications, 1(1), 39–56.   10.1080/10646178809359668
    https://doi.org/10.1080/10646178809359668 [Google Scholar]
  49. Minooie, M.
    (2019) Agendamelding: How audiences meld agendas in Iran. The Agenda Setting Journal, 3(2), 139–164.   10.1075/asj.18010.min
    https://doi.org/10.1075/asj.18010.min [Google Scholar]
  50. Misachi, J.
    (2017, August1). The 10 Most Popular Daily Newspapers in The United States[News]. World Atlas. https://www.worldatlas.com/articles/the-10-most-popular-daily-newspapers-in-the-united-states.html
    [Google Scholar]
  51. Moeller, J., Trilling, D., Helberger, N., Irion, K., & De Vreese, C.
    (2016) Shrinking core? Exploring the differential agenda setting power of traditional and personalized news media. Info, 18(6), 26–41.   10.1108/info‑05‑2016‑0020
    https://doi.org/10.1108/info-05-2016-0020 [Google Scholar]
  52. Paolacci, G., & Chandler, J.
    (2014) Inside the Turk: Understanding Mechanical Turk as a Participant Pool. Current Directions in Psychological Science, 23(3), 184–188.   10.1177/0963721414531598
    https://doi.org/10.1177/0963721414531598 [Google Scholar]
  53. Ragas, M. W., & Roberts, M. S.
    (2009) Agenda setting and agenda melding in an age of horizontal and vertical media: A new theoretical lens for virtual brand communities. Journalism & Mass Communication Quarterly, 86(1), 45–64.   10.1177/107769900908600104
    https://doi.org/10.1177/107769900908600104 [Google Scholar]
  54. Reese, S. D., & Danielian, L.
    (1989) Intermedia influence and the drug issue: Converging on cocaine. InP. J. Shoemaker (Ed.), Communication campaigns about drugs: Government, media, and the public (pp.29–46). L. Erlbaum Associates.
    [Google Scholar]
  55. Riffe, D., Lacy, S., & Drager, M. W.
    (1996) Sample Size in Content Analysis of Weekly News Magazines. Journalism & Mass Communication Quarterly, 73(3), 635–644.   10.1177/107769909607300310
    https://doi.org/10.1177/107769909607300310 [Google Scholar]
  56. Riffe, D., Lacy, S., & Fico, F.
    (2014) Analyzing media messages: Using quantitative content analysis in research (Third edition). Routledge/Taylor & Francis Group. 10.4324/9780203551691
    https://doi.org/10.4324/9780203551691 [Google Scholar]
  57. Riffe, D., Lacy, S., Nagovan, J., & Burkum, L.
    (1996) The Effectiveness of Simple and Stratified Random Sampling in Broadcast News Content Analysis. Journalism & Mass Communication Quarterly, 73(1), 159–168.   10.1177/107769909607300114
    https://doi.org/10.1177/107769909607300114 [Google Scholar]
  58. Rogers, E. M., & Chang, S. B.
    (1991) Media coverage of technology issues: Ethiopian drought of 1984, AIDS, challenger, and Chernobyl. InL. Wilkins & P. Patterson (Eds.), Risky business: Communicating issues of science, risk, and public policy (pp.75–96). Greenwood Press.
    [Google Scholar]
  59. Rogers, E. M., Dearing, J. W., & Bregman, D.
    (1993) The anatomy of agenda-setting research. Journal of Communication, 43(2), 68–84.   10.1111/j.1460‑2466.1993.tb01263.x
    https://doi.org/10.1111/j.1460-2466.1993.tb01263.x [Google Scholar]
  60. Sayre, B., Bode, L., Shah, D., Wilcox, D., & Shah, C.
    (2010) Agenda setting in a digital age: Tracking attention to California Proposition 8 in social media, online news and conventional news. Policy & Internet, 2(2), 7–32.   10.2202/1944‑2866.1040
    https://doi.org/10.2202/1944-2866.1040 [Google Scholar]
  61. Schoenbach, K., & Semetko, H. A.
    (1992) Agenda-Setting, Agenda-Reinforcing or Agenda-Deflating? A Study of the 1990 German National Election. Journalism Quarterly, 69(4), 837–846.   10.1177/107769909206900404
    https://doi.org/10.1177/107769909206900404 [Google Scholar]
  62. Semetko, H. A., Blumler, J. G., Gurevitch, M., & Weaver, D. H.
    (1991) The formation of campaign agendas: A comparative analysis of party and media roles in recent American and British elections. L. Erlbaum Associates.
    [Google Scholar]
  63. Shank, D. B.
    (2016) Using Crowdsourcing Websites for Sociological Research: The Case of Amazon Mechanical Turk. The American Sociologist, 47(1), 47–55.   10.1007/s12108‑015‑9266‑9
    https://doi.org/10.1007/s12108-015-9266-9 [Google Scholar]
  64. Shaw, D. L., & McCombs, M. E.
    (1977) The emergence of American political issues: The agenda-setting function of the press. West Pub. Co.
    [Google Scholar]
  65. Shaw, D. L., Minooie, M., Aikat, D., & Vargo, C. J.
    (2019) Agendamelding: News, social media, audiences, and civic community. Peter Lang. 10.3726/b15023
    https://doi.org/10.3726/b15023 [Google Scholar]
  66. Shaw, D. L., & Weaver, D. H.
    (2014) Media agenda-setting and audience agendamelding. InM. E. McCombs (Ed.), Setting the agenda: The mass media and public opinion (2nd ed., pp.145–150). Polity.
    [Google Scholar]
  67. Shehata, A., & Strömbäck, J.
    (2013) Not (yet) a new era of minimal effects: A study of agenda setting at the aggregate and individual levels. The International Journal of Press/Politics, 18(2), 234–255.   10.1177/1940161212473831
    https://doi.org/10.1177/1940161212473831 [Google Scholar]
  68. Tan, Y., & Weaver, D. H.
    (2013) Agenda diversity and agenda setting from 1956 to 2004: What are the trends over time?Journalism Studies, 14(6), 773–789.   10.1080/1461670X.2012.748516
    https://doi.org/10.1080/1461670X.2012.748516 [Google Scholar]
  69. Towner, T. L., & Muñoz, C. L.
    (2018) Picture perfect? The role of Instagram in issue agenda setting during the 2016 presidential primary campaign. Social Science Computer Review, 36(4), 484–499.   10.1177/0894439317728222
    https://doi.org/10.1177/0894439317728222 [Google Scholar]
  70. Vargo, C. J., Guo, L., & Amazeen, M. A.
    (2017) The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016. New Media & Society, 146144481771208.   10.1177/1461444817712086
    https://doi.org/10.1177/1461444817712086 [Google Scholar]
  71. Vargo, C. J., Guo, L., McCombs, M. E., & Shaw, D. L.
    (2014) Network issue agendas on Twitter during the 2012 U.S. Presidential Election: Network issue agendas on Twitter. Journal of Communication, 64(2), 296–316.   10.1111/jcom.12089
    https://doi.org/10.1111/jcom.12089 [Google Scholar]
  72. Weaver, D. H., Graber, D. A., McCombs, M. E., & Eyal, C. H.
    (1981) Media agenda-setting in a presidential election: Issues, images, and interest. Praeger.
    [Google Scholar]
  73. Weaver, D. H., Wojdynski, B., McKeever, R., & Shaw, D. L.
    (2010) Vertical and or versus? Horizontal communities: Need for orientation, media use and agenda melding. Proceedings of the Annual Convention of the World Association for Public Opinion Research. WAPOR, Chicago, IL.
    [Google Scholar]
  74. Weimann, G., & Brosius, H.-B.
    (2017) Redirecting the agenda: Agenda-setting in the online Era. The Agenda Setting Journal, 1(1), 63–101.   10.1075/asj.1.1.06wei
    https://doi.org/10.1075/asj.1.1.06wei [Google Scholar]
  75. Whitney, D. C., & Becker, L. B.
    (1982) ‘Keeping the gates’ for gatekeepers: The effects of wire news. Journalism Quarterly, 59(1), 60–65.   10.1177/107769908205900109
    https://doi.org/10.1177/107769908205900109 [Google Scholar]

Data & Media loading...

  • Article Type: Research Article
Keyword(s): agenda setting; audience; content analysis; exposure; media effects; sampling
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error