Addressing Research Quality & FAQ

Addressing Research Quality & FAQ

Learn more about CNTI’s research methods, approach to our work and other frequently asked questions.

Research and legislative summaries are compiled by CNTI researchers with extensive academic training in journalism, technology and digital media research and theory. These researchers take several steps to ensure the highest quality products. These include reaching out directly to experts for input and review, searching relevant academic journals, law reviews and research organizations’ webpages, and reviewing the latest updates in news coverage across countries. 

All work – including both what sources are used as well as what inferences are drawn from the work – is then reviewed at multiple stages by CNTI executive leadership and global research and industry expert partners, including CNTI advisory committee members. Our commitment to engaging with issues on a global scale also means that we actively seek out cross-country or comparative research and a broad range of legislative cases. At times, the external resources we link to in our work (including academic journal articles and news stories) will have paywalls, but we aim to direct all readers to a wide array of sources. We invite you to send us research, legislation and other resources.

CNTI utilizes several criteria to assess the quality of quantitative, qualitative and mixed-methods research – both that CNTI conducts and that the research assesses or draws upon:

  • Reliability: the internal and external consistency and stability of results
  • Validity: the internal and external accuracy and precision of results
  • Generalizability: the transferability of results, either statistically (quantitative) or analytically (qualitative or quantitative)
  • Credibility: the motivations or aims of the research, its funders and its participants
  • Replicability (when applicable): the ability to obtain the same results when the full research process is conducted again

All research has limitations. Consistency, transparency, and awareness of the biases and limitations of methodological approaches are crucial elements of quality research. Research quality is assessed (internally by the CNTI team and collaborative partners, and externally by advisory committee members with relevant expertise) at every stage of research:

Aims and conceptual framework: Are the aims of this work clear? Do the authors have a clear grasp of the existing research in this field? Do the authors have a clear grasp of the contexts they are studying? This is particularly important in cross-disciplinary work – for instance, economists publishing research on social media – as well as in comparative and cross-country research.

Study design, sampling and methodology: Are the methods (1) systematic and (2) transparently explained? Are the methods appropriate for the research questions or hypotheses the authors are trying to address? What data are used, and how were they obtained? Do the authors note any conflicts of interest, or are any obvious conflicts of interest or important caveats not disclosed? Was the study pre-registered? For quantitative research in particular, pre-registered and open-science research is prioritized, both to ensure research results are not distorted and to promote transparency throughout the research process – including broader public access to the evidence upon which we rely. 

Data analysis: Is the methodological approach valid, reliable and (when possible) replicable? Are the data used selectively? Are statistical findings and differences careful and valid? Do the findings effectively answer the research questions or hypotheses?

Conclusions: Are limitations or sources of bias properly addressed? Do the authors address the theoretical and/or practical implications of these findings? Are conclusions correctly drawn from the data? Are the authors open about what is not answered by the data? Are the findings overgeneralized to contexts not studied in this research? 

At times, research may still provide valuable insights in spite of key limitations. When this is the case, CNTI addresses such limitations in-text, often noted in “CNTI’s Assessment” within issue primers. When considering cross-country and comparative research, not all methodological approaches can be applied equally across contexts. As such, CNTI takes into consideration what is available in or applicable to each context. 

As a registered 501(c)(3) non-profit organization, CNTI receives funding from various sources. However, we do not accept funding that exerts influence over what research is pursued, how it is conducted or what inferences are drawn from findings.

CNTI’s in-house research is conducted by researchers with extensive expertise in journalism, technology and digital media. We also collaborate with researchers, academic institutions and independent organizations around the world. For work that requires country-level expertise, we leverage these collaborative partnerships as well as our global advisory committee.

CNTI is committed to conducting objective, independent research in the public interest. The CNTI team ensures that all original and collaborative research, including comparative and cross-national projects, meets the highest standards of data quality and representativeness as well as research ethics. For research and policy analyses conducted outside the U.S., local experts will be engaged in the process to account for nuances in translation as well as sociocultural contexts. Research will be pre-registered when applicable. CNTI research ascribes to the ethical standards of the European Code of Conduct for Research Integrity as well as AAPOR’s Code of Professional Ethics and Practices. CNTI funders do not intervene in research in any capacity.

Internal assessments of data quality will be conducted by the CNTI team and executive leadership on a regular basis. As an organization committed to promoting transparent, evidence-based research, CNTI will share data, instruments and other resources publicly or upon request whenever possible. These include survey questionnaires and (when possible) data sets, interview guides, policy or other source repositories, codebooks and code. Some of these can be found on CNTI’s GitHub.

We at CNTI know the importance of keeping up with the latest developments in news and technology around the world. CNTI issue primers are updated, at minimum, on a quarterly basis, and include the date of the most recent revision at the top of each page. This work is also supplemented by CNTI’s monthly email newsletters. Longer-term trackers, such as our Aggregated Country Data, are updated as external organizations and institutions release new data (often on an annual basis).