In these cases, we systematically documented the modification to the form and returned to previously abstracted publications to abstract any information that might be relevant to the new field. The logic of this strategy is analogous to the logic used in a form of research synthesis called best fit framework synthesis BFFS [ 23 — 25 ]. In that method, reviewers initially code evidence using an a priori framework they have selected.
When evidence cannot be accommodated by the selected framework, reviewers then develop new themes or concepts from which they construct a new expanded framework. Both the strategy proposed and the BFFS approach to research synthesis are notable for their rigorous and transparent means to adapt a final set of concepts to the content under review. An important complication affecting the abstraction process in methods overviews is that the language used by authors to describe methods-related concepts can easily vary across publications.
For example, authors from different qualitative research traditions often use different terms for similar methods-related concepts. Furthermore, as we found in the sampling overview [ 18 ], there may be cases where no identifiable term, phrase, or label for a methods-related concept is used at all, and a description of it is given instead.
This can make searching the text for relevant concepts based on keywords unreliable. Since accepted terms may not be used consistently to refer to methods concepts, it is necessary to rely on the definitions for concepts, rather than keywords, to identify relevant information in the publication to abstract.
An effective means to systematically identify relevant information is to develop and iteratively adjust written definitions for key concepts corresponding to abstraction fields that are consistent with and as inclusive of as much of the literature reviewed as possible. Reviewers then seek information that matches these definitions rather than keywords when scanning a publication for relevant data to abstract. In the abstraction process for the sampling overview [ 18 ], we noted the several concepts of interest to the review for which abstraction by keyword was particularly problematic due to inconsistent terminology across publications: Using a method of constant comparison, we used text from definition fields to inform and modify a centrally maintained definition of the corresponding concept to optimize its fit and inclusiveness with the literature reviewed.
Final definition for qualitative sampling , including methodological tradition-specific variations. Developed after numerous iterations in the methods overview on sampling [ 18 ]. We applied iteratively developed definitions when making decisions about what specific text to abstract for an existing field, which allowed us to abstract concept-relevant data even if no recognized keyword was used.
This comparative analytic strategy and our approach to analysis more broadly as described in strategy 7, below is analogous to the process of reciprocal translation —a technique first introduced for meta-ethnography by Noblit and Hare [ 27 ] that has since been recognized as a common element in a variety of qualitative metasynthesis approaches [ 28 ].
In practice, it has been operationalized in different ways. Melendez-Torres and colleagues developed a typology from their review of the metasynthesis literature, describing four overlapping categories of specific operations undertaken in reciprocal translation: The approaches suggested in both strategies 6 and 7, with their emphasis on constant comparison, appear to fall within the line-by-line coding category.
The analysis in a systematic methods overview must support its more general objective, which we suggested above is often to offer clarity and enhance collective understanding regarding a chosen methods topic. In our experience, this involves describing and interpreting the relevant literature in qualitative terms. Furthermore, any interpretative analysis required may entail reaching different levels of abstraction, depending on the more specific objectives of the review.
For example, in the overview on sampling [ 18 ], we aimed to produce a comparative analysis of how multiple sampling-related topics were treated differently within and among different qualitative research traditions.
Considering the qualitative nature of the analysis required in systematic methods overviews, it is important to select an analytic method whose interpretations can be verified as being consistent with the literature selected, regardless of the level of abstraction reached.
We suggest employing the constant comparative method of analysis [ 29 ] because it supports developing and verifying analytic links to the source data throughout progressively interpretive or abstract levels.
In applying this approach, we advise a rigorous approach, documenting how supportive quotes or references to the original texts are carried forward in the successive steps of analysis to allow for easy verification.
The analytic approach used in the methods overview on sampling [ 18 ] comprised four explicit steps, progressing in level of abstraction—data abstraction, matrices, narrative summaries, and final analytic conclusions Fig.
While we have positioned data abstraction as the second stage of the generic review process prior to Analysis , above, we also considered it as an initial step of analysis in the sampling overview for several reasons.
First, it involved a process of constant comparisons and iterative decision-making about the fields to add or define during development and modification of the abstraction form, through which we established the range of concepts to be addressed in the review.
At the same time, abstraction involved continuous analytic decisions about what textual quotes ranging in size from short phrases to numerous paragraphs to record in the fields thus created. This constant comparative process was analogous to open coding in which textual data from publications was compared to conceptual fields equivalent to codes or to other instances of data previously abstracted when constructing definitions to optimize their fit with the overall literature as described in strategy 6.
Finally, in the data abstraction step, we also recorded our first interpretive thoughts in dedicated fields, providing initial material for the more abstract analytic steps. Summary of progressive steps of analysis used in the methods overview on sampling [ 18 ]. In the second step of the analysis, we constructed topic-specific matrices , or tables, by copying relevant quotes from abstraction forms into the appropriate cells of matrices for the complete set of analytic matrices developed in the sampling review, see Additional file 1 matrices 3 to Each matrix ranged from one to five pages; row headings, nested three-deep, identified the methodological tradition, author, and publication, respectively; and column headings identified the concepts, which corresponded to abstraction fields.
Matrices thus allowed us to make further comparisons across methodological traditions, and between authors within a tradition. In the third step of analysis, we recorded our comparative observations as narrative summaries , in which we used illustrative quotes more sparingly. In the final step, we developed analytic conclusions based on the narrative summaries about the sampling-related concepts within each methodological tradition for which clarity, consistency, or comprehensiveness of the available guidance appeared to be lacking.
Higher levels of analysis thus built logically from the lower levels, enabling us to easily verify analytic conclusions by tracing the support for claims by comparing the original text of publications reviewed.
The analytic product of systematic methods overviews is comparable to qualitative evidence syntheses, since both involve describing and interpreting the relevant literature in qualitative terms. Most qualitative synthesis approaches strive to produce new conceptual understandings that vary in level of interpretation. Dixon-Woods and colleagues [ 30 ] elaborate on a useful distinction, originating from Noblit and Hare [ 27 ], between integrative and interpretive reviews.
Integrative reviews focus on summarizing available primary data and involve using largely secure and well defined concepts to do so; definitions are used from an early stage to specify categories for abstraction or coding of data, which in turn supports their aggregation; they do not seek as their primary focus to develop or specify new concepts, although they may achieve some theoretical or interpretive functions.
For interpretive reviews, meanwhile, the main focus is to develop new concepts and theories that integrate them, with the implication that the concepts developed become fully defined towards the end of the analysis.
We suggest that most systematic methods overviews will be classifiable as predominantly integrative aggregative. Nevertheless, more highly interpretive methods overviews are also quite possible—for example, when the review objective is to provide a highly critical analysis for the purpose of generating new methodological guidance.
In such cases, reviewers may need to sample more deeply see strategy 4 , specifically by selecting empirical research reports i. In this paper, we have outlined tentative guidance in the form of seven principles and strategies on how to conduct systematic methods overviews, a review type in which methods-relevant literature is systematically analyzed with the aim of offering clarity and enhancing collective understanding regarding a specific methods topic.
Our proposals include strategies for delimiting the set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology, and generating credible and verifiable analytic interpretations.
We hope the suggestions proposed will be useful to others undertaking reviews on methods topics in future. It is important to note that our primary objective was to initiate methodological discussion by stimulating reflection on what rigorous methods for this type of review should look like, leaving the development of more complete guidance to future work.
While derived from the experience of reviewing a single qualitative methods topic, we believe the principles and strategies provided are generalizable to overviews of both qualitative and quantitative methods topics alike. However, it is expected that additional challenges and insights for conducting such reviews have yet to be defined. Formalized guidance and standards would improve the quality of future methods overviews, something we believe has important implications for advancing qualitative and quantitative methodology.
When undertaken to a high standard, rigorous critical evaluations of the available methods guidance have significant potential to make implicit controversies explicit, and improve the clarity and precision of our understandings of problematic qualitative or quantitative methods issues.
A review process central to most types of rigorous reviews of empirical studies, which we did not explicitly address in a separate review step above, is quality appraisal. The reason we have not treated this as a separate step stems from the different objectives of the primary publications included in overviews of the methods literature i.
This is not to say that appraising quality of the methods literature is not an important concern for systematic methods overviews.
As a second important difference regarding appraisal, we currently advocate appraising the aforementioned aspects at the level of the literature in aggregate rather than at the level of individual publications. One reason for this is that methods guidance from individual publications generally builds on previous literature, and thus we feel that ahistorical judgments about comprehensiveness of single publications lack relevance and utility.
Additionally, while different methods authors may express themselves less clearly than others, their guidance can nonetheless be highly influential and useful, and should therefore not be downgraded or ignored based on considerations of clarity—which raises questions about the alternative uses that quality appraisals of individual publications might have.
Finally, legitimate variability in the perspectives that methods authors wish to emphasize, and the levels of generality at which they write about methods, makes critiquing individual publications based on the criterion of clarity a complex and potentially problematic endeavor that is beyond the scope of this paper to address. By appraising the current state of the literature at a holistic level, reviewers stand to identify important gaps in understanding that represent valuable opportunities for further methodological development.
To summarize, the principles and strategies provided here may be useful to those seeking to undertake their own systematic methods overview. Additional work is needed, however, to establish guidance that is comprehensive by comparing the experiences from conducting a variety of methods overviews on a range of methods topics. Efforts that further advance standards for systematic methods overviews have the potential to promote high-quality critical evaluations that produce conceptually clear and unified understandings of problematic methods topics, thereby accelerating the advance of research methodology.
Sampling in qualitative research: The Qual Rep , 20 SJG wrote the first draft of this article, with CC contributing to drafting. All authors contributed to revising the manuscript. All authors except CC deceased approved the final draft. Cathy Charles is deceased.
National Center for Biotechnology Information , U. Journal List Syst Rev v. Published online Oct Gentles , 1, 4 Cathy Charles , 1 David B. Nicholas , 2 Jenny Ploeg , 3 and K.
Received Jun 6; Accepted Sep This article has been cited by other articles in PMC. Abstract Background Overviews of methods are potentially useful means to increase clarity and enhance collective understanding of specific methods topics that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness.
Results The guidance is organized into several principles that highlight specific objectives for this type of review given the common challenges that must be overcome to achieve them. Conclusions We believe that the principles and strategies provided here will be useful to anyone choosing to undertake a systematic methods overview.
Electronic supplementary material The online version of this article doi: Systematic review, Literature selection, Research methods, Research methodology, Overview of methods, Systematic methods overview, Review methods.
Background While reviews of methods are not new, they represent a distinct review type whose methodology remains relatively under-addressed in the literature despite the clear implications for unique review procedures. The example systematic methods overview on sampling in qualitative research The principles and strategies we propose in this paper are derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research [ 18 ].
Organization of the guidance into principles and strategies For the purposes of this article, principles are general statements outlining what we propose are important aims or considerations within a particular review process, given the unique objectives or challenges to be overcome with this type of review.
Results and discussion Literature identification and selection The identification and selection process includes search and retrieval of publications and the development and application of inclusion and exclusion criteria to select the publications that will be abstracted and analyzed in the final review.
Delimiting a manageable set of publications One aspect of methods overviews that can make identification and selection challenging is the fact that the universe of literature containing potentially relevant information regarding most methods-related topics is expansive and often unmanageably so. Searching beyond standard bibliographic databases An important reality affecting identification and selection in overviews of the methods literature is the increased likelihood for relevant publications to be located in sources other than journal articles which is usually not the case for overviews of empirical research, where journal articles generally represent the primary publication type.
Purposefully selecting literature on conceptual grounds A final consideration in methods overviews relates to the type of analysis used to generate the review findings. Open in a separate window. Data abstraction The purpose of data abstraction in rigorous literature reviews is to locate and record all data relevant to the topic of interest from the full text of included publications, making them available for subsequent analysis. Iteratively defining conceptual information to abstract In the overview on sampling [ 18 ], while we surveyed multiple sources beforehand to develop a list of concepts relevant for abstraction e.
Accounting for inconsistent terminology An important complication affecting the abstraction process in methods overviews is that the language used by authors to describe methods-related concepts can easily vary across publications.
Table 1 Final definition for qualitative sampling , including methodological tradition-specific variations.
Analysis Generating credible and verifiable analytic interpretations The analysis in a systematic methods overview must support its more general objective, which we suggested above is often to offer clarity and enhance collective understanding regarding a chosen methods topic. Integrative versus interpretive methods overviews The analytic product of systematic methods overviews is comparable to qualitative evidence syntheses, since both involve describing and interpreting the relevant literature in qualitative terms.
Conclusions In this paper, we have outlined tentative guidance in the form of seven principles and strategies on how to conduct systematic methods overviews, a review type in which methods-relevant literature is systematically analyzed with the aim of offering clarity and enhancing collective understanding regarding a specific methods topic.
Funding There was no funding for this work. Competing interests The authors declare that they have no competing interests. Consent for publication Not applicable. Ethics approval and consent to participate Not applicable. Additional file Additional file 1: Footnotes Cathy Charles is deceased.
Contributor Information Stephen J. Hutton JL, Ashcroft R. Health services research methods: BMJ Publishing Group; Cochrane handbook for systematic reviews of interventions. The Cochrane Collaboration; Centre for Reviews and Dissemination: Centre for Reviews and Dissemination; The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Barnett-Page E, Thomas J.
Methods for the synthesis of qualitative research: What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. Systematic approaches to successful literature review. Hannes K, Lockwood C. Towards methodologically inclusive research syntheses: Considering methodological options for reviews of theory: Evaluative criteria for qualitative research in health care: Int J Qual Health Care.
Guidance for developers of health research reporting guidelines. Epidemiology and reporting characteristics of systematic reviews. Epidemiology and reporting of randomised trials published in PubMed journals. Inconsistent definitions for intention-to-treat in relation to missing outcome data: Harzing A-W, Alakangas S.
Google Scholar, Scopus and the Web of Science: Google Scholar as a new source for citation analysis. Ethics Sci Environ Polit. Kousha K, Thelwall M. J Assoc Inf Sci Technol. If you do this, it is important to review the literature from an explicit angle and identify some themes to make the review distinctive. You might, for example, explore empirical debates in your chosen field across different countries or time periods.
Whilst it is possible for dissertations to be entirely literature-based, the most common form of dissertation takes the form of a case study.
Here the focus of attention is on a particular community, organisation or set of documents. The attraction of this kind of dissertation is that it stems from empirical curiosity but is at the same time practical. You may be interested in a wider question but a case study enables you to focus on a specific example. A major challenge in case study dissertations is connecting your own primary research or re-analysis with the broader theoretical themes and empirical concerns of the existing literature.
Most dissertations demand either primary or secondary research. In other words, you usually have to analyse data that you have either collected yourself or data that is already available. The reason for this is that the questions dissertations usually address take the following form: Why is x happening? Why is x changing? These questions demand primary or secondary analysis of data.
Case Study 9 Think hard before you decide to undertake empirical research: Secondary analysis is when you analyse data which was collected by another researcher.
It allows the researcher to explore areas of interest without having to go through the process of collecting data themselves in the field. The problem with using fieldwork methods in an undergraduate dissertation, however, is that they are costly in terms of time which is relatively scarce in your final year!
You may choose, therefore, to undertake secondary research, analysing existing data. There are a range of documents that already contain research data that you can analyse. You may, for example, be interested in exploring whether gender stereotypes in the media are changing. This might entail content analysis of newspapers, magazines, video or other media over different time periods. Here you would not be collecting your own data but instead would be analysing existing documents.
Download Case Study 6 Media research. If you are interested, for example, in doing historical research, you may need to visit archives. Government reports and autobiographies may also be used as data. Other documents include official statistics, datasets statistical data , and banks of interview transcripts which are all freely available to the academic community. Increasingly, documents, databases and archives are readily accessible online. Research Methods tutors on your course will be able to advise on the availability and accessibility of such data sets.
There are some advantages of doing secondary analysis, particularly if you are doing a quantitative study. You will be able to work with much larger datasets than you could have collected yourself. This has the following advantages:.
Quantitative data may also result from non-participant observations or other measurements e. Also, sometimes data that are collected through qualitative processes participant observation, interviews are coded and quantified.
Your research methods tutor can give you further information on these types of data, but here are some common quantitative data collection methods and their definitions:.
A series of questions that the respondent answers on their own. Self-completion questionnaires are good for collecting data on relatively simple topics, and for gaining a general overview of an issue. Questionnaires need to have clear questions, an easy to follow design, and not be too long.
Similar to a self-completion questionnaire, except that the questions that are asked by an interviewer to the interviewee. The same questions are read out in the same way to all respondents. There will typically be a fixed choice of answers for the respondents. Watching people and recording systematically their behaviour. Prior to the observation, an observation schedule will be produced which details what exactly the researcher should look for and how those observations should be recorded.
If you are conducting a qualitative analysis you are likely to wish to use at least some original material. This may be collected through in-depth interviews, participant observation recordings and fieldnotes, non-participant observation, or some combination of these.
Below are some data collection methods that you might want to use for your dissertation:. A way of asking questions which allows the interviewee to have more control of the interview. A form of interviewing where there are several participants; there is an emphasis in the questioning on a tightly defined topic; the accent is on interaction within the group and the joint construction of meaning.
The moderator tries to provide a relatively free rein to the discussion. This involves studying people in naturally occurring settings. The researcher participates directly in the setting and collects data in a systematic manner. The researcher will observe behaviour, listen to conversations, and ask questions.
Spend some time looking at general books about research - they will give you an overview of the data collection methods available and help you to make the best choice for your project. Bryman would be a useful starting point. For any piece of research you conduct, be it empirically based quantitative or qualitative or library based, its methods must be justified. You need to show in the final dissertation how you have given consideration to different methods, and why you have chosen and eliminated these.
Often in early supervision meetings they ask students to justify their reasons for choosing a library-based or an empirical study. Todd, Smith and Bannister , p This was particularly useful for one of our respondents:. With other essays you can rush them if you have to Todd, Bannister and Clegg, , p …. My reasons for data collection is literature based as my research question involved sensitive subjects which would have been unsuitable for primary data collection.
Level 6 students at Sheffield Hallam University I chose primary data because it would enable me to build skills that would be useful for postgraduate study. Level 6 students at Sheffield Hallam University It will involve primary data, secondary data, quantitative and qualitative research methods, lit reviews, theory and policy studies and an exploration of alternatives.
My dissertation is to be based around the experience of 'poverty', as poverty is the experience. Theories and policies are not. However, to do justice to the subject, theories and policies will be included so Iam able to demonstrate where failures in the system may exist.
Level 6 students at Sheffield Hallam University. Research must be conducted in a sensible and ethical manner; data must be analysed and presented in a rational manner. It is important that students do not expose themselves or others to dangers or risks when conducting research.
Students need the approval of their dissertation supervisor before embarking on any type of fieldwork see the section on Research Ethics for more information. In general, deductive research is theory-testing and inductive research is theory-generating. Often people link deductive research with quantitative experiments or surveys, and inductive research with qualitative interviews or ethnographic work.
These links are not hard and fast — for instance, experimental research, designed to test a particular theory through developing a hypothesis and creating an experimental design, may use quantitative or qualitative data or a combination.
If your research starts with a theory and is driven by hypotheses that you are testing e. However much research combines deductive and inductive elements. Research design is vital to conducting a good piece of work. At the start of your research you need to set down clearly:. You and your supervisor will discuss your design and decide whether the research is 'do-able'.
Your university may require you to produce a report e. Other people may have to look at the design to ascertain whether there are ethical issues that affect your research. Qualitative, Quantitative, and Mixed Methods Approaches. Researching society and culture. London, Sage Here are some references for specific methods: Interviewing for social scientists: Questionnaire Design, Interviewing and Attitude Measurement.
Identifying a research topic: A template for structured observation: Guide to undergraduate dissertations in the social sciences. Content About this site What is a Dissertation? How to start your dissertation Help with finding literature and research Formulating the research question Methodologies. Introduction What approach should I take - qualitative or quantitative?
Can my dissertation be entirely literature-based? What is case study research? What's an empirical study? What is secondary analysis? Where do I find existing research data? Collecting you own data - primary research Will my research be inductive or deductive? What about research design? Resources Further reading Research papers.
Methodologies 1 Introduction The way you approach your question will have a profound effect upon the way you construct your dissertation, so this section discusses the types of research you might undertake for your dissertation.
This video clip contains comments from the following academics: What if I want to find out about social trends, or the measurable effects of particular policies? What if I want to record people's views on an issue, and give them a 'voice'? Whether you choose qualitative or quantitative analysis will depend on several things: Your preferred philosophical approach realist, phenomenologist or constructionist. Your skills and abilities with methods of data collection if needed and analysis.
The topic or issue you are interested in. How you frame your research question. Can I combine qualitative and quantitative methods? You may be interested in doing an analysis that is primarily quantitative, looking at social trends, or policy implications.
Process involved in conducting literature based research methodology.
Literature research methodology is to read through, analyze and sort literatures in order to identify the essential attribute of materials. Its significant difference from other methodologies is that it does not directly deal with the object under.
There are many types of literature review and the following types of literature review are the most popular in business studies: Narrative literature review critiques the literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps. Early in the research process, it is important to conduct a review of the research literature on your topic to refine your research question, identify appropriate research methods, place your question in the context of other research, and prepare to write an effective research report.
This is likely to be the methodology of theoretical analysis: selection and discussion of theoretical material and descriptive material, in context, and detailed comparison of theories in terms of their applicability. What does the research literature in this field tell us about x? While all dissertations will include a literature review. LITERATURE REVIEW The research methods are divided into three broad categories; quantitative,qualitative and participatory research method. These researc.