What is Validity and Reliability in Research Understanding a Comprehensive Guide for Quantitative and Qualitative Studies in Nursing Education. In research, validity refers to the precision of a measurement in assessing the outcome being measured, while reliability refers to its consistency.
Comprehensive Guide for Quantitative and Qualitative Studies in Nursing Education for Validity and Reliability in Research Understanding
In quantitative studies, validity is typically assessed through face validity, content validity, and construct validity, while reliability is assessed through methods such as test-retest reliability and internal consistency. In contrast, qualitative research uses other concepts to demonstrate accuracy, such as dependability and truth value. Techniques such as triangulation and member checking are often employed to ensure that the results accurately reflect the data.
Why Validity and Reliability Matter in Research?
Validity and Reliability
The concepts of validity and reliability are multifaceted; there are many different types of validity and different types of reliability. Hence there will be several ways in which they can be ad dressed. It is unwise to think that threats to validity and reliability can ever be erased completely; rather, the effects of these threats can be attenuated by attention to validity and reliability throughout a piece of research. This Blog post discusses validity and reliability in quantitative and qualitative, naturalistic research.
It suggests that both terms can be applied to these two types of research, though how validity and reliability are addressed in these two approaches varies. Finally validity and reliability using different instruments for data col lection are addressed. It is suggested that reliability is a necessary but insufficient condition for validity in research; reliability is a necessary precondition of validity. Brock-Utne (1996:612) contends that the widely held view that reliability is the sole preserve of quantitative research must be exploded, and this blog post demonstrates the significance of her view.
What Is Validity?
Defining Validity
Validity is an important key to effective research. If a piece of research is invalid then it is worth less. Validity is thus a requirement for both quantitative and qualitative/naturalistic research. Whilst earlier versions of validity were based on the view that it was essentially a demonstration that a particular instrument in fact measures what it purports to measure, more recently validity has taken many forms. For example, in qualitative data validity might be addressed through the honesty, depth, richness and scope of the data achieved, the participants approached, the extent of triangulation and the disinterestedness or objectivity of the researcher.
In quantitative data validity might be improved through careful sampling, appropriate instrumentation and appropriate statistical treatments of the data. It is impossible for research to be 100 per cent valid; that is the optimism of perfection. Quantitative research possesses a measure of standard error which is inbuilt, and which has to be acknowledged.
What are Types of Validity in Research?
Types of Validity in Research: Complete Classification
In qualitative data the subjectivity of respondents, their opinions, attitudes and perspectives together contribute to a degree of bias. Validity, then, should be seen as a matter of degree rather than as an absolute state (Gronlund, 1981). Hence at best we strive to minimize invalidity and maximize validity. There are several different kinds of validity, for example:
- Content Validity
- Criterion-Related Validity
- Construct Validity
- Internal Validity
- External Validity
- Concurrent Validity
- Face Validity
- Jury Validity
- Predictive Validity
- Consequential Validity
- Systemic Validity
- Catalytic Validity
- Ecological Validity
- Cultural Validity
- Descriptive Validity
- Interpretive Validity
- Theoretical Validity
- Evaluative Validity
Validity Across Research Paradigms
It is not our intention in this blog post to discuss all these terms in depth. Rather, the main types of validity will be addressed. The argument will be made that, whilst some of these terms are more comfortably the preserve of quantitative methodologies, this is not exclusively the case. Indeed, validity is the touchstone of all types of educational research.
That said, it is important that validity in different research traditions is faithful to those traditions; it would be absurd to declare a piece of research invalid if it were not striving to meet certain kinds of validity, e.g. generalizability, replicability, controllability.
Positivist vs. Naturalistic Research: Key Principles for Validity
Hence the researcher will need to locate her discussions of validity within the research paradigm that is being used. This is not to suggest, however, that research should be paradigm-bound, that is a recipe for stagnation and conservatism. Nevertheless, validity must be faithful to its premises and positivist research has to be faithful to positivist principles, e.g.
- Controllability
- Replicability
- Predictability
- The Derivation of Laws and Universal Statements of Behavior
- Context-Freedom
- Fragmentation And Atomization of Research
- Randomization Of Samples
- Observability
Naturalistic Research Principles
By way of contrast, naturalistic research has several principles (Lincoln and Guba, 1985; Bogdan and Biklen, 1992):
- The natural setting is the principal source of data
- Context-boundedness and ‘thick description
- Data are socially situated, and socially and culturally saturated
- The researcher is part of the researched world
- As we live in an already interpreted world, a doubly hermeneutic exercise (Giddens, 1979) is necessary to understand others’ understandings of the world; the paradox here is that the most sufficiently complex instrument to understand human life is another human (Lave and Kvale, 1995; 220), but that this risks human error in all its forms.
- holism in the research;
- the researcher—rather than a research tool— is the key instrument of research;
- the data are descriptive;
- there is a concern for processes rather than simply with outcomes;
- data are analyses inductively rather than using a priori categories;
- data are presented in terms of the respondents rather than researchers;
- seeing and reporting the situation through the eyes of participants—from the native’s point of view (Geertz, 1974);
- respondent validation is important;
- catching meaning and intention are essential.
Understanding vs. Validity in Qualitative Research
Indeed Maxwell (1992) argues that qualitative researchers need to be cautious not to be working within the agenda of the positives in arguing for the need for research to demonstrate con current, predictive, convergent, criterion-related, internal and external validity. The discussion be low indicates that this need not be so. He argues, with Guba and Lincoln (1989), for the need to replace positive notions of validity in qualitative research with the notion of authenticity. Maxwell, echoing Mishler (1990), suggests that ‘understanding’ is a more suitable term than ‘validity’ in qualitative research.
We, as researchers, are part of the world that we are researching, and we cannot be completely objective about that, hence other people’s perspectives are equally as valid as our own, and the task of research is to uncover these. Validity, then, attaches to accounts, not to data or methods (Hammersley and Atkinson, 1983); it is the meaning that subjects give to data and inferences drawn from the data that are important. ‘Fidelity’ (Blumenfeld-Jones, 1995) requires the researcher to be as honest as possible to the self-reporting of the researched.
The claim is made (Agar, 1993) that, in qualitative data collection, the intensive personal involvement and in-depth responses of individuals secure a sufficient level of validity and reliability. This claim is contested by Hammersley (1992:144) and Silverman (1993:153), who argue that these are insufficient grounds for validity and reliability, and that the individuals concerned have no privileged position on interpretation. (Of course, neither are actors ‘cultural dopes’ who need a sociologist or researcher to tell them what is ‘really’ happening!).
Silverman argues that, whilst immediacy and authenticity make for interesting journalism, ethnography must have more rigorous notions of validity and reliability. This involves moving beyond selecting data simply to fit a preconceived or ideal conception of the phenomenon or because they are spectacularly interesting (Fielding and Fielding, 1986).
Data selected must be representative of the sample, the whole data set, the field, i.e. they must address content, construct and concurrent validity. Hammersley (1992:50–1) suggests that validity in qualitative research replaces certainty with confidence in our results, and that, as reality is independent of the claims made for it by researchers, our accounts will only be representations of that reality rather than reproductions of it.
Maxwell’s Five Types of Validity in Qualitative Methods
Maxwell (1992) argues for five kinds of validity in qualitative methods that explore his notion of ‘understanding’descriptive validity (the factual accuracy of the account, that it is not made up, selective, or distorted); in this respect validity subsumes reliability; it is akin to Blumenfeld-Jones’s (1995) notion of ‘truth’ in research—what actually happened (objectively factual);
- Interpretive validity (the ability of the research to catch the meaning, interpretations, terms, intentions that situations and events, i.e. data, have for the participants/subjects themselves, in their terms); it is akin to Blumenfeld-Jones’s (1995) notion of ‘fidelity’—what it means to the re searched person or group (subjectively meaningful); interpretive validity has no clear counterpart in experimental/positivist methodologies;
- Theoretical validity (the theoretical constructions that the researcher brings to the research (including those of the researched)); theory here is regarded as explanation. Theoretical validity is the extent to which the research explains phenomena; in this respect is it akin to construct validity (discussed below); in theoretical validity the constructs are those of all the participants;
- Generalizability (the view that the theory generated may be useful in understanding other similar situations); generalizing here refers to generalizing within specific groups or com munities, situations or circumstances validly) and, beyond, to specific outsider communities, situations or circumstance (external validity); internal validity has greater significance here than external validity;
- Evaluative validity (the application of an evaluative framework, judgmental of that which is being researched, rather than a descriptive, explanatory or interpretive one). Clearly this resonates with critical-theoretical perspectives, in that the researchers’ own evaluative agenda might intrude.
Internal Validity: Ensuring Accurate Research Explanations
Both qualitative and quantitative methods can address internal and external validity.
Internal validity
Internal validity seeks to demonstrate that the explanation of a particular event, issue or set of data which a piece of research provides can actually be sustained by the data. In some degree this concerns accuracy, which can be applied to quantitative and qualitative research. The findings must accurately describe the phenomena being researched.
Internal Validity in Ethnographic Research
This blog post sets out the conventional notions of validity as derived from quantitative methodologies. However, in ethnographic research internal validity can be addressed in several ways (LeCompte and Preissle, 1993:338):
- Using Low-Inference Descriptors
- Using Multiple Researchers
- Using Participant Researchers
- Using Peer Examination of Data
- Using Mechanical Means to Record, Store and Retrieve Data.
Types of Internal Validity in Qualitative Research
In ethnographic, qualitative research there are several overriding kinds of internal validity (LeCompte and Preissle, 1993:323–4):
- Confidence in the data
- The authenticity of the data (the ability of the research to report a situation through the eyes of the participants)
- The cogency of the data
- The soundness of the research design
- The credibility of the data
- The auditability of the data
- The dependability of the data
Authenticity in Research: Key Components
The conformability of the data. The writers provide greater detail on the issue of authenticity, arguing for the following:
- Fairness (that there should be a complete and balanced representation of the multiple realities in and constructions of a situation)
- Ontological authenticity (the research should provide a fresh and more sophisticated understanding of a situation, e.g. Making the familiar strange, a significant feature in reducing ‘cultural blindness’ in a researcher, a problem which might be encountered in moving from being a participant to being an observer (brock-utne, 1996:610))
- Educative authenticity (the research should generate a new appreciation of these understandings)
- Catalytic authenticity (the research gives rise to specific courses of action)
Building Credibility in Research
Tactical authenticity (the research should benefit all those involved—the ethical issue of ‘beneficence’). Hammersley (1992:71) suggests that internal validity for qualitative data requires attention to:
- Plausibility and credibility
- The kinds and amounts of evidence required (such that the greater the claim that is being made, the more convincing the evidence has to be for that claim)
- Clarity on the kinds of claim made from the research (e.g. Definitional, descriptive, explanatory, theory generative).
Lincoln and Guba (1985:219, 301) suggest that credibility in naturalistic inquiry can be ad dressed by:
- Prolonged engagement in the field
- Persistent observation (in order to establish the relevance of the characteristics for the focus);
- Triangulation (of methods, sources, investigators and theories)
- Peer debriefing (exposing oneself to a disinterested peer in a manner akin to cross-examination, in order to test honesty, working hypotheses and to identify the next steps in the research)
- Negative case analysis (in order to establish a theory that fits every case, revising hypotheses retrospectively)
- Member checking (respondent validation) to assess intentionality, to correct factual errors, to offer respondents the opportunity to add further information or to put information on record; to provide summaries and to check the adequacy of the analysis).
Whereas in positivist research history and maturation are viewed as threats to the validity of the research, ethnographic research simply assumes that this will happen; ethnographic research allows for change over time—it builds it in.
Internal validity in ethnographic research is also addressed by the reduction of observer effects by having the observers sample both widely and stay in the situation long enough for their presence to be taken for granted. Further, by tracking and storing information, it is possible for the ethnographer to eliminate rival explanations of events and situations.
External Validity: Generalizability in Research
Understanding External Validity
External validity refers to the degree to which the results can be generalized to the wider population, cases or situations. The issue of generalization is problematical. For positivist researchers generalizability is a sine qua non, whilst this is attenuated in naturalistic research. For one school of thought, generalizability through strip ping out contextual variables is fundamental, whilst, for another, generalizations that say little about the context have little that is useful to say about human behavior (Schofield, 1993).
Generalizability vs. Transferability
For positivists variables have to be isolated and controlled, and samples randomized, whilst for ethnographers human behavior is infinitely complex, irreducible, socially situated and unique. Generalizability in naturalistic research is interpreted as comparability and transferability (Lincoln and Guba, 1985; Eisenhart and Howe, 1992:647).
These writers suggest that it is possible to assess the typicality of a situation—the participants and settings, to identify possible comparison groups, and to indicate how data might translate into different settings and cultures (see also LeCompte and Preissle, 1993:348).
Thick Description and Transferability
Schofield (1992:200) suggests that it is important in qualitative research to provide a clear, detailed and in-depth description so that others can decide the extent to which findings from one piece of research are generalizable to another situation, i.e. to address the twin issues of comparability and translatability.
Indeed, qualitative research can be generalizable, the paper argues (p. 209), by studying the typical (for its applicability to other situations—the issue of transferability (LeCompte and Preissle, 1993:324)) and by performing multi-site studies (e.g. Miles and Huberman, 1984), though it could be argued that this is injecting (or infecting!) a degree of positivism into non-positivist research.
Lincoln and Guba (1985:316) caution the naturalistic researcher against this; they argue that it is not the researcher’s task to provide an index of transferability; rather, they suggest, researchers should provide sufficiently rich data for the readers and users of research to determine whether transferability is possible. In this respect transferability requires thick description.
Generalizability in Qualitative vs. Quantitative Research
Bogdan and Biklen (1992:45) argue that generalizability, construed differently from its usage in positivist methodologies, can be added in qualitative research. Positivist re searchers, they argue, are more concerned to derive universal statements of general social processes rather than to provide accounts of the degree of commonality between various social settings (e.g. schools and classrooms).
Bogdan and Biklen are more interested not with the issue of whether their findings are generalizable in the widest sense but with the question of the settings, people and situations to which they might be generalizable.
Threats to External Validity
In naturalistic research threats to external validity include (Lincoln and Guba, 1985:189, 300):
- selection effects (where constructs selected in fact are only relevant to a certain group);
- setting effects (where the results are largely a function of their context);
- history effects (where the situations have been arrived at by unique circumstances and, therefore, are not comparable);
- construct effects (where the constructs being used are peculiar to a certain group).
Content Validity: Ensuring Comprehensive Coverage
To demonstrate this form of validity the instrument must show that it fairly and comprehensively covers the domain or items that it purports to cover. It is unlikely that each issue will be able to be addressed in its entirety simply because of the time available or respondents’ motivation to complete, for example, a long questionnaire.
If this is the case, then the re searcher must ensure that the elements of the main issue to be covered in the research are both a fair representation of the wider issue under investigation (and its weighting) and that the elements chosen for the research sample are themselves addressed in depth and breadth. Careful sampling of items is required to ensure their representativeness.
For example, if the researcher wished to see how well a group of students could spell 1,000 words in French but decided only to have a sample of fifty words for the spelling test, then that test would have to ensure that it represented the range of spellings in the 1,000 words— maybe by ensuring that the spelling rules had all been included or that possible spelling errors had been covered in the test in the proportions in which they occurred in the 1,000 words.
Construct Validity: Operationalizing Abstract Concepts
Understanding Constructs in Research
A construct is an abstract; this separates it from the previous types of validity which dealt in actualities—defined content. In this type of validity agreement is sought on the ‘operationalized’ forms of a construct, clarifying what we mean when we use this construct. Hence in this form of validity the articulation of the construct is important; is my understanding of this construct similar to that which is generally accepted to be the construct?
For example, let us say that I wished to assess a child’s intelligence (assuming, for the sake of this example, that it is a unitary quality). I could say that I construed intelligence to be demonstrated in the ability to sharpen a pencil. How acceptable a construction of intelligence is this? Is not intelligence something else (e.g. that which is demonstrated by a high results in an intelligence test)?
Establishing Construct Validity
To establish construct validity I would need to be assured that my construction of a particular issue agreed with other constructions of the same underlying issue, e.g. intelligence, creativity, anxiety, motivation. This can be achieved through correlations with other measures of the issue or by rooting my construction in a wide literature search which teases out the meaning of a particular construct (i.e. a theory of what that construct is) and its constituent elements.
Demonstrating construct validity means not only confirming the construction with that given in relevant literature, but looking for counter examples which might falsify my construction. When I have balanced confirming and refuting evidence I am in a position to demonstrate construct validity. I am then in a position to stipulate what I take this construct to be. In the case of conflicting interpretations of a construct, I might have to acknowledge that conflict and then stipulate the interpretation that I shall use.
Construct Validity in Qualitative Research
In qualitative/ethnographic research construct validity must demonstrate that the categories that the researchers are using are meaningful to the participants themselves (Eisenhart and Howe, 1992:648), i.e. that they reflect the way in which the participants actually experience and construe the situations in the research; that they see the situation through the actors’ eyes.
Campbell and Fiske (1959) and Brock-Utne (1996) suggest that convergent validity implies that different methods for researching the same construct should give a relatively high inter-correlation, whilst discriminant validity suggests that using similar methods for researching different constructs should yield relatively low inter-correlations.
Ecological Validity: Research in Natural Settings
In quantitative, positivist research variables are frequently isolated, controlled and manipulated in contrived settings. For qualitative, naturalistic research a fundamental premise is that the researcher deliberately does not try to manipulate variables or conditions that the situations in the research occur naturally. The intention here is to give accurate portrayals of the realities of social situations in their own terms, in their natural or conventional settings.
In education, ecological validity is particularly important and useful in charting how policies are actually happening ‘at the chalk face’ (Brock-Utne, 1996:617). For ecological validity to be demonstrated it is important to include and address in the research as many characteristics in, and factors of, a given situation as possible. The difficulty for this is that the more characteristics are included and described, the more difficult it is to abide by central ethical tenets of much research—non-traceability, anonymity and non identifiability.
Cultural Validity in Cross-Cultural Research
A related type of validity is the emerging notion of cultural validity (Morgan, 1999). This is particularly an issue in cross-cultural, inter-cultural and comparative kinds of research, where the intention is to shape research so that it is appropriate to the culture of the researched. Cultural validity, Morgan (1999) suggests, applies at all stages of the research, and affects its planning, implementation and dissemination. It involves a degree of sensitivity to the participants, cultures and circumstances being studied.
Catalytic Validity: Research That Leads to Action
The Political Dimension of Research
Put neutrally, catalytic validity simply strives to ensure that research leads to action. However, the story does not end there, for discussions of catalytic valid ity are substantive; like critical theory, catalytic validity suggests an agenda. Lather (1986, 1991), Kincheloe and McLaren (1994) suggest that the agenda for catalytic validity is to help participants to understand their worlds in order to transform them. The agenda is explicitly political, for catalytic validity suggests the need to expose whose definitions of the situation are operating in the situation.
Empowerment Through Research
Lincoln and Guba (1986) suggest that the criterion of ‘fairness’ should be applied to research, meaning that it should:
(a) augment and improve the participants’ experience of the world
(b) that it should improve the empowerment of the participants.
In this respect the research might focus on what might be (the leading edge of innovations and future trends) and what could be (the ideal, possible futures) (Schofield, 1992:209). Catalytic validity—a major feature in feminist research which, Usher (1996) suggests, needs to permeate all research—requires solidarity in the participants, an ability of the research to promote emancipation, autonomy and freedom within a just, egalitarian and democratic society (Masschelein, 1991), to reveal the distortions, ideological deformations and limitations that reside in research, communication and social structures (see also LeCompte and Preissle, 1993).
Validity as Power and Knowledge
Validity, it is argued (Mishler, 1990; Scheurich, 1996), is no longer an ahistorical given, but contestable, suggesting that the defi nitions of valid research reside in the academic communities of the powerful. Lather (1986) calls for research to be emancipatory and to empower those who are being researched, suggesting that catalytic validity, akin to Freire’s notion of ‘conscientization’, should empower participants to understand and transform their oppressed situation.
Validity, it is proposed (Scheurich, 1996), is but a mask that in fact polices and sets boundaries to what is considered to be accept able research by powerful research communities; discourses of validity, in fact, are dis courses of power to define worthwhile knowledge. Valid research, if it is to meet the demands of catalytic validity, must demonstrate its ability to empower the researched as well as the researchers.
How defensible it is to suggest that researchers should have such ideological intents is, perhaps, a moot point, though not to address this area is to perpetuate inequality by omission and neglect. Catalytic validity reasserts the centrality of ethics in the research process, for it re quires the researcher to interrogate her allegiances, responsibilities and self-interestedness (Burgess, 1989a).
Criterion-Related Validity: Predictive and Concurrent Approaches
Understanding Criterion-Related Validity
This form of validity endeavors to relate the results of one particular instrument to another external criterion. Within this type of validity there are two principal forms: predictive validity and concurrent validity.
Predictive Validity in Practice
Predictive validity is achieved if the data acquired at the first round of research correlate highly with data acquired at a future date. For example, if the results of examinations taken by 16-year-olds correlate highly with the examination results gained by the same students when aged 18, then we might wish to say that the first examination demonstrated strong predictive validity.
Concurrent Validity Through Multiple Instruments
A variation on this theme is encountered in the notion of concurrent validity. To demonstrate this form of validity the data gathered from using one instrument must correlate highly with data gathered from using another instrument. For example, suppose I wished to research a student’s problem-solving ability. I might observe the student working on a problem, or I might talk to the student about how she is tackling the problem, or I might ask the student to write down how she tackled the problem.
Here I have three different data-collecting instruments—observation, interview and documentation respectively. If the results all agreed—concurred—that, according to given criteria for problem-solving ability, the student demonstrated a good ability to solve a problem, then I would be able to say with greater confidence (validity) that the student was good at problem-solving than if I had arrived at that judgements simply from using one instrument.
Concurrent validity is very similar to its partner—predictive validity—in its core concept (i.e. agreement with a second measure); what differentiates concurrent and predictive validity is the absence of a time element in the former; concurrence can be demonstrated simultaneously with another instrument. An important partner to concurrent validity, which is also a bridge into later discussions of reliability, is triangulation.
Triangulation: Strengthening Research Through Multiple Methods
What is Triangulation in Research?
Triangulation may be defined as the use of two or more methods of data collection in the study of some aspect of human behavior. It is a technique of research to which many subscribe in principle, but which only a minority use in practice.
In its original and literal sense, triangulation is a technique of physical measurement: maritime navigators, military strategists and surveyors, for example, use (or used to use) several locational markers in their endeavors to pinpoint a single spot or objective.
Benefits of Multi-Method Approaches
By analogy, triangular techniques in the social sciences attempt to map out, or explain more fully, the rich ness and complexity of human behavior by studying it from more than one standpoint and, in so doing, by making use of both quantitative and qualitative data. Triangulation is a powerful way of demonstrating concurrent validity, particularly in qualitative research (Campbell and Fiske, 1959). The advantages of the multimethod approach in social research are manifold and we examine two of them.
Overcoming Method Boundedness
First, whereas the single observation in fields such as medicine, chemistry and physics normally yields sufficient and unambiguous information on selected phenomena, it pro vides only a limited view of the complexity of human behavior and of situations in which human beings interact. It has been observed that as research methods act as filters through which the environment is selectively experienced, they are never a theoretical or neutral in representing the world of experience (Smith, 1975).
Exclusive reliance on one method, therefore, may bias or distort the researcher’s picture of the particular slice of reality she is investigating. She needs to be confident that the data generated are not simply artefacts of one specific method of col lection (Lin, 1976). And this confidence can only be achieved as far as normative research is concerned when different methods of data collection yield substantially the same results. (Where triangulation is used in interpretive research to investigate different actors’ viewpoints, the same method, e.g. accounts, will naturally produce different sets of data.)
Further, the more the methods contrast with each other, the greater the researcher’s confidence. If, for example, the outcomes of a questionnaire survey correspond to those of an observational study of the same phenomena, the more the researcher will be confident about the findings.
Or, more extreme, where the results of a rigorous experimental investigation are replicated in, say, a role-playing exercise, the researcher will experience even greater assurance. If findings are artefacts of method, then the use of contrasting methods considerably reduces the chances that any consistent findings are attributable to similarities of method (Lin, 1976).
Breaking Free from Limited Methods
We come now to a second advantage: some theorists have been sharply critical of the limited use to which existing methods of inquiry in the social sciences have been put (Smith, 1975). The use of triangular techniques, it is argued, will help to overcome the problem of ‘method boundedness’, as it has been termed. One of the earliest scientists to predict such a condition was Boring, who wrote: as long as a new construct has only the single operational definition that it received at birth, it is just a construct.
When it gets two alternative operational definitions, it is beginning to be validated. When the defining operations, because of proven correlations, are many, then it becomes reified. (Boring, 1953) In its use of multiple methods, triangulation may utilize either normative or interpretive techniques; or it may draw on methods from both these approaches and use them in combination.
Six Types of Triangulations for Better Research
Denzin’s Comprehensive Triangulation Framework
We have just seen how triangulation is characterized by a multi-method approach to a problem in contrast to a single-method approach. Denzin (1970) has, however, extended this view of triangulation to take in several other types as well as the multi-method kind which he terms ‘methodological triangulation’, including:
1. Time Triangulation: Addressing Temporal Factors
- time triangulation (expanded by Kirk and Miller (1986) to include diachronic reliability—stability over time—and synchronic re liability—similarity of data gathered at the same time);
2. Space Triangulation: Cross-Cultural Validation
- space triangulation;
3. Combined Levels of Triangulation: Multi-Level Analysis
- combined levels of triangulation (e.g. individual, group, organization, societal);
4. Theoretical Triangulation: Testing Competing Theories
- theoretical triangulation (drawing on alternative theories);
- investigator triangulation (more than one observer);
- methodological triangulation (using the same method on different occasions or different methods on the same object of study).
The vast majority of studies in the social sciences are conducted at one point only in time, thereby ignoring the effects of social change and process. Time triangulation goes some way to rectifying these omissions by making use of cross-sectional and longitudinal approaches. Cross-sectional studies collect data concerned with time-related processes from different groups at one point in time; longitudinal studies collect data from the same group at different points in the time sequence.
The use of panel studies and trend studies may also be mentioned in this connection. The former compare the same measurements for the same individuals in a sample at several different points in time; and the latter examine selected processes continually over time. The weaknesses of each of these methods can be strengthened by using a combined approach to a given problem.
Space triangulation attempts to overcome the limitations of studies conducted within one culture or subculture. As one writer says, ‘Not only are the behavioral sciences culture-bound, they are sub-culture-bound. Yet many such scholarly works are written as if basic principles have been discovered which would hold true as tendencies in any society, anywhere, anytime’ (Smith, 1975).
Cross-cultural studies may involve the testing of theories among different people, as in Piagetian and Freudian psychology; or they may measure differences between populations by using several different measuring instruments.
Levine describes how he used this strategy of convergent validation in his comparative studies: I have studied differences of achievement motivation among three Nigerian ethnic groups by the analysis of dream reports, written expressions of values, and public opinion survey data. The convergence of findings from the diverse set of data (and samples) strengthens my conviction…that the differences among the groups are not artifacts produced by measuring instruments. (Levine, 1966)
Social scientists are concerned in their research with the individual, the group and society. These reflect the three levels of analysis adopted by researchers in their work. Those who are critical of much present-day research argue that some of it uses the wrong level of analysis, individual when it should be societal, for instance, or limits itself to one level only when a more meaningful picture would emerge by using more than one level.
Smith extends this analysis and identifies seven possible levels: the aggregative or individual level, and six levels that are more global in that ‘they characterize the collective as a whole, and do not derive from an accumulation of individual characteristics’ (Smith, 1975). The six include:
- group analysis (the interaction patterns of individuals and groups);
- organizational units of analysis (units which have qualities not possessed by the individuals making them up);
- institutional analysis (relationships within and across the legal, political, economic and familial institutions of society);
- ecological analysis (concerned with spatial explanation);
- cultural analysis (concerned with the norms, values, practices, traditions and ideologies of a culture); and
- societal analysis (concerned with gross factors such as urbanization, industrialization, education, wealth, etc.)
Where possible, studies combining several levels of analysis are to be preferred. Researchers are sometimes taken to task for their rigid adherence to one particular theory or theoretical orientation to the exclusion of competing theories. Thus, advocates of Piaget’s developmental theory of cognition rarely take into consideration Freud’s psychoanalytic theory of development in their work; and Gestaltists work without reference to S–R theorists.
Few published works, as Smith (1975) points out, even go as far as to discuss alternative theories after a study in the light of methods used, much less consider alternatives prior to the research. As he recommends: The investigator should be more active in designing his research so that competing theories can be tested.
Research which tests competing theories will normally call for a wider range of research techniques than has historically been the case; this virtually assures more confidence in the data analysis since it is more oriented towards the testing of rival hypotheses. (Smith, 1975) Investigator triangulation refers to the use of more than one observer (or participant) in a re search setting (Silverman, 1993:99).
Observers and participants working on their own each have their own observational styles and this is reflected in the resulting data. The careful use of two or more observers or participants independently, therefore, can lead to more valid and reliable data. Smith comments: Perhaps the greatest use of investigator triangulation centers on validity rather than reliability checks. More to the point, investigators with differing perspectives or paradigmatic biases may be used to check out the extent of divergence in the data each collects.
Under such conditions if data divergence is minimal then one may feel more confident in the data’s validity. On the other hand, if their data are significantly different, then one has an idea as to possible sources of biased measurement which should be further investigated. (Smith, 1975) In this respect the notion of triangulation bridges issues of reliability and validity. We have already considered methodological triangulation earlier.
Denzin identifies two categories in his typology: ‘within methods’ triangulation and ‘between methods’ triangulation. Triangulation within methods concerns the replication of a study as a check on reliability and theory confirmation (see Smith, 1975). Triangulation between methods, as we have seen, involves the use of more than one method in the pursuit of a given objective. As a check on validity, the between methods approach embraces the notion of convergence between independent measures of the same objective as has been defined by Campbell and Fiske (1959).
Of the six categories of triangulation in Denzin’s typology, four are frequently used in education. These are: time triangulation with its longitudinal and cross-sectional studies; space triangulation as on the occasions when a number of schools in an area or across the country are investigated in some way; investigator triangulation as when two observers independently rate the same classroom phenomena; and methodological triangulation.
Of these four, methodological triangulation is the one used most frequently and the one that possibly has the most to offer. Triangular techniques are suitable when a more holistic view of educational outcomes is sought. An example of this can be found in Mortimore et al.’s (1988) search for school effectiveness. Triangulation has special relevance where a complex phenomenon requires elucidation. Multiple methods are suitable where a controversial aspect of education needs to be evaluated more fully.
Triangulation is useful when an established approach yields a limited and frequently distorted picture. Finally, triangulation can be a useful technique where a researcher is engaged in case study, a particular example of complex phenomena (Adelman et al., 1980). For an example of the use of triangular techniques in educational research we refer the reader to Blease and Cohen’s (1990) account of investigator triangulation and methodological triangulation. Triangulation is not without its critics.
For example, Silverman (1985) suggests that the very notion of triangulation is positivistic, and that this is exposed most clearly in data triangulation, as it is presumed that a multiple data source (concurrent validity) is superior to a single data source or instrument. The assumption that a single unit can always be measured more than once violates the interactionist principles of emergence, fluidity, uniqueness and specificity (Denzin, 1997:320).
Further, Patton (1980) suggests that even having multiple data sources, particularly of qualitative data, does not ensure consistency or replication. Fielding and Fielding (1986) hold that methodological triangulation does not necessarily increase validity, re duce bias or bring objectivity to research. With regard to investigator triangulation Lincoln and Guba (1985:307) do not contend that it is erroneous to assume that one investigator will corroborate another, nor is this defensible, particularly in qualitative, reflexive inquiry.
They ex tend their concern to include theory and methodological triangulation, arguing that the search for theory and methodological triangulation is epistemologically incoherent and empirically empty (see also Patton, 1980). No two theories, it is argued, will ever yield a sufficiently complete explanation of the phenomenon being re searched. These criticisms are trenchant, but they have been answered equally trenchantly by Denzin (1997)
Read More:
https://nurseseducator.com/didactic-and-dialectic-teaching-rationale-for-team-based-learning/
https://nurseseducator.com/high-fidelity-simulation-use-in-nursing-education/
First NCLEX Exam Center In Pakistan From Lahore (Mall of Lahore) to the Global Nursing
Categories of Journals: W, X, Y and Z Category Journal In Nursing Education
AI in Healthcare Content Creation: A Double-Edged Sword and Scary
Social Links:
https://www.facebook.com/nurseseducator/
https://www.instagram.com/nurseseducator/
https://www.pinterest.com/NursesEducator/
https://www.linkedin.com/in/nurseseducator/
https://www.researchgate.net/profile/Afza-Lal-Din
https://scholar.google.com/citations?hl=en&user=F0XY9vQAAAAJ