Research Interviews and Hidden Challenges Why Your Data Might Not Tell the Whole Story

Hidden Challenges of Research Interviews Why Your Data Might Not Tell the Whole Story. Research interviews present hidden challenges, such as inaccurate participant recollections, interviewer bias, cultural differences in meaning, and difficulty addressing sensitive topics.

The Hidden Challenges of Research Interviews Why Your Data Might Not Tell the Whole Story

Furthermore, the time required for interviews, the possibility of misinterpreting subtle clues, and the influence of the interviewer’s own training on data analysis can distort the data’s meaning. Ethical considerations regarding anonymity and participant burden further complicate the process.

The Problem of Validity and Bias

Studies reported by Cannell and Kahn (1968), in which the interview was used, seemed to indicate that validity was a persistent problem. In one such study, subjects interviewed on the existence and state of their bank accounts often presented a misleading picture: fewer accounts were reported than actually existed and the amounts declared frequently differed from bank records, often in the direction of understating assets.

The cause of invalidity, they argue, is bias which they define as ‘a systematic or persistent tendency to make errors in the same direction, that is, to overstate or understate the “true value” of an attribute’. (Lansing, Ginsberg and Braaten, 1961).

The problem, it seems, is not limited to a narrow range of data but is widespread. One way of validating interview measures is to compare the interview measure with another measure that has already been shown to be valid. This kind of comparison is known as ‘convergent validity’. If the two measures agree, it can be assumed that the validity of the interview is comparable with the proven validity of the other measure. Perhaps the most practical way of achieving greater validity is to minimize the amount of bias as much as possible.

Sources of Interviewer Bias

The sources of bias are the characteristics of the interviewer, the characteristics of the respondent, and the substantive content of the questions. More particularly, these will include:

• The attitudes, opinions, and expectations of the interviewer;
• A tendency for the interviewer to see the respondent in her own image;
• A tendency for the interviewer to seek answers that support her preconceived notions;
• Misperceptions on the part of the interviewer of what the respondent is saying;
• Misunderstandings on the part of the respondent of what is being asked.

Interviewer Effects and Demographics

Studies have also shown that race, religion, gen der, sexual orientation, status, social class and age in certain contexts can be potent sources of bias, i.e. interviewer effects (Lee, 1993; Scheurich, 1995). Interviewers and interviewees alike bring their own, often unconscious experiential and biographical baggage with them into the interview situation. Indeed Hitchcock and Hughes (1989) argue that because inter views are interpersonal, humans interacting with humans, it is inevitable that the researcher will have some influence on the interviewee and, thereby, on the data.

Fielding and Fielding (1986:12) make the telling comment that even the most sophisticated surveys only manipulate data that at some time had to be gained by asking people! Interviewer neutrality is a chimera (Denscombe, 1995). Lee (1993) indicates the problems of conducting interviews perhaps at their sharpest, where the researcher is researching sensitive subjects, i.e. research that might pose a significant threat to those involved (be they interviewers or interviewees).

Here the interview might be seen as an intrusion into private worlds, or the inter viewer might be regarded as someone who can impose sanctions on the interviewee, or as someone who can exploit the powerless; the inter viewed is in the searchlight that is being held by the interviewer (see also Scheurich, 1995). The issues also embrace transference and countertransference, which have their basis in psychoanalysis. In transference the interviewees project onto the interviewer their feelings, fears, desires, needs and attitudes that derive from their own experiences (Scheurich, 1995).

In countertransference the process is reversed. One way of controlling for reliability is to have a highly structured interview, with the same format and sequence of words and questions for each respondent (Silverman, 1993), though Scheurich (1995:241–9) suggests that this is to misread the infinite complexity and open-endedness of social interaction: controlling the wording is no guarantee of controlling the interview.

Common Sources of Bias in Practice

Oppenheim (1992:147) argues that wording is a particularly important factor in attitudinal questions rather than factual questions. He suggests that changes in wording, con text and emphasis undermine reliability, because it ceases to be the same question for each respondent. Indeed he argues that error and bias can stem from alterations to wording, procedure, sequence, recording, rapport, and that training for interviewers is essential to minimize this.

Silverman (1993) suggests that it is important for each interviewee to understand the question in the same way. He suggests that the reliability of interviews can be enhanced by: careful piloting of interview schedules; training of interviewers; inter-rater reliability in the coding of responses; and the extended use of closed questions.

On the other hand Silverman (1993) argues for the importance of open-ended interviews, as this enables respondents to demonstrate their unique way of looking at the world—their definition of the situation. It recognizes that what is a suitable sequence of questions for one respondent might be less suitable for another, and open ended questions enable important but unanticipated issues to be raised. Oppenheim (1992:96–7) suggests several causes of bias in interviewing:

• Biased sampling (sometimes created by the researcher not adhering to sampling instructions)
• Poor rapport between interviewer and interviewee
• Changes to question wording (e.g. in attitudinal and factual questions)
• Poor prompting and biased probing
• Poor use and management of support materials (e.g. show cards)
• Alterations to the sequence of questions
• Inconsistent coding of responses
• Selective or interpreted recording of data/ transcripts
• Poor handling of difficult interviews. There is also the issue of leading questions

A leading question is one which makes assumptions about interviewees or ‘puts words into their mouths’, i.e. where the interviewee might be withholding. Here it might be important to put the ‘burden of denial’ onto the interviewee (e.g. ‘when did you last stop beating your wife?’). Leading questions, frequently used in police interviews, may be used for reliability checks with what the interviewee has already said, or may be deliberately used to elicit particular non-verbal behaviors that give an indication of the sensitivity of the interviewee’s remarks.

Hence reducing bias becomes more than simply: careful formulation of questions so that the meaning is crystal clear; thorough training procedures so that an interviewer is more aware of the possible problems; probability sampling of respondents; and sometimes matching inter viewer characteristics with those of the sample being interviewed. Oppenheim (1992:148) argues, for example, that interviewers seeking attitudinal responses have to ensure that people with known characteristics are included in the sample—the criterion group. We need to recognize that the interview is a shared, negotiated and dynamic social moment.

Power Dynamics in Interviews

The notion of power is significant in the interview situation, for the interview is not simply a data collection situation but a social and frequently a political situation. Power can reside with interviewer and interviewee alike, though Scheurich (1995:246) argues that, typically, more power resides with the interviewer: the inter viewer generates the questions and the interviewee answers them; the interviewee is under scrutiny whilst the interviewer is not.

This view is supported by Kvale (1996:126), who suggests that there are definite asymmetries of power as the interviewer tends to define the situation, the topics, and the course of the interview. Cassell (in Lee, 1993) suggests that elites and powerful people might feel demeaned or insulted when being interviewed by those with a lower status or less power.

Further, those with power, resources and expertise might be anxious to maintain their reputation, and so will be more guarded in what they say, wrapping this up in well-chosen, articulate phrases. Lee (1993) comments on the asymmetries of power in several question influences the answer perhaps illegitimately. For example (Morrison, 1993:66–7) the question ‘when did you stop complaining to the head teacher?’ assumes that the interviewee had been a frequent complainer and the question ‘how satisfied are you with the new Mathematics scheme?’ assumes a degree of satisfaction with the scheme.

The Issue of Leading Questions

The leading questions here might be rendered less leading by rephrasing, for example: ‘how frequently do you have conversations with the head teacher?’ and ‘what is your opinion of the new Mathematics scheme?’ respectively. In discussing the issue of leading questions we are not necessarily suggesting that there is not a place for them.

Indeed Kvale (1996:158) makes a powerful case for leading questions, arguing that they may be necessary in order to obtain information that the interviewer suspects interview situations, with one party having more power and control over the interview than the other. Interviewers need to be aware of the potentially distorting effects of power, a significant feature of critical theory, as discussed in the opening topic. Neal (1995) draws attention to the feelings of powerlessness and anxieties about physical presentation and status on the part of interviewers when interviewing powerful people.

This is particularly so for frequently lone, low status research students interviewing powerful people; a low status female research student might find that an interview with a male in a position of power (e.g. a university vice-chancellor, a dean or a senior manager) might turn out to be very different from an interview with the same per son if conducted by a male university professor where it is perceived by the interviewee to be more of a dialogue between equals (see also Gewirtz and Ozga, 1993, 1994).

Ball (1994b) comments that, when powerful people are being interviewed, interviews must be seen as an extension of the ‘play of power’—with its game like connotations. He suggests that powerful people control the agenda and course of the interview, and are usually very adept at this because they have both a personal and professional investment in being interviewed (see also Batteson and Ball, 1995; Phillips, 1998).

The effect of power can be felt even before the interview commences, notes Neal (1995), where she instances being kept waiting, and subsequently being interrupted, being patronized, and being interviewed by the interviewee (see also Walford, 1994). Indeed Scheurich (1995) suggests that many powerful interviewees will rephrase or not answer the question.

Connell, Ashenden, Kessler and Dowsett (in Limerick, Burgess-Limerick and Grace (1996)) argue that a working-class female talking with a multinational director will be very different from a middle-class professor talking to the same person. Limerick, Burgess-Limerick and Grace (1996) comment on occasions where interviewers have felt themselves to be passive, vulnerable, helpless and indeed manipulated.

Controlling for Reliability Through Structure

One way of overcoming this is to have two interviewers conducting each interview (Walford, 1994:227). On the other hand, Hitchcock and Hughes (1989) observe that if the researchers are known to the interviewees and they are peers, however powerful, then a degree of reciprocity might be taking place, with interviewees giving answers that they think the researchers might want to hear.

The issue of power has not been lost on feminist research; that is, research that emphasizes subjectivity, equality, reciprocity, collaboration, non-hierarchical relations and emancipatory potential (catalytic validity) (Neal, 1995), echoing the comments about research that is influenced by the paradigm of critical theory. Here feminist research addresses a dilemma of interviews that are constructed in the dominant, male paradigm of pitching questions that demand answers from a passive respondent.

Limerick, Burgess-Limer ick and Grace (1996) suggest that, in fact, it is wiser to regard the interview as a gift, as interviewees have the power to withhold information, to choose the location of the interview, to choose how seriously to attend to the interview, how long it will last, when it will take place, what will be discussed—and in what and whose terms—what knowledge is important, even how the data will be analyzed and used. Echoing Foucault, they argue that power is fluid and is discursively constructed through the interview rather than Bing the province of either party.

Telephone Interview Challenges

Miller and Cannell (1997) identify some particular problems in conducting telephone inter views, where the reduction of the interview situation to just auditory sensory cues can be particularly problematical. There are sampling problems, as not everyone will have a telephone. Further, there are practical issues, for example, the interviewee can only retain a certain amount of information in her/his short term memory, so bombarding the interviewee with too many choices (the non-written form of ‘show cards’ of possible responses) becomes unworkable.

Hence the reliability of responses is subject to the memory capabilities of the interviewee—how many scale points and descriptors, for example, can an interviewee retain in her head about a single item? Further, the absence of non-verbal cues is significant, e.g. facial expression, gestures, posture, the significance of silences and pauses (Robinson, 1982), as interviewees may be un clear about the meaning behind words and statements.

This problem is compounded if the interviewer is unknown to the interviewee. Miller and Cannell present important research evidence to support the significance of the nonverbal mediation of verbal dialogue. As was dis cussed earlier, the interview is a social situation; in telephone interviews the absence of essential social elements could undermine the salient con duct of the interview, and hence its reliability and validity. Non-verbal paralinguistic cues affect the conduct, pacing, and relationships in the interview and the support, threat, confidence felt by the interviewees.

Telephone interviews can easily slide into becoming mechanical and cold. Further, telephone interviewing is becoming increasingly used by general medical practitioners (the practice of ‘triaging’). Here the problem of loss of non-verbal cues is compounded by the asymmetries of power that often exist between doctor and patient. This contains a useful lesson for telephone interviews in educational research—the issue of power is itself a cogent mediating influence between researcher and re searched; the interviewer will need to take immediate steps to address these issues (e.g. by putting interviewees at their ease).

On the other hand, Nias (1991) and Miller and Cannell (1997) suggest that the very factor that interviews are not face-to-face may strengthen their reliability, as the interviewee might disclose information that may not be so readily forthcoming in a face-to-face, more intimate situation. Hence, telephone interviews have their strengths and weaknesses, and their use should be governed by the criterion of fitness for-purpose. They tend to be shorter, more focused and useful for contacting busy people (Harvey, 1988; Miller, 1995).

The Validity vs. Reliability Dilemma

In his critique of the interview as a research tool, Kitwood draws attention to the conflict it generates between the traditional concepts of validity and reliability. Where increased reliability of the interview is brought about by greater control of its elements, this is achieved, he argues, at the cost of reduced validity. He explains: In proportion to the extent to which ‘reliability’ is enhanced by rationalization, ‘validity’ would decrease.

For the main purpose of using an interview in research is that it is believed that in an interpersonal encounter people are more likely to disclose aspects of themselves, their thoughts, their feelings and values, than they would in a less human situation. At least for some purposes, it is necessary to generate a kind of conversation in which the ‘respondent’ feels at ease.

In other words, the distinctively human element in the interview is necessary to its ‘validity’. (Kitwood, 1977) Kitwood suggests that a solution to the problem of validity and reliability might lie in the direction of a ‘judicious compromise’.

Respondent-Related Challenges

A cluster of problems surround the person being interviewed. Tuckman (1972), for example, has observed that when formulating her questions an interviewer has to consider the extent to which a question might influence the respondent to show herself in a good light; or the extent to which a question might influence the respondent to be unduly helpful by attempting to anticipate what the interviewer wants to hear; or the extent to which a question might be asking for information about a respondent that she is not certain or likely to know herself.

Further, interviewing procedures are based on the assumption that the person interviewed has insight into the cause of her behavior. It has now come to be realized that insight of this kind is rarely achieved and that when it is, it is after long and difficult effort, usually in the context of repeated clinical interviews.

Special Considerations for Interviewing Children

In educational circles interviewing might be a particular problem in working with children. Simons (1982) and McCormick and James (1988) comment on particular problems involved in interviewing children, for example:

• Establishing trust
• Overcoming reticence
• Maintaining informality
• Avoiding assuming that children ‘know the answers
• Overcoming the problems of inarticulate children
• Pitching the question at the right level
• Choice of vocabulary
• Non-verbal cues
• Moving beyond the institutional response or receiving what children think the interviewer wants to hear
• Avoiding the interviewer being seen an authority spy or plant
• Keeping to the point
• Breaking silences on taboo areas and those which are reinforced by peer-group pressure;
• Children being seen as of lesser importance than adults (maybe in the sequence in which interviews are conducted, e.g. the headteacher, then the teaching staff, then the children).

These are not new matters. The studies by Labov in the 1970s showed how students reacted very strongly to contextual matters in an interview situation (Labov, 1969). The language of children varied according to the ethnicity of the interviewer, the friendliness of the surroundings, the opportunity for the children to be interviewed with friends, the ease with which the scene was set for the interview, the demeanor of the adult (e.g. whether the adult was standing or sitting), the nature of the topics covered.

The differences were significant, varying from monosyllabic responses by children in unfamiliar and uncongenial surroundings to extended responses in the more congenial and less threatening surroundings— more sympathetic to the children’s everyday world. The language, argot and jargon (Edwards, 1976), social and cultural factors of the inter viewer and interviewee all exert a powerful influence on the interview situation.

The issue is also raised here (Lee, 1993) of whether there should be a single interview that maintains the detachment of the researcher (perhaps particularly useful in addressing sensitive topics), or whether there should be repeated interviews to gain depth and to show fidelity to the collaborative nature of research (a feature, as was noted above, which is significant for feminist research (Oakley, 1981)).

Qualifications for Effective Interviewers

Kvale (1996:148–9) sets out a range of qualifications for an effective interviewer, that she should be:

• Knowledgeable (of the subject matter so that an informed conversation can be held);
• Structuring (making clear the purpose, con duct, completion of the interview);
• Clear (in choice of language, in presentation of subject matter);
• Gentle (enabling subjects to say what they want to say in its entirety and in their own time and way);
• Sensitive (employing empathic, active listening, taking account of non-verbal communication and how something is said);
• Open (sensitive to which aspects of the interview are significant for the interviewee);
• Steering (keeping to the point);
• Critical (questioning to check the reliability, consistency and validity of what is being said);
• Remembering (recalling earlier statements and relating to them during the interview);
• Interpreting (clarifying, confirming and disconfirming the interviewee’s statements with the interviewee).

Walford (1994:225) adds to this the need for the interviewer to have done her homework when interviewing powerful people, as such people could well interrogate the interviewer—they will assume up-to-dateness, competence and knowledge in the interviewer. Powerful inter viewees are usually busy people and will expect the interviewer to have read the material that is in the public domain.

Recording and Analysis Reliability Issues

The issues of reliability do not reside solely in the preparations for and conduct of the interview; they extend to the ways in which interviews are analyzed. For example, Lee (1993) and Kvale (1996:163) comment on the issue of ‘transcriber selectivity’. Here transcripts of interviews, however detailed and full they might be, remain selective, since they are interpretations of social situations.

They become decontextualized, abstracted, even if they record silences, intonation, non-verbal behavior etc. The issue, then, is how useful they are to researchers overall rather than whether they are completely reliable. One of the problems that has to be considered when open-ended questions are used in the interview is that of developing a satisfactory method of recording replies.

One way is to summarize responses in the course of the interview. This has the disadvantage of breaking the continuity of the interview and may result in bias because the interviewer may unconsciously emphasize responses that agree with her expectations and fail to note those that do not. It is sometimes possible to summarize an individual’s responses at the end of the interview.

Al though this preserves the continuity of the interview, it is likely to induce greater bias because the delay may lead to the interviewer forgetting some of the details. It is these forgotten details that are most likely to be the ones that disagree with her own expectations.

Read More:

https://nurseseducator.com/didactic-and-dialectic-teaching-rationale-for-team-based-learning/

https://nurseseducator.com/high-fidelity-simulation-use-in-nursing-education/

First NCLEX Exam Center In Pakistan From Lahore (Mall of Lahore) to the Global Nursing 

Categories of Journals: W, X, Y and Z Category Journal In Nursing Education

AI in Healthcare Content Creation: A Double-Edged Sword and Scary

Social Links:

https://www.facebook.com/nurseseducator/

https://www.instagram.com/nurseseducator/

https://www.pinterest.com/NursesEducator/

https://www.linkedin.com/in/nurseseducator/

https://www.researchgate.net/profile/Afza-Lal-Din

https://scholar.google.com/citations?hl=en&user=F0XY9vQAAAAJ

https://youtube.com/@nurseslyceum2358