Questionnaire Design in Research: Sequencing, Layout, Piloting, and Data Processing Explained (Complete Guide 2025)

Learn how to design effective research questionnaires step by step — from sequencing and layout to covering letters, pilot testing, postal surveys, and data processing. A complete practical guide for educators and researchers.

Questionnaire Design: A Complete Step-by-Step Guide

Sequencing Questions

The order of the questions in a questionnaire, to some extent, is a function of the target sample (e.g. how they will react to certain questions), the purposes of the questionnaire (e.g. to gather facts or opinions), the sensitivity of the research (e.g. how personal and potentially disturbing the issues are that will be addressed), and the overall balance of the questionnaire (e.g. where best to place sensitive questions in relation to less threatening questions, and how many of each to include).

Understanding Question Order

The ordering of the questionnaire is important, for early questions may set the tone of, or the mind-set of the respondent to, the later questions. For example, a questionnaire that makes a respondent irritated or angry early on is unlikely to have managed to enable that respondent’s irritation or anger to subside by the end of the questionnaire. As Oppenheim remarks (1992:121) one covert purpose of each question is to ensure that the respondent will continue to co-operate.

Further, a respondent might ‘read the signs’ in the questionnaire, seeking similarities and resonances between statements, so that responses to early statements will affect responses to later statements and vice versa. Whilst multiple items may act as a cross-check, this very process might be irritating for some respondents. The key principle, perhaps, is to avoid creating a mood-set or a mind-set early on in the questionnaire.

Building Flow and Engagement

For this reason it is important to commence the questionnaire with non-threatening questions that they can readily answer. After that it might be possible to move towards more personalized questions. Completing a questionnaire can be seen as a learning process in which respondents become more at home with the task as they proceed. Initial questions should therefore be simple, have high interest value, and encourage participation. This will build up the confidence and motivation of the respondent. The middle section of the questionnaire should contain the difficult questions; the last few questions should be of high interest in order to encourage respondents to return the completed schedule. A common sequence of a questionnaire is:

1 to commence with unthreatening factual questions (that, perhaps, will give the re searcher some nominal data about the sample, e.g. age group, sex, occupation, years in post, qualifications etc.);

2 to move to closed questions (e.g. dichotomous, multiple choice, rating scales) about given statements or questions, eliciting responses that require opinions, attitudes, perceptions, views;

3 to move to more open-ended questions (or, maybe, to intersperse these with more closed questions) that seek responses on opinions, attitudes, perceptions and views, together with reasons for the responses given.

These responses and reasons might include sensitive or more personal data. The move is from objective facts to subjective attitudes and opinions through justifications and to sensitive, personalized data. Clearly the or dering is neither as discrete nor as straight forward as this. For example, an apparently innocuous question about age might be offensive to some respondents, a question about income is unlikely to go down well with somebody who has just become unemployed, and a question about religious belief might be seen as an un warranted intrusion into private matters.

The issue here is that the questionnaire designer has to anticipate the sensitivity of the topics in terms of the respondents, and this has a large socio-cultural dimension. What is being argued here is that the logical ordering of a questionnaire has to be mediated by its psychological ordering. The instrument has to be viewed through the eyes of the respondent as well as the designer.

Funneling and Filtering Techniques

In addition to the overall sequencing of the questionnaire, Oppenheim (1992: Chapter 7) suggests that the sequence within sections of the questionnaire is important. He indicates that the questionnaire designer can use funnels and filters within the question. A funneling process moves from the general to the specific, asking questions about the general context or issues and then moving toward specific points within that.

A filter is used to include and exclude certain respondents, i.e. to decide if certain questions are relevant or irrelevant to them, and to instruct respondents about how to proceed (e.g. which items to jump to or proceed to). For ex ample, if a respondent indicates a ‘yes’; or a ‘no’ to a certain question, then this might exempt her/him from certain other questions in that section or subsequently.

Questionnaires Containing Few Verbal Items

The discussion so far has assumed that questionnaires are entirely word-based. This might be off-putting for many respondents, particularly children. In these circumstances a questionnaire might include visual information and ask participants to respond to this (e.g. pictures, cartoons, diagrams) or might include some projective visual techniques (e.g. to draw a picture or diagram, to join two related pictures with a line, to write the words or what someone is saying or thinking in a ‘bubble’ picture), to tell the story of a sequence of pictures together with personal reactions to it.

The issue here is that, in tailoring the format of the questionnaire to the characteristics of the sample, a very wide embrace might be necessary to take in non-word based techniques. This is not only a matter of appeal to respondents, but, perhaps more significantly, is a matter of accessibility of the questionnaire to the respondents, i.e. a matter of re liability and validity.

The Layout of the Questionnaire

Visual Appeal and Clarity

The appearance of the questionnaire is vitally important. It must look easy, attractive and interesting rather than complicated, unclear, for bidding and boring. A compressed layout is uninviting and it clutters everything together; a larger questionnaire with plenty of space for questions and answers is more encouraging to respondents. Verma and Mallick (1999:120) also suggest the use of high quality paper if funding permits. It is important, perhaps, for respondents to be introduced to the purposes of each section of a questionnaire, so that they can become involved in it and maybe identify with it.

If space permits, it is useful to tell the respondent the purposes and foci of the sections/of the questionnaire, and the reasons for the inclusion of the items. Clarity of wording and simplicity of design are essential. Clear instructions should guide respondents: ‘Put a tick’, for example, invites participation, whereas complicated instructions and complex procedures intimidate respondents. Putting ticks in boxes by way of answering a questionnaire is familiar to most respondents, whereas requests to circle preceded numbers at the right-hand side of the questionnaire can be a source of confusion and error.

Instructions and Numbering

In some cases it might also be useful to include an example of how to fill in the questionnaire (e.g. ticking a box, circling a statement), though, clearly, care must be exercised to avoid leading the respondents to answering questions in a particular way by dint of the example provided (e.g. by suggesting what might be a desired answer to the subsequent questions).

Verma and Mallick (1999:121) suggest the use of emboldening to draw the respondent’s attention to significant features. Ensure that short, clear instructions accompany each section of the questionnaire. Repeating instructions as often as necessary is good practice in a postal questionnaire. Since everything hinges on respondents knowing exactly what is required of them, clear, unambiguous instructions, boldly and attractively displayed, are essential.

Clarity and presentation also impact on the numbering of the questions. For example a four page questionnaire might contain sixty questions, broken down into four sections. It might be off-putting to respondents to number each question (1–60) as the list will seem interminably long, whereas to number each section (1–4) makes the questionnaire look manageable. Hence it is useful, in the interests of clarity and logic to break down the questionnaire into subsections with section headings.

This will also indicate the overall logic and coherence of the questionnaire to the respondents, enabling them to ‘find their way’ through the questionnaire. It might be useful to preface each subsection with a brief introduction that tells them the purpose of that section. The practice of sectionalizing and sub lettering questions (e.g. Q9 (a) (b) (c)…) is a useful technique for grouping together questions to do with a specific issue.

It is also a way of making the questionnaire look smaller than it actually is! This previous point also requires the questionnaire designer to make it clear if respondents are exempted from completing certain questions or sections of the questionnaire (discussed earlier in the section on filters). If so, then it is vital that the sections or questions are numbered so that the respondent knows exactly where to move to next. Here the instruction might be, for example: ‘if you have answered “yes” to question 10 please go to question 15, otherwise continue with question 11’, or, for example: ‘if you are the school principal please answer this section, otherwise proceed to section three’.

Arrange the contents of the questionnaire in such a way as to maximize co-operation. For example, include questions that are likely to be of general interest. Make sure that questions which appear early in the format do not suggest to respondents that the inquiry is not intended for them. Intersperse attitude questions throughout the schedule to allow respondents to air their views rather than merely describe their behaviour. Such questions relieve boredom and frustration as well as providing valuable information in the process. Coloured pages can help to clarify the overall structure of the questionnaire and the use of different colors for instructions can assist respondents.

Ethical Considerations

It is important to include in the questionnaire, perhaps at the beginning, assurances of confidentiality, anonymity, and non-traceability, for example by indicating that they need not give their name, that the data will be aggregated, that individuals will not be able to be identified through the use of categories or details of their location etc. (i.e. that it will not be possible to put together a traceable picture of the respondents through the compiling of nominal, descriptive data about the respondents).

In some cases, however, the questionnaire might ask respondents to put their name so that they can be traced for follow-up interviews in the research (Verma and Mallick, 1999:121); here the guarantee of eventual anonymity and non-traceability will still need to be given. Finally, a brief note at the very end of the questionnaire can: (a) ask respondents to check that no answer has been inadvertently missed out; (b) solicit an early return of the completed schedule; (c) thank respondents for their participation and co-operation, and offer to send a short abstract of the major findings when the analysis is completed.

Covering Letters and Follow-up Communication

The purpose of the covering letter/sheet is to indicate the aim of the research, to convey to respondents its importance, to assure them of confidentiality, and to encourage their replies.

Purpose of the Cover Letter

The covering letter/sheet should:

Provide a title to the research;
Introduce the researcher, her/his name, ad dress, organization, contact telephone/ fax/ e-mail address, together with an invitation to feel free to contact the researcher for further clarification or details;
Indicate the purposes of the research;
Indicate the importance and benefits of the research;
Indicate any professional backing, endorsement, or sponsorship of, or permission for, the research (e.g. Professional associations, government departments);
Set out how to return the questionnaire (e.g. In the accompanying stamped, addressed envelope, in a collection box in a particular institution, to a named person; whether the questionnaire will be collected—and when, where and by whom);
Indicate the address to which to return the questionnaire;
Indicate what to do if questions or uncertain ties arise ;
Indicate a return-by date;
Indicate any incentives for completing the questionnaire;
Provide assurances of confidentiality, ano nymity and non-traceability;
Thank respondents in advance for their cooperation.

Best Practices

Verma and Mallick (1999:122) also suggest that, where possible, it is useful to personalize the letter, avoiding ‘Dear colleague’, ‘Dear Madam/ Ms/Sir’ etc., and replacing these with exact names. With these intentions in mind, the following practices are to be recommended:

The appeal in the covering letter must be tailored to suit the particular audience. Thus, a survey of teachers might stress the importance of the study to the profession as a whole.
Neither the use of prestigious signatories, nor appeals to altruism, nor the addition of hand written postscripts affect response levels to postal questionnaires.
The name of the sponsor or the organization conducting the survey should appear on the letterhead as well as in the body of the covering letter.
A direct reference should be made to the confidentiality of respondents’ answers and them purposes of any serial numbers and coding should be explained.
A pre-survey letter advising respondents of the forthcoming questionnaire has been shown to have substantial effect on response rates.
A short covering letter is most effective; aim at no more than one page.

Piloting the Questionnaire

Why Piloting is Crucial

It bears repeating that the wording of questionnaires is of paramount importance and that pre testing is crucial to its success.

Key Pilot Objectives

A pilot has several functions, principally to increase the reliability, validity and practicability of the questionnaire (Oppenheim, 1992; Morrison, 1993; Wilson and McLean, 1994:47), it thus serves:

To check the clarity of the questionnaire items, instructions and layout;
To gain feedback on the validity of the questionnaire items, the operationalization of the constructs and the purposes of the research;
To eliminate ambiguities or difficulties in wording;
To gain feedback on the type of question and its format (e.g. Rating scale, multiple choice, open, closed etc.);
To gain feedback on response categories for closed questions, and for the appropriateness of specific questions or stems of questions;
To gain feedback on the attractiveness and appearance of the questionnaire;
To gain feedback on the layout, sectionalizing, numbering and itemization of the questionnaire;
To check the time taken to complete the questionnaire;
To check whether the questionnaire is too long or too short, too easy or too difficult, too unsnagging, too threatening, too intrusive, too offensive;
To generate categories from open-ended responses to use as categories for closed response-modes (e.g. Rating scale items);
To identify redundant questions (e.g. Those questions which consistently gain a total ‘yes’ or ‘no’ response (youngman, 1984:172)), i.e. Those questions with little discriminability;
To identify commonly misunderstood or non-completed items (e.g. By studying common patterns of unexpected response and non-response (verma and mallick, 1999:120));
To try out the coding/classification system for data analysis.

In short, as Oppenheim (1992:48) remarks, everything about the questionnaire should be piloted; nothing should be excluded, not even the type face or the quality of the paper.

Practical Considerations in Questionnaire Design

Best Practices

Taking the issues discussed so far in questionnaire design, a range of practical implications for designing a questionnaire can be highlighted:

Operationalize the purposes of the questionnaire carefully.
Decide on the most appropriate type of question—dichotomous, multiple choice, rank orderings, rating scales, closed, open.
Ensure that every issue has been explored exhaustively and comprehensively; decide on the content and explore it in depth and breadth.
Ensure that the data acquired will answer the research questions.
Ask, for ease of analysis (particularly of a large sample), more closed than open questions.
Balance comprehensiveness and exhaustive coverage of issues with the demotivating factor of having respondents complete several pages of a questionnaire.
Ask only one thing at a time in a question.
Strive to be unambiguous and clear in the wording.
Be simple, clear and brief wherever possible.
Balance brevity with politeness (Oppenheim, 1992:122). It might be advantageous to re place a staccato phrase like ‘marital status with a gentler ‘please indicate whether you are married, living with a partner, or single…’or ‘I would be grateful if would tell me if you are married, living with a partner, or single’.
Ensure a balance of questions which ask for facts and opinions (this is especially true if statistical correlations and cross-tabulations are required).
Avoid leading questions.
Try to avoid threatening questions.
Do not assume that respondents know the answer, or have information to answer the questions, or will always tell the truth (wit tingly or not). Therefore include ‘don’t know’, ‘not applicable’, ‘unsure’, ‘neither agree nor disagree’ and ‘not relevant’ categories.
Avoid making the questions too hard.
Consider the readability levels of the questionnaire and the reading and writing abilities of the respondents (which may lead the researcher to conduct the questionnaire as a structured interview).
Put sensitive questions later in the questionnaire in order to avoid creating a mental set in the mind of respondents, but not so late in the questionnaire that boredom and lack of concentration have occurred.
Be very clear on the layout of the questionnaire so that it is clear and attractive (this is particularly the case if a computer program is going to be used for data analysis).
Avoid, where possible, splitting an item over more than one page, as the respondent may think that the item from the previous page is finished.
Ensure that the respondent knows how to enter a response to each question, e.g. by underlining, circling, ticking, writing; provide the instructions for introducing, completing and returning (or collection of) the questionnaire (provide a stamped addressed envelope if it is to be a postal questionnaire).
Pilot the questionnaire, using a group of respondents who are drawn from the possible sample but who will not receive the final, re fined version
Decide how to avoid falsification of responses (e.g. introduce a checking mechanism into the questionnaire responses to another question on the same topic or issue).
Be satisfied if you receive a 50 per cent response to the questionnaire; decide what you will do with missing data and what is the significance of the missing data (that might have implications for the strata of a stratified sample targeted in the questionnaire), and why the questionnaires have not been completed and returned (e.g. were the questions too threatening?, was the questionnaire too long?—this might have been signaled in the pilot).
Include a covering explanation, giving thanks for anticipated co-operation, indicating the purposes of the research, how anonymity and confidentiality will be addressed, who you are and what position you hold, and who will be party to the final report.
If the questionnaire is going to be administered by someone other than the researcher, ensure that instructions for administration are provided and that they are clear.

Respondent Perspective

A key issue that runs right through this lengthy list is for the reader to pay considerable attention to respondents, and to see the questionnaire through their eyes, and how they will regard it (e.g. from hostility to suspicion to apathy to grudging compliance to welcome; from easy to difficult, from motivating to boring, from straightforward to complex etc.).

Postal Questionnaires: Maximizing Response Rates

Advantages

Frequently, the postal questionnaire is the best form of survey in an educational inquiry. Take, for example, the researcher intent on investigating the adoption and use made of a new curriculum series in secondary schools. An interview survey based upon some sampling of the population of schools would be both expensive and time-consuming. A postal questionnaire, on the other hand, would have several distinct advantages. Moreover, given the usual constraints over finance and resources, it might well prove the only viable way of carrying through such an inquiry.

What evidence we have about the advantages and disadvantages of postal surveys derives from settings other than educational. Many of the findings, however, have relevance to the educational researcher. Here, we focus upon some of the ways in which educational researchers can maximize the response level that they obtain when using postal surveys. Research shows that a number of myths about postal questionnaires are not borne out by the evidence (see Hoinville and Jowell, 1978).

Response levels to postal surveys are not invariably less than those obtained by interview procedures; frequently they equal, and in some cases surpass, those achieved in interviews. Nor does the questionnaire necessarily have to be short in order to obtain a satisfactory response level. With sophisticated respondents, for example, a short questionnaire might appear to trivialize complex issues with which they are familiar. Hoinville and Jowell identify a number of factors in securing a good response rate to a postal questionnaire

Initial Mailing Tips

Use good-quality envelopes, typed and ad dressed to a named person wherever possible.
Use first-class—rapid—postage services, with stamped rather than franked envelopes wherever possible.
Enclose a stamped envelope for the respondent’s reply.
In surveys of the general population, Thurs day is the best day for mailing out; in surveys of organizations, Monday or Tuesday are recommended.
Avoid at all costs a December survey (questionnaires will be lost in the welter of Christ mas postings in the western world).

Follow-up Letters

Of the four factors that Hoinville and Jowell discuss in connection with maximizing response levels, the follow-up letter has been shown to be the most productive. The following points should be borne in mind in preparing reminder letters:

All of the rules that apply to the covering letter apply even more strongly to the follow up letter.
The follow-up should re-emphasize the importance of the study and the value of the respondents’ participation.
The use of the second person singular, the conveying of an air of disappointment at non response and some surprise at non-cooperation have been shown to be effective ploys.
Nowhere should the follow-up give the impression that non-response is normal or that numerous non-responses have occurred in the particular study.
The follow-up letter must be accompanied by a further copy of the questionnaire together with a stamped addressed envelope for its return.
Second and third reminder letters suffer from the law of diminishing returns, so how many follow-ups are recommended and what success rates do they achieve? It is difficult to generalize, but the following points are worth bearing in mind.

A well-planned postal survey should obtain at least a 40 per cent response rate and with the judicious use of re minders, a 70 per cent to 80 per cent response level should be possible. A preliminary pilot survey is invaluable in that it can indicate the general level of response to be expected. The main survey should generally achieve at least as high as and normally a higher level of return than the pilot inquiry.

The Government Social Survey (now the Office of Population Censuses and Surveys) recommends the use of three reminders which, they say, can increase the original return by as much as 30 per cent in surveys of the general public. A typical pattern of responses to the three follow-ups is as follows: Original dispatch First follow-up Second follow-up Third follow-up Total 40 per cent +20 per cent +10 per cent +5 per cent 75 per cent.

Incentives

An important factor in maximizing response rates is the use of incentives. Although such us age is comparatively rare in British surveys, it can substantially reduce non-response rates particularly when the chosen incentives accompany the initial mailing rather than being mailed subsequently as rewards for the return of completed schedules. The explanation of the effectiveness of this particular ploy appears to lie in the sense of obligation that is created in the recipient. Care is needed in selecting the most appropriate type of incentive. It should clearly be seen as a token rather than a payment for the respondent’s efforts and, according to Hoinville and Jowell, should be as neutral as possible.

In this respect, they suggest that books of postage stamps or ballpoint pens are cheap, easily packaged in the questionnaire envelopes, and appropriate to the task required of the respondent. The preparation of a flow chart can help the researcher to plan the timing and the sequencing of the various parts of a postal survey. One such flow chart suggested by Hoinville and Jowell (1978) is shown in Box 14.2. The researcher might wish to add a chronological chart alongside it to help plan the exact timing of the events shown here.

Ensuring Validity and Reliability

Our discussion, so far, has concentrated on ways of increasing the response rate of postal questionnaires; we have said nothing yet about the validity of this particular technique. Validity of postal questionnaires can be seen from two viewpoints according to Belson (1986). First, whether respondents who complete questionnaires do so accurately and, second, whether those who fail to return their questionnaires would have given the same distribution of answers as did the returnees. The question of accuracy can be checked by means of the intensive interview method, a technique consisting of twelve principal tactics that include familiarization, temporal reconstruction, probing and challenging,

The interested reader should consult Belson (1986:35–8). The problem of non-response (the issue of ‘volunteer bias’ as Belson calls it) can, in part, be checked on and controlled for, particularly when the postal questionnaire is sent out on a continuous basis. It involves follow-up contact with non-respondents by means of interviewers trained to secure interviews with such people. A comparison is then made between the replies of respondents and non-respondents.

Processing Questionnaire Data

Editing and Coding

Let us assume that researchers have followed the advice we have given about the planning of postal questionnaires and have secured a high response rate to their surveys. Their task is now to reduce the mass of data they have obtained to a form suitable for analysis. ‘Data reduction’, as the process is called, generally consists of coding data in preparation for analysis—by hand in the case of small surveys; by computers when numbers are larger. First, however, prior to coding, the questionnaires have to be checked.

This task is referred to as editing. Editing questionnaires is intended to identify and eliminate errors made by respondents. (In addition to the clerical editing that we discuss in this section, editing checks are also performed by the computer, e.g. SphinxSurvey, HyperRESEARCH, Results for Research™. For an account of computer-run structure checks and valid coding range checks, see also Hoinville and Jowell (1978) pp. 150–5. Moser and Kalton (1977) point to three central tasks in editing:

1 Completeness A check is made that there is an answer to every question. In most surveys, interviewers are required to record an answer to every question (a ‘not applicable’ category always being available). Missing answers can sometimes be cross-checked from other sections of the survey. At worst, respondents can be contacted again to supply the missing in formation.

2 Accuracy As far as is possible a check is made that all questions are answered accurately. In accuracies arise out of carelessness on the part of either interviewers or respondents. Sometimes a deliberate attempt is made to mislead. A tick in the wrong box, a ring round the wrong code, an error in simple arithmetic— all can reduce the validity of the data unless they are picked up in the editing process.

3 Uniformity A check is made that interviewers have interpreted instructions and questions uniformly. Sometimes the failure to give explicit instructions over the interpretation of respondents’ replies leads to interviewers recording the same answer in a variety of answer codes instead of one.

Coding for Analysis

A check on uniformity can help eradicate this source of error. The primary task of data reduction is coding, that is, assigning a code number to each answer to a survey question. Of course, not all answers to survey questions can be reduced to code numbers. Many open-ended questions, for example, are not reducible in this way for computer analysis. Coding can be built into the construction of the questionnaire itself. In this case, we talk of pre coded answers.

Where coding is developed after the questionnaire has been administered and answered by respondents, we refer to post coded answers. Pre coding is appropriate for closed-ended questions—male 1, female 0, for example; or single 0, married 1, separated 2, divorced 3. For questions such as those whose answer categories are known in advance, a cod ing frame is generally developed before the interviewing commences so that it can be printed into the questionnaire itself.Shorten with AI

For open-ended questions (Why did you choose this particular in service course rather than XYZ?), a coding frame has to be devised after the completion of the questionnaire. This is best done by taking a random sample of the questionnaires (10 per cent or more, time permitting) and generating a frequency tally of the range of responses as a preliminary to coding classification.

Having devised the coding frame, the researcher can make a further check on its validity by using it to code up a further sample of the questionnaires. It is vital to get coding frames right from the outset—extending them or making alterations at a later point in the study is both expensive and wearisome.

Digital Data Processing

There are several computer packages that will process questionnaire survey data. At the time of writing one such is SphinxSurvey. This package, like others of its type, assists researchers in the design, administration and processing of questionnaires, either for paper-based or for on screen administration. Responses can be entered rapidly, and data can be examined automatically, producing graphs and tables, as well as a wide range of statistics. (The Plus2 edition offers lexical analysis of open-ended text, and the Lexica Edition has additional functions for qualitative data analysis.)

A website for previewing a demonstration of this program can be found at http: //www.scolari.co.uk and is typical of several of its kind. Whilst coding is usually undertaken by the researcher, Sudman and Bradburn (1982:149) also make the case for coding by the respondents themselves, to increase validity. This is particularly valuable in open-ended questionnaire items, though, of course, it does assume not only the willingness of respondents to become involved post hoc but, also, that the researcher can identify and trace the respondents, which, as was indicated earlier, is an ethical matter.