Much of questionnaire construction is intuitive, but there are intricacies with which evaluation authors should be familiar. It is common sense to require that the concepts be clearly defined and questions unambiguously phrased; otherwise, the resulting data are apt to be seriously misleading. To overcome this problem, Online Course Evaluations offers a range of sample evaluation questions and responses to help you in constructing your questionnaires. Here are some ground rules to keep in mind:

• Each question should relate directly to your evaluation objectives.
• Every respondent should be able to answer every question (unless instructed otherwise).
• Each question should be phrased so that all respondents interpret it the same way.
• Each question should provide answers to what you need to know, not what would be nice to know.

The following are the four main parts of a evaluation questionnaire. Though each of these sections is different from each other, it is important to understand that all of them are necessary for drafting a good questionnaire.

Four main parts of a evaluation questionnaire

Part One: Determine the objective

Invitation

Invitation involves identifying how you are going to invite your respondents to take the evaluation. By identifying at least one tangible or intangible benefit to respondents for answering the evaluation will help you compose an invitation that encourages respondents to answer it. A tangible benefit could be in the form of an extra point on a students final exam, whereas an intangible benefit is a chance to improve the school the student is taking classes at. There are five main parts of an invitation:

1. Introduction
2. Why the respondents have been selected to respond
3. How long will the evaluation take
4. What benefit will they get for responding
5. How their responses will be used / confidentiality

Introduction

The beginning of your evaluation should include an introduction that is enticing and clearly states the purpose of your research. It is important that your introduction grabs the attention of potential respondents and encourages their participation. It is easy for online evaluation participants to abandon a evaluation, so you must communicate up-front why they should help you with your evaluation. Failure to do so will decrease the number of participants. The introduction should also include any instructions about completing the evaluation, and an estimate of how much time it will take.

Question types

Based on your evaluation objective you have to decide what types of questions will give you the information you need. Online Course Evaluations helps you develop your evaluations quickly. The different types of questions provided by Online Course Evaluations are as follows:

1. Select only one Single select questions with responses shown vertically, horizontally, in columns, or in a pull-down menu.
2. Select all that apply
   Multiple select questions where users may select several different responses to a specific question.
3. Free form text
   1 to 6000 characters long.
4. Numeric value
   Requires respondents to enter a numeric value within a range you specify.
5. Rank order
   Place in order of importance items from a defined list. You can specify the number of rank options.

Additionally, you can make certain responses required. This will enable you to have complete responses and reduce the number of incomplete questionnaires.
You can include text, instructions, or additional information at the end of your evaluation. Many evaluationors find that the "Closing Text" option under evaluation Details gives them an easy way to thank their respondents for their time and effort.

Part Two: Decide the attributes to measure
As with determining the objective, choose which attributes to measure based on your objectives, and with the data evaluation you plan to do. Several attributes you may choose to measure are:

1. Attitude
2. Knowledge
3. Skills
4. Behaviors and practices
5. Perceptions of knowledge, skills or behavior
6. Goals, intentions, aspirations
7. Demographics

Of course, it's possible you might measure more than one, but the questions will be clearly different based on the information you are trying to gather.

Part Three: Choose measurement scales.

Use scales that provide the information needed and are appropriate for respondents. Some choices are:

1. Fixed Response (Quantitative)
   • Yes-No
   • Multiple Choice
   • Rating scale/Continuum (such as Likert-type scale)
   • Rank ordering
   Fixed response questions are quick to answer and Online Course Evaluations graphs them automatically, which facilitates analyzing the results. Occasionally, however, fixed response questions may draw misleading conclusions because the respondent cannot qualify responses, e.g. "Yes, but." or "It depends" where only Yes/No are given as options.
2. Narrative Response (Qualitative)
   Narrative responses allow respondents greater freedom of expression. There is no bias due to limited response ranges and the respondents can qualify their answers. On the other hand, these responses are time consuming to code and the researcher may misinterpret (and therefore misclassify) a response.

Part Four: Check reliability

Reliability is a measure of how consistent the results of using a questionnaire will be. By consistent we mean that respondents understand the true meaning of the question as it is stated. Reliability is often first determined using a "pilot test" with the proposed questionnaire and might also be repeated with the final version.

Checking for errors

Sometimes evaluations fail to achieve their objectives due to presence of errors. Random error is the most common cause for diminished questionnaire reliability, and occurs when questions are poorly worded, or presented leading to inaccurate or uninterpretable answers. For example: A evaluation of small business owners asked, "How has the instructors lectures changed during the past year?" The response options were:

• Increased a lot
• Increased somewhat
• Increased a little
• Decreased

A better way to word this would be: During the past year, the instructor's presentation skills have

• Increased significantly
• Increased slightly
• Decreased slightly
• Decreased significantly

We can see in the second example that the question is very specific. Plus, the scale in the second question is evenly distributed versus the first one, which is biased towards the positive end of the scale. However, a great deal can usually be done to address random error issues. For example, you can reduce "random" error by removing unusual or confusing questions or by changing their arrangement. Return to top

Here are the Top 12 points to keep in mind as you write your questionnaire:

1. Be brief: You should focus on "need to know" questions and minimize "nice to know" information. Brief questionnaires have higher response rates.
2. Start with non-threatening questions: Make the first question relevant to the title/purpose, and make it easy to answer. Avoid asking for demographic information in the beginning of the evaluation.
3. Use plain language: Be direct and use simple language so that respondents can answer quicker and more accurately.
4. Include simple instructions: When necessary, include instructions about how to complete each section and how to mark the answers to ensure that the evaluation is completed correctly.
5. Make sure it looks professional: Always proofread your questionnaire and assure that the evaluation design is appropriate to the topic. A professional evaluation creates a favorable image in the mind of the respondents about you.
6. Ask only one question at a time: Avoid "double-barreled" questions that confuse the respondent. Consider the confusion created by these examples:
   • Do you like the books and the overheads?
   • Were the exams fair and cover all of the material?
7. Use open-ended questions only when the responses add value to the evaluation research.
8. Provide space to tell more: At the end of the evaluation , give respondents an opportunity to comment about the evaluation or general topic using an open-ended question.
9. Put important questions first: Respondents may get fatigued or hurried by later questions. Include questions about demographic information at the end so the earlier parts of questionnaire focus on gathering data necessary to meet your evaluation objectives.
10. Avoid agreement bias: By framing both positive and negative questions, your respondents will evaluate each question rather than uniformly agreeing or disagreeing to all of the responses.
11. Avoid the response option "other": Careless responders will overlook the option they should have designated and conveniently mark the option "other."
12. Keep your evaluation short: Limit the number of questions based on your target audience. For example you can ask more questions to customers as compared to web-site visitors.

By evaluating how important each question is to gathering the information you need, and by carefully wording the response options, you will collect information which will yield more satisfactory and meaningful results.
Return to top

Invest in the future with the innovations of today.