The Survey method – Technique of gathering data

Dr. V.K. Maheshwari, Former Principal

K.L.D.A.V (P. G) College, Roorkee, India

The Survey method is the technique of gathering data by asking questions to people who are thought to have desired information. A formal list of questionnaire is prepared. Generally a non disguised approach is used. The respondents are asked questions on their demographic interest opinion.

The survey is a non-experimental, descriptive research method. Surveys can be useful when a researcher wants to collect data on phenomena that cannot be directly observed (such as opinions on library services). Surveys are used extensively in library and information science to assess attitudes and characteristics of a wide range of subjects, from the quality of user-system interfaces to library user reading habits. In a survey, researchers sample a population. Basha and Harter (1980) state that “a population is any set of persons or objects that possesses at least one common characteristic.”

Survey research is one of the most important areas of measurement in applied social research. The broad area of survey research encompasses any measurement procedures that involve asking questions of respondents. A “survey” can be anything form a short paper-and-pencil feedback form to an intensive one-on-one in-depth interview.

Types of Survey

Different types of surveys are actually composed of several research techniques, developed by a variety of disciplines.

Data are usually collected through the use of questionnaires, although sometimes researchers directly interview subjects. Surveys can use qualitative (e.g. ask open-ended questions) or quantitative (e.g. use forced-choice questions) measures. There are two basic types of surveys: cross-sectional surveys and longitudinal surveys. Much of the following information was taken from an excellent book on the subject, called Survey Research Methods, by Earl R. Babbie.

Cross-Sectional Surveys

Cross-sectional surveys are used to gather information on a population at a single point in time

Longitudinal Surveys

Longitudinal surveys gather data over a period of time. The researcher may then analyze changes in the population and attempt to describe and/or explain them. The three main types of longitudinal surveys are trend studies, cohort studies, and panel studies.

Trend Studies

Trend studies focus on a particular population, which is sampled and scrutinized repeatedly. While samples are of the same population, they are typically not composed of the same people. Trend studies, since they may be conducted over a long period of time, do not have to be conducted by just one researcher or research project. A researcher may combine data from several studies of the same population in order to show a trend.

Cohort Studies

Cohort studies also focus on a particular population, sampled and studied more than once. But cohort studies have a different focus.

Panel Studies

Panel studies allow the researcher to find out why changes in the population are occurring, since they use the same sample of people every time. That sample is called a panel.

Techniques of Survey Method

There are mainly 4 methods by which we can collect data through the Survey Method

1. Telephonic Interview
2. Personal Interview
3. Mail Interview
4. Electronic Interview

1. Telephonic Interview

Telephone Interviewing stands out as the best method for gathering quickly needed information. Responses are collected from the respondents by the researcher on telephone.

Advantages of Telephonic Interview

1. It is very fast method of data collection.
2. It has the advantage over “Mail Questionnaire” of permitting the interviewer to talk to one or more persons and to clarifying his questions if they are not understood.
3. Response rate of telephone interviewing seems to be a little better than mail questionnaires
4. The quality of information is better
5. It is less costly method and there are less administration problems

Disadvantages of Telephonic Interview

1. 1. They cant handle interview which need props
   2. It cant handle unstructured interview
   3. It cant be used for those questions which requires long descriptive answers
   4. Respondents cannot be observed
   5. People are reluctant to disclose personal information on telephone
   6. People who don’t have telephone facility cannot be approached
2. Personal Interviewing

It is the most versatile of the all methods. They are used when props are required along with the verbal response non-verbal responses can also be observed.

Advantages of Personal Interview

1. 1. The person interviewed can ask more questions and can supplement the interview with personal observation.
   2. They are more flexible. Order of questions can be changed
   3. Knowledge of past and future is possible.
   4. In-depth research is possible.
   5. Verification of data from other sources is possible.
   6. The information obtained is very reliable and dependable and helps in establishing cause and effect relationship very early.

Disadvantages of Personal Interview

1. 1. It requires much more technical and administrative planning and supervision
   2. It is more expensive
   3. It is time consuming
   4. The accuracy of data is influenced by the interviewer
   5. A number of call banks may be required
   6. Some people are not approachable
2. Mail Survey

Questionnaires are send to the respondents, they fill it up and send it back.

Advantages of Mail Survey

1. 1. It can reach all types of people.
   2. Response rate can be improved by offering certain incentives.

Disadvantages of Mail Survey

1. 1. It can not be used for unstructured study.
   2. It is costly.
   3. It requires established mailing list.
   4. It is time consuming.
   5. There is problem in case of complex questions.
2. Electronic Interview

Electronic interviewing is a process of recognizing and noting people, objects, occurances rather than asking for information. For example-When you go to store, you notice which product people like to use. The Universal Product Code (UPC) is also a method of observing what people are buying.

Advantages of Electronic Interview

1. 1. There is no relying on willingness or ability of respondent.
   2. The data is more accurate and objective.

Disadvantages of Electronic Interview

1. 1. Attitudes can not be observed.
   2. Those events which are of long duration can not be observed.
   3. There is observer bias. It is not purely objective.
   4. If the respondents know that they are being observed, their response can be biased.
   5. It is a costly method.

Observation Method

The observation method involves human or mechanical observation of what people actually do or what events take place during a buying or consumption situation. “Information is collected by observing process at work. ”The following are a few situations:-

1. Service Stations-Pose as a customer, go to a service station and observe.
2. To evaluate the effectiveness of display of Dunlop Pillow Cushions-In a departmental store, observer notes:- a) How many pass by; b) How many stopped to look at the display; c) How many decide to buy.
3. Super Market-Which is the best location in the shelf? Hidden cameras are used.
4. To determine typical sales arrangement and find out sales enthusiasm shown by various salesmen-Normally this is done by an investigator using a concealed tape-recorder.

Advantages of Observation Method

1. If the researcher observes and record events, it is not necessary to rely on the willingness and ability of respondents to report accurately.
2. The biasing effect of interviewers is either eliminated or reduced. Data collected by observation are, thus, more objective and generally more accurate.

Disadvantages of Observation Method

1. The most limiting factor in the use of observation method is the inability to observe such things such as attitudes, motivations, customers/consumers state of mind, their buying motives and their images.
2. It also takes time for the investigator to wait for a particular action to take place.
3. Personal and intimate activities, such as watching television late at night, are more easily discussed with questionnaires than they are observed.
4. Cost is the final disadvantage of observation method. Under most circumstances, observational data are more expensive to obtain than other survey data. The observer has to wait doing nothing, between events to be observed. The unproductive time is an increased cost.

Categories   of Surveys

Surveys can be divided into two broad categories: the questionnaire and the interview. Questionnaires are usually paper-and-pencil instruments that the respondent completes. Interviews are completed by the interviewer based on the respondent says.

Questionnaires

When most people think of questionnaires, they think of the mail survey. There are many advantages to mail surveys. They are relatively inexpensive to administer. You can send the exact same instrument to a wide number of people. They allow the respondent to fill it out at their own convenience. But there are some disadvantages as well. Response rates from mail surveys are often very low. And, mail questionnaires are not the best vehicles for asking for detailed written responses.

A second type is the group administered questionnaire. A sample of respondents is brought together and asked to respond to a structured sequence of questions. Traditionally, questionnaires were administered in group settings for convenience. The researcher could give the questionnaire to those who were present and be fairly sure that there would be a high response rate. If the respondents were unclear about the meaning of a question they could ask for clarification. And, there were often organizational settings where it was relatively easy to assemble the group .

A less familiar type of questionnaire is the household drop-off survey. In this approach, a researcher goes to the respondent’s home or business and hands the respondent the instrument. In some cases, the respondent is asked to mail it back or the interview returns to pick it up. This approach attempts to blend the advantages of the mail survey and the group administered questionnaire. Like the mail survey, the respondent can work on the instrument in private, when it’s convenient. Like the group administered questionnaire, the interviewer makes personal contact with the respondent — they don’t just send an impersonal survey instrument. And, the respondent can ask questions about the study and get clarification on what is to be done. Generally, this would be expected to increase the percent of people who are willing to respond.

Interviews

Interviews are a far more personal form of research than questionnaires. In the personal interview, the interviewer works directly with the respondent. Unlike with mail surveys, the interviewer has the opportunity to probe or ask follow-up questions. And, interviews are generally easier for the respondent, especially if what is sought is opinions or impressions. Interviews can be very time consuming and they are resource intensive.

Telephone interviews enable a researcher to gather information rapidly. Most of the major public opinion polls that are reported were based on telephone interviews. Like personal interviews, they allow for some personal contact between the interviewer and the respondent. And, they allow the interviewer to ask follow-up questions.

Selecting the Survey Method

Selecting the type of survey you are going to use is one of the most critical decisions in many  research contexts.

Population Issues

The first set of considerations have to do with the population and its accessibility.

• Can the population be enumerated?

For some populations, you have a complete listing of the units that will be sampled. For others, such a list is difficult or impossible to compile.

• Is the population literate?

Questionnaires require that your respondents can read. While this might seem initially like a reasonable assumption for many adult populations, we know from recent research that the instance of adult illiteracy is alarmingly high. Young children would not be good targets for questionnaires.

• Are there language issues?

We live in a multilingual world. Virtually every society has members who speak other than the predominant language.

• Will the population cooperate?

People who do research on immigration issues have a difficult methodological problem. Why would we expect those respondents to cooperate? Although the researcher may mean no harm, the respondents are at considerable risk legally if information they divulge should get into the hand of the authorities.

• What are the geographic restrictions?

Is your population of interest dispersed over too broad a geographic range for you to study feasibly with a personal interview?

Sampling Issues

The sample is the actual group you will have to contact in some way. There are several important sampling issues you need to consider when doing survey research.

• What data is available?

What information do you have about your sample? Do you know their current addresses? Their current phone numbers? Are your contact lists up to date?

• Can respondents be found?

Can your respondents be located? Some people are very busy. Some travel a lot. Some work the night shift. Even if you have an accurate phone or address, you may not be able to locate or make contact with your sample.

• Can all members of population be sampled?

If you have an incomplete list of the population (i.e., sampling frame) you may not be able to sample every member of the population. Lists of various groups are extremely hard to keep up to date. People move or change their names. Even though they are on your sampling frame listing, you may not be able to get to them.

• Are response rates likely to be a problem?

Even if you are able to solve all of the other population and sampling problems, you still have to deal with the issue of response rates. Some members of your sample will simply refuse to respond. Others have the best of intentions, but can’t seem to find the time to send in your questionnaire by the due date. Still others misplace the instrument or forget about the appointment for an interview. Low response rates are among the most difficult of problems in survey research.

Content Issues

The content of your study can also pose challenges for the different survey types you might utilize.

• Can the respondents be expected to know about the issue?

If the respondent does not keep up with the news (e.g., by reading the newspaper, watching television news, or talking with others), they may not even know about the news issue you want to ask them about.

• Will respondent need to consult records?

Even if the respondent understands what you’re asking about, you may need to allow them to consult their records in order to get an accurate answer.

Administrative Issues

Last, but certainly not least, you have to consider the feasibility of the survey method for your study.

• costs

Cost is often the major determining factor in selecting survey type. You might prefer to do personal interviews, but can’t justify the high cost of training and paying for the interviewers.

• facilities

Do you have the facilities (or access to them) to process and manage your study? In phone interviews, do you have well-equipped phone surveying facilities? For focus groups, do you have a comfortable and accessible room to host the group? Do you have the equipment needed to record and transcribe responses?

• time

Some types of surveys take longer than others. Have you allowed for enough time to get enough personal interviews to justify that approach?

• personnel

Different types of surveys make different demands of personnel. Interviews require interviewers who are motivated and well-trained. Group administered surveys require people who are trained in group facilitation. Some studies may be in a technical area that requires some degree of expertise in the interviewer.

Question Issues

Sometimes the nature of what you want to ask respondents will determine the type of survey you select.

• What types of questions can be asked?

Are you going to be asking personal questions? Are you going to need to get lots of detail in the responses? Can you anticipate the most frequent or important types of responses and develop reasonable closed-ended questions?

• How complex will the questions be?

Sometimes you are dealing with a complex subject or topic. The questions you want to ask are going to have multiple parts. You may need to branch to sub-questions.

• Will screening questions be needed?

A screening question may be needed to determine whether the respondent is qualified to answer your question of interest. For instance, you wouldn’t want to ask someone their opinions about a specific computer program without first “screening” them to find out whether they have any experience using the program.

• Can question sequence be controlled?

Is your survey one where you can construct in advance a reasonable sequence of questions? Or, are you doing an initial exploratory study where you may need to ask lots of follow-up questions that you can’t easily anticipate?

• Will lengthy questions be asked?

If your subject matter is complicated, you may need to give the respondent some detailed background for a question. Can you reasonably expect your respondent to sit still long enough in a phone interview to ask your question?

• Will long response scales be used?

If you are asking people about the different computer equipment they use, you may have to have a lengthy response list (CD-ROM drive, floppy drive, mouse, touch pad, modem, network connection, external speakers, etc.). Clearly, it may be difficult to ask about each of these in a short phone interview.

Bias Issues

People come to the research endeavor with their own sets of biases and prejudices. Sometimes, these biases will be less of a problem with certain types of survey approaches.

• Can social desirability be avoided?

Respondents generally want to “look good” in the eyes of others. None of us likes to look like we don’t know an answer. We don’t want to say anything that would be embarrassing. If you ask people about information that may put them in this kind of position, they may not tell you the truth, or they may “spin” the response so that it makes them look better.

• Can interviewer distortion and subversion be controlled?

Interviewers may distort an interview as well. They may not ask questions that make them uncomfortable. They may not listen carefully to respondents on topics for which they have strong opinions

• Can false respondents be avoided?

With mail surveys it may be difficult to know who actually responded. Is the person you’re speaking with on the phone actually who they say they are? At least with personal interviews, you have a reasonable chance of knowing who you are speaking with. In mail surveys or phone interviews, this may not be the case.

Constructing the Survey

Constructing a survey instrument is an art in itself. There are numerous small decisions that must be made — about content, wording, format, placement — that can have important consequences for your entire study.

First of all you’ll learn about the two major types of surveys that exist, the questionnaire and the interview and the different varieties of each. There are three areas involved in writing a question:

• choosing the response format that you use for collecting information from the respondent
• determining the question content, scope and purpose
• figuring out how to word the question to get at the issue of interest

Finally, once you have your questions written, there is the issue of how best to place them in your survey.

Types Of Questions

Survey questions can be divided into two broad types:

1. Structured
2. Unstructured.

From an instrument design point of view, the structured questions pose the greater difficulties (see Decisions About the Response Format). From a content perspective, it may actually be more difficult to write good unstructured questions.

Dichotomous Questions

When a question has two possible responses, we consider it dichotomous. Surveys often use dichotomous questions that ask for a Yes/No, True/False or Agree/Disagree response.

Questions Based on Level of Measurement

We can also classify questions in terms of their level of measurement

We can also construct survey questions that attempt to measure on an interval level. One of the most common of these types is the traditional 1-to-5 rating (or 1-to-7, or 1-to-9, etc.). This is sometimes referred to as a Likert response scale . Here, we see how we might ask an opinion question on a 1-to-5 bipolar scale (it’s called bipolar because there is a neutral point and the two ends of the scale are at opposite positions of the opinion):

Another interval question uses an approach called the semantic differential. Here, an object is assessed by the respondent on a set of bipolar adjective pairs (using 5-point rating scale):

Finally, we can also get at interval measures by using what is called a cumulative or Guttman scale. Here, the respondent checks each item with which they agree. The items themselves are constructed so that they are cumulative — if you agree to one, you probably agree to all of the ones above it in the list:

Filter or Contingency Questions

Sometimes you have to ask the respondent one question in order to determine if they are qualified or experienced enough to answer a subsequent one. This requires using a filter or contingency question.

Filter questions can get very complex. Sometimes, you have to have multiple filter questions in order to direct your respondents to the correct subsequent questions. There are a few conventions you should keep in mind when using filters:

•             try to avoid having more than three levels (two jumps) for any question

Too many jumps will confuse the respondent and may discourage them from continuing with the survey.

•             if only two levels, use graphic to jump (e.g., arrow and box)

Question Content

For each question in your survey, you should ask yourself how well it addresses the content you are trying to get at.

Do Respondents Have the Needed Information?

Look at each question in your survey to see whether the respondent is likely to have the necessary information to be able to answer the question

Does the Question Need to be More Specific?

Sometimes we ask our questions too generally and the information we obtain is more difficult to interpret.

Is Question Sufficiently General?

You can err in the other direction as well by being too specific.

Is Question Biased or Loaded?

One danger in question-writing is that your own biases and blind-spots may affect the wording .

Will Respondent Answer Truthfully?

For each question on your survey, ask yourself whether the respondent will have any difficulty answering the question truthfully. If there is some reason why they may not, consider rewording the question.

Response Format

The response format is how you collect the answer from the respondent.

Structured Response Formats

Structured formats help the respondent to respond more easily and help the researcher to accumulate and summarize responses more efficiently. But, they can also constrain the respondent and limit the researcher’s ability to understand what the respondent really means. There are many different structured response formats, each with its own strengths and weaknesses. We’ll review the major ones here.

Fill-In-The-Blank. One of the simplest response formats is a blank line. A blank line can be used for a number of different response types

Check the Answer. The respondent places a check next to the response(s). The simplest form would be the example given above where we ask the person to indicate their gender.

Circle the Answer. Sometimes the respondent is asked to circle an item to indicate their response. Usually we are asking them to circle a number

Unstructured Response Formats

While there is a wide variety of structured response formats, there are relatively few unstructured ones. Generally, it’s written text. If the respondent (or interviewer) writes down text as the response, you’ve got an unstructured response format. In almost every short questionnaire, there’s one or more short text field questions

Question Wording

One of the major difficulties in writing good survey questions is getting the wording right. Even slight wording differences can confuse the respondent or lead to incorrect interpretations of the question.

• Can the Question be Misunderstood?
• How personal is the wording?
• Is the time frame specified?
• Some terms are just too vague to be useful
• What Assumptions Does the Question Make?

Other Wording Issues

The nuances of language guarantee that the task of the question writer will be endlessly complex. Without trying to generate an exhaustive list, here are a few other questions to keep in mind:

• Decisions About Placement
• Does the question contain difficult or unclear terminology?
• Does the question make each alternative explicit?
• Is the wording loaded or slanted?
• Is the wording objectionable?
• One of the most difficult tasks facing the survey designer involves the ordering of questions.
• Question Placement

Whenever you think about question placement, consider the following questions:

• Does question come too early or too late to arouse interest?
• Does the question receive sufficient attention?
• Is the answer influenced by prior questions?
• The Opening Questions

Just as in other aspects of life, first impressions are important in survey work. The first few questions you ask will determine the tone for the survey, and can help put your respondent at ease.. You should never begin your survey with sensitive or threatening questions.

The Golden Rule

You are imposing in the life of your respondent. You are asking for their time, their attention, their trust, and often, for personal information. Therefore, you should always keep in mind the “golden rule” of survey research :

• Assure the respondent that you will send a copy of the final results
• Be alert for any sign that the respondent is uncomfortable
• Be sensitive to the needs of the respondent
• Do unto your respondents as you would have them do unto you!
• Keep your survey as short as possible — only include what is absolutely necessary
• Thank the respondent at the beginning for allowing you to conduct your study
• Thank the respondent at the end for participating

Interviews

Interviews are among the most challenging and rewarding forms of measurement. They require a personal sensitivity and adaptability as well as the ability to stay within the bounds of the designed protocol.

Preparation

• The Role of the Interviewer

• . Respondents may raise objections or concerns that were not anticipated. The interviewer has to be able to respond candidly and informatively.
• Clarify any confusion/concerns
• Last, and certainly not least, the interviewer has to conduct a good interview
• Locate and enlist cooperation of respondents
• Motivate respondents to do good job
• Observe quality of responses
• The interviewer has to be motivated and has to be able to communicate that motivation to the respondent. Often, this means that the interviewer has to be convinced of the importance of the research.
• The interviewer has to find the respondent. In door-to-door surveys, this means being able to locate specific addresses
• The interviewer is really the “jack-of-all-trades” in survey research. The interviewer’s role is complex and multifaceted. It includes the following tasks:
• Whether the interview is personal or over the phone, the interviewer is in the best position to judge the quality of the information that is being receivedConduct a good interview

Training the Interviewers

One of the most important aspects of any interview study is the training of the interviewers themselves. Here are some of the major topics that should be included in interviewer training:

• Describe the entire study
• Explain interviewer bias
• Explain respondent selection procedures, including reading maps
• Explain scheduling
• Explain supervision
• Explain the sampling logic and process
• identify respondents
• Interviewers need to  learn about the background for the study, previous work that has been done, and why the study is important.
• Rehearse interview
• State who is sponsor of research
• Teach enough about survey research
• When you first introduce the interview, it’s a good idea to walk through the entire protocol so the interviewers can get an idea of the various parts or phases and how they interrelate.

The Interviewer’s Kit

Usually, you will want to assemble an interviewer kit that can be easily carried and includes all of the important materials such as:

• a “professional-looking” 3-ring notebook (this might even have the logo of the organization conducting the interviews)
• a cover letter from the Principal Investigator or Sponsor
• a phone number the respondent can call to verify the interviewer’s authenticity
• maps
• official identification (preferable a picture ID)
• sufficient copies of the survey instrument

The Interview

Every interview includes some common components. There’s the opening, where the interviewer gains entry and establishes the rapport and tone for what follows. There’s the middle game, the heart of the process, that consists of the protocol of questions and the improvisations of the probe. And finally, there’s the endgame, the wrap-up, where the interviewer and respondent establish a sense of closure. Whether it’s a two-minute phone interview or a personal interview that spans hours, the interview is a bit of theater, a mini-drama that involves real lives in real time.

Opening Remarks

In many ways, the interviewer has the same initial problem that a salesperson has. You have to get the respondent’s attention initially for a long enough period that you can sell them on the idea of participating in the study.

Gaining entry

• The first thing the interviewer must do is gain entry. Several factors can enhance the prospects. Probably the most important factor is your initial appearance. The interviewer needs to dress professionally and in a manner that will be comfortable to the respondent. The way the interviewer appears initially to the respondent has to communicate some simple messages — that you’re trustworthy, honest, and non-threatening

Asking the Questions

• Use questionnaire carefully, but informally
• The questionnaire is your friend. It was developed with a lot of care and thoughtfulness.
• Ask questions exactly as written
• Follow the order given
• Ask every question
• Don’t finish sentences

Obtaining Adequate Responses – The Probe

• Silent probe-The most effective way to encourage someone to elaborate is to do nothing at all – just pause and wait
• . Overt encouragement- Overt encouragement could be as simple as saying “Uh-huh” or “OK” after the respondent completes a thought.
• Elaboration- You can encourage more information by asking for elaboration
• Ask for clarification- Sometimes, you can elicit greater detail by asking the respondent to clarify something that was said earlier.
• Repetition-This is the old psychotherapist trick. You say something without really saying anything new.

Recording the Response

Although we have the capability to record a respondent in audio and/or video, most interview methodologists don’t think it’s a good idea. In general, personal interviews are still best when recorded by the interviewer using pen and paper. Here, I assume the paper-and-pencil approach.

• Include all probes
• Record responses immediately
• The interviewer should record responses as they are being stated.
• Use abbreviations where possible

Concluding the Interview

• When you’ve gone through the entire interview, you need to bring the interview to closure. Some important things to remember:
• Thank the respondent
• Don’t forget to do this. Even if the respondent was troublesome or uninformative, it is important for you to be polite and thank them for their time.
• Tell them when you expect to send results
• . It’s common practice to prepare a short, readable, jargon-free summary of interviews that you can send to the respondents.
• Don’t be brusque or hasty
• Allow for a few minutes of winding down conversation. The respondent may want to know a little bit about you or how much you like doing this kind of work. They may be interested in how the results will be used. Use these kinds of interests as a way to wrap up the conversation.
• You have to find a way to politely cut off the conversation and make your exit.
• Immediately after leaving — write down any notes about how the interview went

Analyzing Survey Results

After creating and conducting your survey, you must now process and analyze the results. These steps require strict attention to detail and, in some cases, knowledge of statistics and computer software packages.

Processing the Results

It is clearly important to keep careful records of survey data in order to do effective work. Most researchers recommend using a computer to help sort and organize the data. Additionally, Glastonbury and MacKean point out that once the data has been filtered though the computer, it is possible to do an unlimited amount of analysis.

Jolliffe (1986) believes that editing should be the first step to processing this data. He writes, “The obvious reason for this is to ensure that the data analyzed are correct and complete . At the same time, editing can reduce the bias, increase the precision and achieve consistency between the tables. Of course, editing may not always be necessary, if for example you are doing a qualitative analysis of open-ended questions, or the survey is part of a larger project and gets distributed to other agencies for analysis. However, editing could be as simple as checking the information input into the computer.

All of this information should be used to test for statistical significance. Information may be recorded in any number of ways. Charts and graphs are clear, visual ways to record findings in many cases. For instance, in a mail-out survey where response rate is an issue, you might use a response rate graph to make the process easier. The day the surveys are mailed out should be recorded first. Then, every day thereafter, the number of returned questionnaires should be logged on the graph. Be sure to record both the number returned each day, and the cumulative number, or percentage. Also, as each completed questionnaire is returned, each should be opened, scanned and assigned an identification number.

Analyzing the Results

Before actually beginning the survey the researcher should know how they want to analyze the data. If you are collecting quantifiable data, a code book is needed for interpreting your data and should be established prior to collecting the survey data. This is important because there are many different formulas needed in order to properly analyze the survey research and obtain statistical significance. Since computer programs have made the process of analyzing data vastly easier than it was, it would be sensible to choose this route

After the survey is conducted and the data collected, the results must be assembled in some useable format that allows comparison within the survey group, between groups, or both. The results could be analyzed in a number of ways. A T-test may be used to determine if scores of two groups differ on a single variable–whether writing ability differs among students in two classrooms, for instance. Correlation measurements could also be constructed to compare the results of two interacting variables within the data set.

Reliability and Validity

Surveys tend to be weak on validity and strong on reliability. The artificiality of the survey format puts a strain on validity. Since people’s real feelings are hard to grasp in terms of such dichotomies as “agree/disagree,” “support/oppose,” “like/dislike,” etc., these are only approximate indicators of what we have in mind when we create the questions. Reliability, on the other hand, is a clearer matter. Survey research presents all subjects with a standardized stimulus, and so goes a long way toward eliminating unreliability in the researcher’s observations. Careful wording, format, content, etc. can reduce significantly the subject’s own unreliability.

 

Strengths and Weaknesses of Surveys

Strengths:

• Consequently, very large samples are feasible, making the results statistically significant even when analyzing multiple variables.
• Many questions can be asked about a given topic giving considerable flexibility to the analysis.
• Standardization ensures that similar data can be collected from groups then interpreted comparatively (between-group study).
• Standardized questions make measurement more precise by enforcing uniform definitions upon the participants.
• Surveys are relatively inexpensive (especially self-administered surveys).
• Surveys are useful in describing the characteristics of a large population. No other method of observation can provide this general capability.
• There is flexibilty at the creation phase in deciding how the questions will be administered: as face-to-face interviews, by telephone, as group administered written or oral survey, or by electonic means.
• They can be administered from remote locations using mail, email or telephone.
• Usually, high reliability is easy to obtain–by presenting all subjects with a standardized stimulus, observer subjectivity is greatly eliminated.

Weaknesses:

• A methodology relying on standardization forces the researcher to develop questions general enough to be minimally appropriate for all respondents, possibly missing what is most appropriate to many respondents.
• As opposed to direct observation, survey research (excluding some interview approaches) can seldom deal with “context.”
• It may be hard for participants to recall information or to tell the truth about a controversial question.
• Surveys are inflexible in that they require the initial study design (the tool and administration of the tool) to remain unchanged throughout the data collection.
• The researcher must ensure that a large number of the selected sample will reply.

Survey methodology as a scientific field seeks to identify principles about the sample design, data collection instruments, statistical adjustment of data, and data processing, and final data analysis that can create systematic and random survey errors. Survey errors are sometimes analyzed in connection with survey cost. Cost constraints are sometimes framed as improving quality within cost constraints, or alternatively, reducing costs for a fixed level of quality. Survey methodology is both a scientific field and a profession, meaning that some professionals in the field focus on survey errors empirically and others design surveys to reduce them. For survey designers, the task involves making a large set of decisions about thousands of individual features of a survey in order to improve it