Quality Control

Sample Design
Quality assurance in the sample design process is done in two dimensions. Regular database updates of population statistics published by state statistical institutions ensures that sample plans designed for the research process are representative of target populations. However, an important part of good sampling is having up to date sampling sources such as postal of telephone directories ensuring the greatest possible probability that each member of a target population has a chance to be sampled. M-prospect holds an updated postal and telephone directory with both fixed and mobile telephone lines. In telephone surveys, numbers are chosen randomly to avoid sampling bias. In F2F surveys, where pollsters need to complete the last phases of sampling and select a respondent, a strict set of rules are enforced to avoid sampling bias.

Questionnaire design
A careful and well-designed Questionnaire ensures quality data. We employ several mechanisms in this stage ensuring that the questionnaire will adhere to the best standards of the industry. The questions are generated by a panel of experts in order to provide different viewpoints during the initial design phase. In order for the questionnaire to be comprehended by the target respondents, the vocabulary and formulation is being adjusted. The interpretation of questions is tested with a focus group in order to understand possible ambiguities in comprehension. Once a set of questions is selected, the question order is reviewed to avoid suggestive priming. After an initial draft, the interview duration is tested and adjusted to avoid interviewee fatigue. Once a working draft of a questionnaire is achieved, a pilot test on 50 respondents is administered as a final check before starting a production survey.

Piloting
Once all preparatory phases in a survey design are completed, a pilot survey is used to gather insight about possible problems during fieldwork. The pilot is administered on a sample of 50 respondents from various regions, educational attainment, gender and ethnicity. Both survey data and data about administration are collected in order to understand if there are problems of various types. This includes problems of comprehension, logic, pollster work or any other problems in administering an interview. After the pilot phase is completed, a review process addresses the uncovered issues before a production survey starts.

Pollster training
M-prospect holds a network of experienced and highly skilled pollsters. However, in our efforts to ensure quality in our products, pollster training is an integral part of every survey. Before a survey starts, we provide training to our pollsters about administering an interview going through the questionnaire. Secondly, in cases of face-to-face surveys, we explain the last stages of sampling if the pollsters are involved in respondent selection. Every pollster is given a research guide which has the training information as well as answers to frequently asked questions. The pollster training activities ensure that every pollster covers the field in a standardized manner.

Data Collection
During the data collection, a comprehensive supervision is put over the process. About 10% of the sample is supervised and controlled. In the case of face-to-face surveys, 10% of the surveyed respondents are either re-contacted by our supervisors by telephone or field visits are administered to ensure that data collection follows previously defined standards. In the case of telephone surveys, 10% of the interviews are monitored by listening the surveys which provides a review of the work to the pollster. If anomalies are detected (both f2f or telephone) all the interviews provided by a pollster are rechecked. In the case of f2f survey, field supervisors randomly review completed questionnaires and provide feedback or require an interview to be redone if errors occurred. In both F2F and telephone surveys several techniques are employed to achieve higher response rates.

Data Entry and Processing
Quality control of data entry in F2F surveys: All data is entered with double entry coding ensuring that errors are avoided in this process. Where mismatch occurs, a questionnaire is re-entered. Once a database of cases is completed, various tests are applied in order to check validity of cases. Skip (filter) logic tests examine if entered answers follow various filters in the questionnaire. Invalid answers are filtered out of the database to achieve answer validity.

Quality control of data entry in telephone surveys: By using a CATI system during a telephone survey, data entering is part of the data collection process. At the same time, our CATI system ensures the quality of the collected data by applying part of the validation during the collection process. Skip logic validation and basic logic validation prevents cases where mandatory answers are missing or extra data is being collected.
After data is entered in both F2F or telephone surveys, another set of tests examine if there are logical inconsistencies by comparing if there are conflicting answers on different questions. Another test identifies outliers in some of the cases. Such selected cases are re-contacted and resolved.

Statistical analysis
The last stages of quality control provide reports about response rates and reports about the context in which the survey was administered. Any issues and problems during all the stages of the research are documented and reported. If necessary, sample weights are provided in order to adjust the completed sample to better fit towards ideal sample.