User Sampling Methods

1 Sampling Methods

When surveying, for any purpose, it is important to recognise that the results are only as representative as the survey subjects (the sample), and as such much academic research has been performed in to techniques for selection, broadly placing them in one of two categories - probability sampling and non-probability sampling.

In short, with probability sampling the participants are selected by chance. There are dozens of methods of selecting members, using a variety of mathematical techniques, but the key is that each subject has a random, calculable chance of being selected. There is no human intervention involved in the selection.

Method Characteristics

Simple (random) Sampling The sample is selected entirely at random

Stratified The population is first divided in to exclusive subgroups based on some predetermined criteria (e.g. location), then samples are selected at random

Proportionate Stratified As above, but a smaller group that would otherwise not provide statistically valid results may be oversampled then the results weighted to correct for this. For example, if a particular group is too small to provide a statistically significant sample, more members of that group would be sampled

Clustering The starting point for the sample is randomised, then assumes that the sample at that point is representative of the region. For example, selecting a street corner, interviewing the first 10 people, and assuming that they are representative of the area

Table 2 - Probability Sampling Methods

Non-probability samples, however, contain an element of human bias in the sample selection. Again, there are various methods for the selection, the most common of which are:

Method Characteristics

Quota Respondents are preselected to ensure that the sample is representative

Purposive Subjects are selected as they have some specific characteristic, for example, hold a certain position or job type

Convenience The sample is selected by availability

Snowballing Contacts provide information about other potential respondents

Self-selection Respondents volunteer themselves for selection

Judgement An "expert" uses his or her judgement to nominate people for sample

Table 3 - Non-probability Sampling Methods

Most samples use a combination of sample selection methods. For example, quota sampling is often used to ensure that a random sample is actually representative of the population.

2 Survey types

There are three basic surveying techniques that are suitable for consideration in the project:

* Interview

* Telephone

* Self Administered Survey

2.1 Interview (face to face)

This category includes in-depth interviews, focus groups and projective methods (such as word association tests) which are not suitable for use in this project. There are several distinct advantages to this type of survey, such as:

* Response rates tend to be higher than other methods

* Ability to reassure the subject about their responses

* Questions can be clarified

* The interviewer can supplement the data with observation (for example, other feedback)

* Longer and more complex interviews can be performed (although this is not recommended here)

* Visual aids, such as queue cards, can be used

* Response bias can be largely removed from the results if respondents see follow up interviews as an essential part of the implementation process

However, there are disadvantages:

* It is often costly to meet each subject, even in a group session. Travel costs and time must be taken in to account.

* There is much more significant chance of interviewer bias being introduced to the sample, for example, by tone of voice or interpretation of response

* Although response rates are higher, as people are reluctant to appear "rude" the response may include some level of bias here

2.2 Telephone

Telephone interviewing, especially within a single organisation such as a company, represents a highly convenient, accurate sampling technique

* Telephone interviews tend to have less interviewer bias than face to face interviews

* People are more inclined to say what they think as it is more anonymous

* Cheaper than interviewing, but more expensive than self administered tests. The cost per response required must be evaluated.

* Shorter - max 15 minutes

* Simpler questions - no cue cards

* Some response bias, although can be mitigated

* Generally higher response rate than self administered tests

2.3 Self administered

A survey that the respondent completes themselves, either as a paper form, or through the internet, is a self administered survey.

* Anonymity is a major selling point of a self administered survey - respondents do not feel the pressure to be polite

* The respondent can complete the survey at their convenience

* They are cheap - especially in clients where there is an established tool

* Uptake is low - people do not tend to respond as there is no human contact requesting it

* People get bored of taking self assessment surveys. Over surveying further lowers the response rate for subsequent contacts

* The least likely to produce any accurate results, because:

o there is a high degree of response bias - typically respondents are those with a problem or complaint