Blog

The importance of socio-demographics in online surveys

9 comments

In this new blog article we are looking into the use of socio-demographics in surveys. More specifically, we will try to answer three questions. First of all, should you ask about them and why (not)? Second, if you decide to ask about them, which ones should you ask? Third, if you ask about them, where do you put these questions in your survey?

However, first things first. What are socio-demographics? Socio-demographics are nothing more than characteristics of a population. Generally, characteristics such as age, gender, ethnicity, education level, income, type of client, years of experience, location, etc. are being considered as socio-demographics and are being asked in all kinds of surveys.

1) Should you ask about socio-demographics and why (not)?

Yes, you should

There are in fact various reasons to do so. First of all, in some cases it might be imperative to know who is filling in your survey. For instance, if your survey targets a specific audience, it allows you to determine whether you are actually reaching your target audience and whether or not you are gathering the information you are effectively seeking. Furthermore, if you aim for a representative sample of a population, knowing the distribution of the demographic characteristics of your respondents will help you in determining how close the sample replicates the population.

Second, if sample sizes are large enough, it enables you to differentiate between different sub-groups. This segmentation might offer you insights that you would have missed by only looking at the aggregate data. For instance, you might conclude that your employees are generally satisfied with their career opportunities in your organisation. Nevertheless, the aggregate data might hide the fact that the employees of the IT department are not at all satisfied with their career opportunities.

No, you shouldn’t

There are also various reasons not to ask about socio-demographics. First of all, you need a substantial sample to draw any statistically meaningful conclusions. If you have a sample at your disposal, differentiating between certain socio-demographic sub-groups might trim down your sample to a collection of less relevant sample(s). These samples could then be too small to draw any meaningful conclusions from. For instance, if you find that a particular product is very popular with 15 to 25 years olds but you have only 50 respondents in that particular age category, the sub-sample is simply too small to draw any conclusions.    

Second, adding socio-demographic questions lengthens your survey significantly. As many researchers have already shown, the dropout rate of a survey correlates positively with the length of the survey. The longer a survey, the higher the dropout rate. In short, adding (too many) socio-demographic questions might make it more difficult to gather a substantial sample size.

Especially if, thirdly, respondents might become concerned by having to answer a large number of (identifying) demographic questions. They might feel that these questions compromise the anonymity of the survey or experience these questions as an invasion of their privacy.

So, what do I do?

So, to summarize, it is imperative that you strike the right balance. We believe that you should ask your respondents about their socio-demographic profile. However, asking about socio-demographic characteristics is not without its pitfalls. That is why we advise you to determine the most important socio-demographics to your research in the start-up phase of the survey and only ask about them. Furthermore, you should use them wisely while analysing the survey results.

2) What are the right demographic questions?

Asking the right demographic questions will allow you to discover meaningful and actionable insights to assist you in making better business decisions.

Below we discuss two commonly asked socio-demographics:

1. Age:

It has been shown in various scientific disciplines that opinions on a vast number of topics differ between different age groups. When asking respondents about their age, ideally, you should ask them to fill in their age (expressed as a number of years) or their birthday. As such you have (or you can calculate) for each respondent a value (in number of years) which you can use as a continuous variable in the analysis. In other words, a variable which you can express differences in magnitude with. For instance, “respondent A is forty years old and is twice as old as respondent B”. Furthermore, it allows you to ‘recode’ the age data infinitely in many different age categories.

Alternatively, as age might be a sensitive topic for some people, you could use age categories. Depending on how specific you would like to be, you can use more/less narrow/wide categories. However, the use of non-overlapping equal categories is recommended. As such, categories are equally wide and each data point can only fall in one category.

At CheckMarket we generally use the following categorisation: ‘18-24’, ‘25-34’, ‘35-44’, etc.

2. Education:

More often than not there are clear differences in opinion between respondents with a different educational level. Moreover, educational level – generally asked as ‘the highest level of education completed’ – is also quite often used as a proxy for income.

In certain countries – such as Belgium – respondents are not too keen on ‘openly’ discussing their income, even if it is an anonymous survey. Therefore, to keep as many respondents as possible on board, researchers often opt for excluding questions about the income. It is then argued that the educational level gives an impression of the respondent’s income, or more generally its socio-economic status (SES). When polling respondents about their educational level, it is important to keep in mind that the school system is not the same in each country. In practice, it is even hardly the same in two countries! So, depending on which countr(y)(ies) you are surveying in and how specific you would like to be, you can use a more/less specific categorisation.

At CheckMarket we usually use the following general categorisation:

  • None
  • High/Secondary school
  • Bachelor
  • Master
  • PhD

Besides these two socio-demographics also gender, marital status, number of children, employment status (with categories ‘student’, ‘unemployed’, ‘retired’, etc.), etc. are often being asked.

3) Where to put socio-demographic questions in the survey?

As we have established that you should only ask the socio-demographics that are the most important to your research, does it matter where you put them in your survey? Yes, it does. Conventional wisdom was to put these questions at the end of the survey. An often cited reason for doing so is, amongst others, that respondents are hesitant to answer – or become annoyed with answering – demographic questions and, as a result, might drop out if you put these questions at the beginning of the survey.

Another argument says that it might make a difference in respondents’ answers if they know that their socio-demographics (e.g. race, gender, etc.) are being taken into account. This refers to the so-called ‘stereotype threat’. The ‘stereotype threat’ is defined as the “experience of anxiety or concern in a situation where a person has the potential to confirm a negative stereotype about their social group”[1]. In other words, if a respondent belongs to a negatively stereotyped social group they might perform less than at their maximum level, if they know their social group is taken into consideration. So, if you put the socio-demographic questions at the end of the survey, the reasoning goes, this will not affect the results.

However, recent research shows that placing socio-demographic questions at the beginning of the survey in fact increases the response rate to these questions. This should not come as a huge surprise as it seems rather intuitive that respondents are more likely to lose interest after a while in the survey, drop out and, as a consequence, do not answer the socio-demographic questions at the end of the survey. If these questions were placed at the beginning of the survey, respondents would already have answered them. As a result, if demographics play an important part in your survey (which they do in most research projects) or if you want to branch on particular demographics, these socio-demographic questions should be put up front.

That is why at CheckMarket we always recommend to put the socio-demographic questions at the beginning of the survey.

[1] Gilovich, Thomas; Keltner, Dacher; Nisbett, Richard E. (2006). Social psychology. W.W. Norton. pp. 467–468.

Related articles:

9 comments

Join the conversation
  • Vishal Shelat - October, 2016 reply

    I am a medical doctor and intending to conduct a population survery in my small country. Population is less than 6 million according to sample size calculator i need 1000 sample at 95% CI and error rate of 2.83%. This is fine.

    Survey is on population awareness on a body organ “pancreas”. I wish to include some sociodemographic questions too.

    I wish to recruit 10% patients with postgraduate and high study. 90% below postgraduate.
    90% patients coming to hospital
    10% healthy people
    50% men and 50% women etc etc ideas i have.

    The more indepth i go, the more confusing it becomes as i believe study will loose power for subgroups.

    Further i wish to keep this ad popation nationwide survery and wanted input on conducting on patients and family members coming to hospital?

    I need some assistance and possible help to draft a survey form.

    For sake of confidentiality etc i cant declare country, hospital, personal profile etc. Would appreciate if those details is not asked to me.

    Thanks

    Gert Van Dessel - November, 2016 reply

    Vishal,

    Your error rate for a population of 6 million and sample size 1000 will be 3,10 for a confidence level of 95%.
    If you want to look at subgroup level and combine different characteristics, your margin of error will of course increase.
    If you want to distribute your survey via our survey tool, you can always create an account (a free test account to start with) and we can always assist you in the creation of your questionnaire.

  • H erbert Archard - January, 2016 reply

    After numerous tries my demographic is being rejected, I believe by virtue of my age (84). I am given to perceive that surveys often seek responses from specific age groups based on the kind of products geared to that group from which they seek answers. Am I simply “out of the box”?

    Gert Van Dessel - January, 2016 reply

    Herbert,

    Often you see surveys are targetted to the age group of 18-65.
    Companies probably have the idea that people do not tend to change their consumer (or other) behaviour much after they passed a certain age.
    Whether this is corect or not is another question.

  • Mark Twain - September, 2015 reply

    Paid surveys are favoured more these days due to its authentic approach. Placing socio-demographic questions at the beginning provides momentum to replies by customers, thus making the surveys beneficial.

  • What’s Your Demographics? Confounding and Controling | Community of Social Service Providers - July, 2015 reply

    […] The importance of socio-demographics in online surveys […]

  • Tilahun - May, 2014 reply

    THANK YOU !!!

  • hanerry - December, 2013 reply

    Nice Read!!! Online surveys are important in every field. Socio-demographics in the surveys are gaining popularity. With the help of this your response rate is higher. If you are estimating anything, so with the help of this you will get an actual response very easily. Some surveys are paid and some are not paid. It’s all up to you that which one you select. It is advisable if you select a paid survey because this is authenticated. In every country, now this method is very much popular, so everybody adopts this.

  • manuel quiroz - September, 2013 reply

    lei todo el articulo de analisis y me parece muy acertado en las condiciones del encuestado a la hora de emitir opiniones. hay que recordar que las situaciones de inseguridad es global.

Leave a Reply

Your email address will not be published. Required fields are marked *