Going back to our example of the Employee Satisfaction Manager, chances are that (some of) your employees do not want to provide you with this information and/or will feel offended. Generally, respondents can do two things if they do not want to answer the question:

1. Skip the sensitive question. However, mostly, closed questions are mandatory in online surveys. As they can’t skip the question there is a distinct possibility that respondents will leave the survey altogether. Furthermore it is most likely that especially members of a minority group will decide to skip the sensitive questions and/or leave the survey, which will have serious implications on the validity of your data.

2. Give a little twist to the truth. Social desirability is a tendency to answer questions in a manner respondents feel is socially expected. This usually leads to over-reporting ‘good behaviour’ or under-reporting ‘bad behaviour’. For instance, when confronted with the question “How many glasses of alcohol do you consume per week?” respondents will tend to downplay this number. Or, social research on feelings of insecurity has already shown many times that men tend to downplay their feelings of insecurity as they are – rather stereotypically – expected to have less fear than women.

In short, asking about sensitive issues can have a detrimental effect on your survey data. So, what can you do to avoid the pitfalls of surveying sensitive topics?

1. Tinker with the wording of the questions. Two techniques are rather common:

   1. Normalisation of the sensitive issue

      1. You have to phrase the question as such that you make it clear that you think it is completely normal – which it is! – that there are differences between people/employees on religious beliefs, sexual orientation, etc. An example could be: “At Company X we are very proud of having a staff with such a diverse set of religious beliefs. What is your religious affiliation?”

   2. Asking indirect questions

      1. This is a technique that rephrases questions as such that respondents have to answer questions from the perspective of another person or group. From the perspective of a person that is highly similar to the respondent. The idea is that respondents will reveal their true feelings on sensitive issues if they respond from the perspective from a (similar) other person.

2. Guarantee anonymity

   1. You must convince/guarantee your respondents – in this case your employees – that the survey is completely anonymous. In other words that you will not use any identifying information when analysing the data. As a result, respondents might be more inclined to answer truthfully.

3. Use ‘social desirability scales’

   1. These scales are widely used by researchers and practitioners to screen individuals who bias self-reports in a self-favouring manner. As you identify the respondents/employees that are more likely to answer socially desirable, you can account for them in your analysis. However, these scales are based on 40 items (more or less depending on which scale you use), which makes them more difficult to incorporate in your survey as they significantly increase the workload for the respondents/employees.

4. Make sensitive questions facultative

   1. In other words, let your respondents choose if they want to answer the sensitive questions by not making the question required or by adding an escape response option like “Rather not say”. However, even though it might help not to offend your respondents and keep them on board for the entire survey,  it is our least favourable option. Yes, it is an easy option but at the end of the survey you still will not have much data on the sensitive issues that are crucial for your research.

So, what do you do to deal with these sensitive issues?

This can be useful for several different scenarios:

• You may want panelists to receive an e-mail invitation at a certain time of day in their time zone to improve response rates. Some large companies process transactions from the day in batch at night. During the batch process, they call our API to send out e-mail invitations to customers with which they had contact. This works great, but it also means that the mails go out during the night. Unfortunately 3am is not a great time to send an e-mail. The recipient only sees it in their mailbox in the morning, under a pile of other mails and newsletters that have arrived during the night. Setting the mail to arrive at 10am in the recipient’s time zone will increase the chance that they see it and open it.
• You may have a transaction, like an online- or phone sale where the transaction is not really complete until the purchased items arrive. In that case you want to delay the e-mail invitation from going out so that the customer can evaluate the whole process including delivery.
• You may be a travel agency. Customers book with you well in advance. In that case you only want the e-mail invitation to go out once the person returns from their journey.

I’m sure there are many more scenarios that our user base will think of. If you have a good one, please share here below…

So far, so good. Yet, the following question will almost instantly arise: “What is the population that I would like to survey?”. Or, who do you need to survey to gain valuable insights on the success of your new furniture line? In this case the answer is rather straightforward. Assuming that you are launching the new line on the European market, that minors do not buy furniture and that your furniture is reasonably priced, your population consists of all adults in the EU.

For obvious reasons it is impossible to survey those (roughly) 400 million adults in the EU. A sample of adults living in the EU offers the solution for this issue. A sample is a selection of respondents chosen in such a way that they represent the total population as good as possible.

However, instantly a new question comes to the fore: “How many people should my sample consist of?”. Using a correct sample size is crucial for your research. After all, a sample that is too big will lead to the waste of precious resources such as time and money, while a sample that is too small will not allow you to gain reliable insights.

So, how large should your sample be? Should you survey 1%, 5%, 10%, … of the adult citizens in the EU? Well, this depends largely on how accurate you want your survey data to be. In other words, how closely you want your results to match those of the entire population. There are two measures that affect the accurateness of the data.

• First of all there is the margin of error (or confidence intervals). In short, this is the positive and negative deviation you allow on your survey results for the sample. Or, in other words, the deviation between the opinions of your respondents and the opinion of the entire population. An example will shed some light on this statistical explanation. Suppose you set your margin of error on 5%. If – let’s hope so! – 90% of your survey respondents like the ‘Fall 2013’ line, a 5% margin of error means that you can be ‘sure’ that between 85% (90%-5) and 95% (90%+5) of the entire population actually likes the ‘Fall 2013’ line.
• Second there is the confidence level. This tells you how often the percentage of the population that likes the ‘Fall 2013’ line actually lies within the boundaries of the margin of error. Or, following on our previous example, it tells you how sure you can be that between 85% and 95% of the population likes the ‘Fall 2013’ campaign. Suppose you chose the 95% confidence level – which is pretty much the standard in quantitative research¹ – then in 95% of the time between 85% and 95% of the population likes the ‘Fall 2013’ line².

Once you have decided how accurate you want your sample data to be, you can start calculating how many respondents (i.e. people who have completely filled in the survey or completes as we call them at CheckMarket) you actually need.

Below you find an indicative table on how to calculate your number of completes. Remember that your population consist of approximately 400 million adults in the EU. As a consequence, the appropriate number of completes will be found on the last row of the table below. Depending on the confidence level and the margin of error, the number of completes will vary. As we chose a margin of error of 5% and a confidence level of 95% for our ‘Fall 2013’ campaign, you need approximately 400 completes (it is advisable to round to the nearest hundred) for your sample.

Alternatively, on the CheckMarket website, you find an easy calculator to calculate the number of completes…

Before you start sending out your survey to 400 respondents, remember there is such a thing called response rate. Response rate is the ratio of respondents that fill in the questionnaire they received compared to the total number of surveys you send out. For instance, if you send out your survey to 400 people and you receive 200 filled in surveys, your response rate is 50%.

For an online survey, conventionally, a response rate of 20% is considered as a good response rate, while a 30% response rate is considered to be really really good. As we calculated that we need 400 completes, this means that you will definitely have to send the survey to more than 400 people in order to reach those 400 completes. Obviously, you cannot predict beforehand what response rate you will achieve. However, assuming that your survey will achieve a response rate of 20%, we divide the objective of 400 completes by a response rate of 20%. As a consequence, you will have to send your survey to approximately 2.000 adults in the EU.

¹ In some quantitative research, stricter confidence levels are used (e.g. the 99% confidence level)
² To put it more precisely: 95% of the samples you pull from the population.

Companies like Google, Dropbox and Paypal have all recently started allowing users to add an extra layer of protection to their accounts by having them authenticate in two steps. In addition to a username and password, users enter a code sent via SMS or a mobile app. This is based on a white paper of the Internet Engineering Task Force (IETF) TOTP: Time-Based One-Time Password.

Social hacking, phishing and brute force attacks of stolen passwords have shown that passwords alone are no longer secure enough. Remember the recent leak of 400,000 passwords by LinkedIn. Since many people use their e-mail as a username and the same password on multiple sites, once hackers have the password from one site they can use it to break into other sites. If the hackers get into an e-mail account, they can reset passwords on any site. Just ask Mat of Wired Magazine.
 
As an enterprise survey tool, with many banks, telecoms, governments and FMCGs as customers, we need to insure the highest level of security possible. Today on Safer Internet Day 2013, we are launching 2-step verification for our users. This makes us the first survey tool to offer this level of security. We are also the first European company to offer it.

We encourage all users to activate this free security feature immediately.

Related articles:

• How do I sign in with 2-step verification?

Zo krijg je direct een zicht op de tevredenheid en identificeer je verbeterpunten. Waar moet je op letten?

1. Vraag snel om feedback
Wacht niet te lang om te polsen naar feedback. Het ideale moment is de dag na je event. Onder het motto ‘Wat vond je van gisteren?’ stuur je een korte uitnodiging via e-mail. Je kunt het ijzer ook smeden als het heet is en op het event zelf om feedback vragen. Dat kan met een klassieke invulkaart of via rondwandelende hostessen met een iPad of zelfs door het projecteren van QR-codes die linken naar de tevredenheidsenquête.

2. Een (kleine) incentive doet wonderen
Om zoveel mogelijk respons te verzamelen kan je feedbackgevers een korting bieden voor een volgend event, toegang tot digitale versies van de gegeven presentaties of ze kans laten maken om een leuke prijs in de wacht te slepen.

3. Spread the word
Kondig je evaluatie aan op het event zelf. Je kunt vermelden dat er de volgende dag een evaluatieverzoek wordt verstuurd en je gasten stimuleren om deel te nemen. Of je vraagt de bezoekers om op het event zelf hun mening te geven.

4. Houd het overzichtelijk en kort
Vraag je gasten om scores te geven voor aspecten als locatie, prijs, tijdstip, sprekers, … en een algemene score. Alleen voor de onderdelen die laag scoren voorzie je dat automatisch een vervolgvraag wordt gesteld waarin men de reden voor de zwakke score kan toelichten.

5. Vraag geen info die je al kent
Je kent je gasten. Je weet hoe ze heten, wat hun achtergrond is, welke taal ze spreken, enz. Ga dus bij een evaluatie niet polsen naar hun taal, naam, sector, enz. Dit komt onprofessioneel en storend over. Het is bovendien totaal overbodig, want bij het analyseren van je resultaten kan je deze info rechtstreeks uit je contactinformatie halen.

6. Geef ruimte voor feedback
Eindig steeds met een (niet-verplichte) open vraag waarin jouw gasten tips of opmerkingen kunnen ventileren. Deze learnings kan je constructief gebruiken voor je volgende event.

7. Meerdere enquêtes
Voorzie ook een aparte korte enquête voor de no shows. Waarom daagden ze niet op? Je enquêteresultaten bieden het antwoord. Had je standhouders op het event, meet dan ook hun tevredenheid. Wat vonden zij over het event, de aanwezigen en het rendement dat ze haalden?

Cigna recognizes the unique strengths of their customers and empowers them to become active participants in pursuing better health, so they can reach their full potential. Cigna uses surveys to get feedback from its customers so that Cigna may reach its full potential.

After a world-wide review of enterprise survey software providers and extensive security, privacy and other audits that lasted 8 months, Cigna chose CheckMarket to conduct surveys at home in the US and world-wide.

Security was the number one priorty for Cigna, followed closely by privacy. CheckMarket’s multi-continent, multi-data center solution answered the needs of Cigna’s world-wide reach and up-time requirements. Cigna will also be using CheckMarket’s API to automate the survey invitation process and automated-alerts to quickly address customer needs that may arise during surveys.

Cigna’s choice of CheckMarket confirms CheckMarket’s place at the top of the short list of enterprise survey providers. Enterprise customers have different needs than consumer-based survey tools like SurveyMonkey offer: Enhanced-security, SLA’s, white-label, multi-user, API, extensive services, etc.

We welcome Cigna and look forward to helping them gather insights from their customers.

For KBC Bank and Insurance, being customer–oriented is high on its list of values. To promote the importance of customer-oriented processes, KBC handed out its prestigious award for the most customer-oriented SME in Flanders. This thematic award is part of the SME Laureate, an initiative in which every two years UNIZO, (the Union for Self-Employed persons) rewards outstanding entrepreneurship.

With CheckMarket’s help the KBC invited Flanders’ SMEs to run as contenders for the prize. Then by email or post the customers of each participating SME were invited to take part in a customer satisfaction survey. CheckMarket carefully crafted the questions to gain insights into how well the companies handle questions, suggestions, complaints. An NPS question was also included to find out if customers would recommend the company to others.
The results of this customer satisfaction survey were used to select ten finalists who appeared before a jury of experts, including our own Alexander Dobronte – CheckMarket Managing Partner and a specialist in market research .The jury grilled the finalists about their customer service policies. Following a comprehensive assessment, the most customer-focused SME of the year was finally chosen: DOVY Kitchens.

www.kmolaureaat.be

Allowing CheckMarket to gather and analyze your survey data undeniably offers some advantages to you. We summarize the main ones:

• During your whole project – from tender to analysis – you will be able to contact the dedicated project manager who is familiar with the finer details of your project
• Your CheckMarket project manager knows exactly what you would like to learn from your survey. Consequently, from the draft of the survey onwards he is capable of advising you on the use of best practices in order to obtain optimal validity and reliability of your survey data. Similarly, your CheckMarket projects manager will be able to support you when deciding on your target audience, the composition of your survey panel and/or the calculation of an optimal survey size.
• The period of time between the gathering of your survey data and the reporting of the results will be significantly shortened. After all, your CheckMarket project manager knows both our survey tool and your survey project. Consequently, it will be possible to obtain research findings in a very short period of time.
• At CheckMarket all your survey results – both from the projects that you manage and from those that you outsource to CheckMarket – are stored at one and the same place.

In order to keep on guaranteeing our service – i.e. quality analyses and quick reporting of research findings – CheckMarket recently strengthened its research department with Didier Dierckx. Didier is a quantitative researcher that proved his mettle in the well-known research group M²P (Media, Movements & Politics) at the University of Antwerp, Department of Political and Social Sciences. As a pre-doctoral researcher and as a PhD-candidate he obtained a broad methodological and statistical knowledge. This knowledge ranges from survey-design, over bivariate and multivariate analyses (amongst others regression-analysis, factor analysis and cluster analysis) to the very advanced multi-level analysis.

Didier’s expertise is directly applicable to your different projects! His knowledge allows, for instance, to predict the probabilities of your clients recommending your products to their respective professional networks, and which factors will influence these probabilities the strongest. Regarding HR-surveys, it is not only possible to distinguish between various departments concerning employee satisfaction, but Key Driver Analysis is equally possible. In other words, which factors contribute the strongest to the employee satisfaction? In short, the research options are quasi endless.

In conclusion, the next time you would like to outsource a full-service survey project, do consider CheckMarket as your preferred partner!

There are several interesting uses for this action.

For instance, you could ask a respondent in a survey if they would like to be contacted and if so have a ticket created in Zendesk so that your customer support staff can call them.

However, to my initial surprise, it took me only a couple of blogs to reach the conclusion that Likert Scales – the same scales that are practically declared holy in my previous universe; the academic one – have come under some scrutiny. Even to the point that it is seriously questioned whether they should be retained or replaced by so-called ‘Slider Scales’. However, in this post I will come to the defence of Likert Scales – if they actually need me defending them. I will argue that the case of Likert Scale v. Slider Scale is based on the design and the customer-experience of online surveys, rather than on sound methodological arguments. Without a shadow of a doubt, slider scales have a more inviting design but as it turns out, there are not too many methodological reasons to favour them over ‘boring’ Likert Scales.

First things first. What is a Likert Scale? A Likert Scale, named after its developer the sociologist Likert, is a scale that is designed to measure underlying attitudes or opinions. This scale usually consists of several items, so-called Likert Items, on which respondents can express their opinions. Say for instance that we are trying to measure the underlying ‘general happiness’ of market researchers, then we would probably ask them several questions regarding their happiness in their private and professional life. These questions such as ‘How happy are you with your current employment?’ or ‘How happy are you with your current family life?’ are the so-called Likert Items. Based on these Likert items, we would then construct the Likert Scale on ‘general happiness of market researchers’.

Usually respondents express their opinions by choosing one of the response alternatives from the response scale. This response scale can take various formats. On the one hand there is a distinction between worded and numerical formats and on the other hand there is the distinction between the most common 5-, 7-, 10- or 11-point format (Dawes, 2008). Consequently, it seems quite logical that the 5-point format is usually worded (e.g. strongly disagree, disagree, neutral, agree, strongly agree) while a 10-point format is most often numerical. After all, the gradation of (dis)agreement on a 10-point rating scale probably becomes too fine to easily express in words[1].

10-point Likert Scale

The Likert Scale – in its various formats – is widely used, for instance in psychology, social sciences but also in commercial market research (Dawes, 2008). In commercial market research respondents may be asked about their attitudes, perceptions or evaluations of amongst others organisations, services or brands. However, these Likert Scales are often used for measuring single-item issues, rather than for measuring an underlying attitude. Recalling our example of the ‘general happiness of market researchers’, this means that we use the Likert Scale for measuring how happy they are with their salary, rather than measuring how happy they are in general. Even though this distinction is hardly ever made, we should be aware of it and the fact that when we are talking about ‘Likert Scales v. Slider Scales’, we are actually talking about the 5-, 7-, 11-point rating scale of Likert Items rather than Likert Scales. Nevertheless, for the sake of clarity I will consistently use Likert Scale, even if I am technically talking about the X-point rating scale.

However, going back to the original purpose for writing this post, it appears that the use of Likert Scales has come under some scrutiny in the commercial market research. While this is not necessarily a bad thing, the various criticisms levelled at the use of traditional Likert Scales ought to be valid. Let us look at the two most common criticisms that I have come across. First of all it is argued that the traditional 5-point rating scales are quite boring, repetitive and certainly overly long. Furthermore, a large battery of questions using 5-point rating scales might discourage respondents. Second it is equally argued that respondents are forced into expressing an opinion that is not their real opinion because there are too few response alternatives offered. In short, it is argued that the 5-point Likert Scale is too blunt a) to detect differences between items and b) to precisely measure specific opinions as the respondent’s true opinion can lie in between the answer categories. While the first criticism is probably true, especially in commercial market research in which respondents have to be convinced to participate in surveys, there are various reasons to question the validity of the second.

The proposed alternative to the traditional Likert Scale is the (admittedly more attractive) Slider Scale. These Slider Scales are basically Likert Scales with much more response categories and, instead of selecting one of the response alternatives, respondents use a slider to position themselves on a certain question. It is argued that this Flash-based alternative does not only enable more interactivity in online surveys but that they equally enable respondents to indicate more precisely their opinions. Respondents have more response options, ergo the results will be more finely grained. This argument is then strengthened by some experimental findings that show that respondents seem to answer questions slightly different when they are first confronted with a traditional Likert Scale and subsequently with a Slider Scale.

Slider Scale

Nevertheless, I tend to argue that this case of ‘Likert Scales v. Slider Scales’ is mainly about the design and interactivity of online surveys, i.e. to make it more a more pleasing experience for the respondents, than about sound methodological reasons. After all, there do not seem to be that many sound methodological arguments to favour umpteen-point rating scales, i.e. Slider Scales. Let us look at some evidence.

First of all, there has been done quite a lot of scientific research on the effect of the number of response alternatives on the psychometric properties of a scale, i.e. the reliability and validity[2][3]. The former refers to the fact that the scale can be interpreted across various situations and the latter refers to the fact that the scale actually measures what it set out to measure. The recurring conclusion is that when the number of response alternatives is increased, the reliability and validity of the underlying factor increase as well. For instance Lozano et al. (2008) have shown that both the reliability and validity of a Likert Scale decrease when the number of response options is reduced. Vice versa, if you increase the number of response options, the reliability and the validity increase. They conclude that the bare minimum of a Likert Scale should be four response categories. So, at first sight, this seems to favour the use of Slider Scales with more response alternatives. Nevertheless, the positive relationship between the number of response categories and the reliability and validity of the scale is not a linear one. Lozano et al. (2008) equally show that the increase in reliability and validity is very rapid at first but it tends to level off at about 7 response alternatives. After 11 response alternatives there is even hardly any gain in reliability and validity from increasing the number of response categories. In short, from a psychometric point of view it is shown that the gains are scarce when including more finely graded scales than a 11-point scale, i.e. more than 11 response categories. They hardly improve the scale reliability nor its validity (Dawes, 2008).

Second, plenty of scientific research has shown that respondents use the meaning of the labels attached to some response categories when mapping judgments to response scales (Rohrmann, 2003; Wegner, Faulbaum, & Maag, 1982; Wildt & Mazis, 1978). So, a larger number of response categories with few labelled points makes it harder for respondents to orientate themselves. Intuitively, it makes sense that labelled response categories are less ambiguous to respondents than when only the end labels are provided as “respondents need to figure out the meaning of the intermediate response categories to determine the option that comes closest to expressing their opinion” (Weijters, Cabooter & Schillewaert, 2010). If we project these findings on the Slider Scale, i.e. most often a scale with many response categories and few labelled points, then it is rather easy to see that the often lauded strong point of this scale type might not be so strong after all. After all, it is commonly argued that the Slider Scales offer respondents more response categories and as a result generate more precise data. Nevertheless, selecting the right response option that corresponds with their real opinion will be more challenging when respondents need to make up the right meaning for each response category (De leeuw, 1992; Krosnick, 1991) and this will only become more challenging if the number of response categories increases.

In short, while a Slider Scale is most definitely more attractive, interactive and is more consumer-friendly than the archetypical Likert Scale, the case of ‘Likert Scale v. Slider Scale’ is almost certainly a discussion about design rather than about methodology. Personally, I would judge the case to be inadmissible.