Overview
A quota is a limit on the number of responses for your survey that fit a specified set of criteria. They enable survey administrators to gather a desired number of responses from different groups of respondents, and enable you receive the desired mix of responses (e.g. equal responses from each age group, gender, or region). Once a quota has been met, the survey will not accept additional responses from respondents that meet that criteria and will display your custom quota full message.
You can find the quotas menu for any survey by first navigating to the survey settings for that survey:
Then choose the menu on the right-hand side called “Automatically Close Survey By Date/Time or Manage Quotas”.
You will see any previously created quotas in the bottom section of the page (this will be blank if you have never set up a quota for this survey) or can create a new one with the Create Quota button.
Quotas can be conditional (based on criteria you specify) or unconditional (limit total responses).
Conditional Quotas
A conditional quota is a quota that limits the number of respondents that meet the criteria that you specify. For example, you can ask for the respondent’s region and then limit your survey to no more than 50 responses from each region. Each of those quotas is a conditional quota.
Conditional quotas can be set based on responses to one or more questions in the survey or custom field values. We’ll demonstrate how to set up each in the How to Create a Quota section below.
Unconditional Quotas
An unconditional quota is a limit on the total number of responses to your survey. For example, if you want to ensure you receive no more than 50 total responses, your unconditional quota limit would be set to 50.
Quota Elements
Groups and Subgroups
Groups
SurveyMethods allows you to set up Groups and Subgroups to evaluate different criteria simultaneously as well as determine the order in which criteria are evaluated.
When you create a new quota, it is automatically assigned to a new group. You can rename the group by clicking on the group title at the top left:
You can add conditions within a group by clicking the “+ Add Condition” link below the last condition in the group. Once you have more than one condition defined, you will see a dropdown between each condition with two options – And and Or:
If you set the operator to And, BOTH conditions will have to be met for the response to count against your quota, while if you set the operator to Or, if EITHER condition is true the response will be counted against the quota limit.
In the example above, the survey quota will be set for individuals in the 26 – 35 age group who have at least 1 child. Once the limit is entered and the quota saved, the selected action will be performed when the limit for these respondents is reached.
Subgroups
Subgroups are sets of conditions that are grouped together within a group, creating a hierarchy to determine order of condition testing. You can use subgroups of groups (and subgroups of subgroups) to create complex sets of conditions and also to ensure that conditions are tested in the desired order.
In the example below, you can think of the subgroup as creating parentheses around the two conditions within it. So the subgroup ensures that the two conditions are tested first before the “Or” operator after the second condition is tested.
What this example does is essentially create two groups:
- Respondents in the 26 – 35 age group who have at least 1 child (note the “And” operator before conditions one and two).
- Respondents in the 36 – 45 age group who have at least 2 children (notice the “And” operator within the subgroup).
The “Or” logic between condition two and the subgroup, means that if a respondent falls into either category, they will count against the quota limit (set to 100). Once that quota limit is reached, the survey will be closed, per the action specified.
You can rename subgroups just like you name groups, or add conditions within subgroups in the same way you add conditions to groups. You can also add subgroups within subgroups by clicking the “Add Subgroup” link on the top right of the subgroup.
Order of Operations
Subgroups are used to create bracketed expressions, so the lowest-level subgroup is tested first before working up the hierarchy. Conditions are evaluated according to logical operator precedence rules (Boolean Logic – AND, then OR).
In this example, conditions are evaluated in the following order:
- The condition in the subgroup “Type of cell phone” is evaluated.
- The condition in the subgroup “Cell phone” is evaluated. The system will then test whether this AND condition 1 are true.
- The condition in the subgroup “Age group” is evaluated. The system will then test whether this AND the result of the combined condition 1 and 2 are true.
- The condition in the main group “Male smartphone users in 25-34 age group” is evaluated. The system will then test whether this AND the result of the combined condition 3 and 1 and 2 are true.
If the result of all of these tests is true, the respondent will be counted towards the quota.
Quota Limit
The quota limit is the number of responses you would like to receive that meet your criteria, before taking an action (such as closing the survey). You can set up multiple quotas with different limits; for example if you want 50 responses from respondents that are under 18 but 100 from respondents that are 18 – 29, you could set up two quotas with different limits.
Quota Actions
When you create a quota, you need to specify an action that will be taken when the quota is met. There are five actions available:
| Action | Description |
|---|---|
| Close Survey | Close the survey after the quota has been met. |
| Redirect to Website | Take respondents to a website after the quota has been met. |
| Show a Message | Enter a customized message that will be displayed to respondents after the quota has been met. |
| Redirect to Another Survey | Take the respondent to another survey after the quota has been met. Note: To redirect to a survey, the survey you are redirecting to has to be live or have been launched at least once. |
| Panel Integration | Configure redirection URLs for use with panel integration. |
How to Create a Quota
- Go to the My Surveys page and click on the Settings icon (
) for the survey you want to add quotas to.
- Find and click on the link titled “Automatically Close Survey By Date/Time or Manage Quotas”. This will take you to the menu shown below. At the bottom of the page will be the quotas for the survey.
- Click the Create Quota button on the top right of the survey quota section.
- Then choose whether your quota is based on responses to questions in the survey, custom fields, or the total number of responses (unconditional).
Question-Based Quotas
Question-based quotas allow you to set logic conditions based on responses to questions on any page in your survey. Once you choose “Questions” as the condition type, you’ll see a dropdown labeled “Respondent answers this question:”.
In the dropdown you will see a list of the questions in your survey, organized by page.
Once you choose the question to test, you will see two more dropdowns, “Where the response is:” and “The following answer:”. The “Where the response is:” is asking how you want to test the respondent’s answer. Your options for this dropdown will vary depending on the type of the question selected in the first column. For example, the question selected below is Choice – One Answer Option Buttons question, so we have options such as is exactly equal to, is exactly not equal to, etc. You will have different choices for open-ended question types, such as a Numeric question type (for more information on question types, see our Question Types help article).
Below is the full list of logic operators for question-based conditions, with an explanation of each:
| Operator | Description |
|---|---|
| is exactly equal to | If this operator is used, the condition will be true if the respondent selects ONLY the answer choice or choices that are specified. If one answer is specified, the respondent must choose exactly that answer (e.g. for multi-answer questions they cannot choose that answer AND another answer) for the condition to return true. If more than one answer is specified, the respondent’s answer must match those answer choices exactly. |
| is equal to any of | The condition will be true if the respondent selects ANY of the answer choices specified. |
| is exactly not equal to | The condition will be true if NONE or ANY (but not ALL) of the answer choices specified are selected by the respondent. |
| is not equal to any of | The condition will be true if the respondent selects NONE of the answer choices specified. If you specify several answer choices and the respondent chooses one of them (by itself or along with other answer choices), the condition will return false and the action will not be completed. |
| did not provide a response | The condition will be true if the respondent does not provide any response to the question. |
| provided any response | The condition will be true if the respondent provides any valid response to the question. Note that if you use fields with validation (e.g. numeric-only fields or an email field), the system will already prevent them from submitting invalid responses. |
| contains | The condition will be true if the response contains the specified sequence of characters. |
| does not contain | The condition will be true if the response does not contain the specified sequence of characters. |
| begins with | The condition will be true if the response begins with the specified sequence of characters. |
| ends with | The condition will be true if the response ends with the specified sequence of characters. |
| is greater than | The condition will be true if the respondent provides a value that is greater than the value specified. |
| is greater than or equal to | The condition will be true if the respondent provides a value that is greater than or equal to the value specified. |
| is less than | The condition will be true if the respondent provides a value that is less than the value specified. |
| is less than or equal to | The condition will be true if the respondent provides a value that is less than or equal to the value specified. |
Once you select your logic operator, you will either enter the value (for open-ended operators, e.g. “is greater than”, “contains”) or select from the options (for defined operators, e.g. “is exactly equal to”, “is equal to any of”).
Open-ended:
Defined:
As discussed above, you can test more than one condition simultaneously by clicking the “+ Add Condition” link and repeating these steps. Then you must specify whether you want to test both conditions (And operator) or either condition (Or operator).
In this example, the conditions will evaluate to “true” and the respondent will be added to the quota count if either the respondent chooses “Under 18” or “19 – 25” on the age question or they specify that they have 0 children on question 2.
Quotas Based on Custom Field Values
Note: this section demonstrates how to set up quotas based on custom fields but does not go into detail on inserting and using custom fields. For more information, visit our Custom Fields help article.
The second option for creating conditions is basing them on Custom Field values. Custom fields are identifying information about respondents that you attach to their email address in an advanced email list (or attach to a web launch URL). Custom fields are commonly used for including a respondent’s name (for mail merge or customization of message displays) but they can also be used to enter information such as region, job title, department, etc. You can have up to 5 custom fields per email list and can use them for displaying information to respondents, analyzing response data, setting quotas, or in setting up page logic.
Creating quotas based on custom fields works very similarly to creating quotas based on questions, except that respondents don’t need to answer a question for the condition to be evaluated.
In the example below, let’s say that Custom Field 1 contains a respondent’s region. We want to limit our survey to 100 responses from a specific region (Europe in this example). Once we reach the limit, other respondents from Europe will see a message that we are no longer receiving responses.
Note: The custom field tokens will be replaced with their corresponding labels (in the above example, “Custom Field 1” will be replaced with “Region”) if you launch the survey using the Email Deployment Method with Advanced Email List Management. If your survey was not launched using the Advanced Email List Management, the generic custom field labels will be shown (e.g. Custom Field 1, Custom Field 2).
Unconditional Quotas
An unconditional quota is a limit on the absolute number of complete responses to your survey. So, if you want to ensure you receive no more than 50 total responses, your unconditional quota would be set to 50.
To set up an unconditional quota, simply choose Unconditional on the quota page, enter the desired quota, and specify the action you want taken when the quota is met.
The quota will be checked each time the survey is accessed. Once the quota is met the survey will be closed and anyone who attempts to access the survey (whether from the link in their email or the survey web URL) will see the survey closed message.
Managing Quotas
You can set up as many quotas as you like and they will be displayed on the Automatically Close Survey By Date/Time or Manage Quotas page.
The quotas will be evaluated in the order shown, meaning that if two quotas are met at the same time (when a single respondent submits their survey and causes two separate quota limits to be met), the action of the quota higher on the list will be performed.
This page allows you to view the quota limits as well as see an updated response count for each quota. Additionally you can deactivate any quota (or activate an inactive quota), edit the quotas, reset the response count, configure advanced settings, and delete any quota.
Which Responses Count Towards Quotas
Only completed responses count towards quotas. Responses received before creating a quota will be treated differently for conditional and unconditional quotas:
- Conditional quotas: Only completed responses received after the quota was created will count towards the quota.
- Unconditional Quotas: All completed responses (before or after the quota was created), will count towards the quota.
Note that if you reset the quota count, only completed responses received after the quota reset will count towards the quota, regardless of the quota type.
Quotas in Survey Analysis Module
If quotas are specified in a survey, they will be listed on each Individual Results report, which will also specify whether the response counts towards the quota or not.
Notes and Additional Information
- Related articles and articles referenced above:
- Multiple quotas can be active at a time for a survey, among which some may be met, while others may still be open.
- Quotas are checked in this order:
- Unconditional
- Custom Field-Based
- Question-Based
- Conditions in quotas are evaluated in this order:
- Custom Field-Based
- Question-Based
- If a survey has the automatically close survey by this date/time setting enabled, this setting gets precedence over quotas (e.g. if a date and time is specified, the survey will be closed on that day and time regardless of the quota response status).
- For quotas to be processed, the group needs to be marked as “Active”. By default, when a quota group is added, the group is expanded, and the “Active” checkbox is checked.
- You can add as many subgroups as you need. However, a maximum of 5 levels of nesting is allowed within the group.
- Each group must contain at least one condition or a subgroup.
- Each subgroup must have at least one condition.
- Deleting the group or subgroup will also delete any subgroups within it.
- You can move the conditions or subgroups by clicking and dragging the Move icon (present on the quota configuration screens).
- Subgroups can be moved only within the same level.
- Conditions can be moved only within their group or subgroup.
- An unconditional quota limit must be set greater than or equal to the number of responses received for a survey at the time the quota is set up.
- If responses are deleted from a survey, they will be removed from the quotas they qualify for. The response count against the quota will also be decremented.
- If a survey is taken in the Do Not Record mode (survey preview), the response will not count towards the quota, i.e. the response count will not be incremented.
- If a survey contains responses towards a quota, you can edit only the quota limit and the action. To make any other changes to your quota, you’ll need to first delete these responses.
- If a respondent submits a survey without answering any questions, the response will not count towards the quota, i.e. the response count will not be incremented.
- If a quota has been used in page logic, you can edit only the quota limit and the action. To edit the conditions, you’ll need to delete the groups in page logic that use this quota.
- The “Previous” button will be hidden on a survey page that has page logic based on quotas using those questions or custom fields.
- If the action is set to “Redirect to Another Survey” and you are the collaborator on the survey (i.e. the survey was shared with you but you are not the survey owner), you will not be able to edit the survey to which respondents are being redirected.
- A survey cannot be moved to another account if:
- It has quotas with the action “Redirect to Another Survey”.
- It has been used in the “Redirect to Another Survey” action of another survey.
- To move such a survey, you will need to select an action other than “Redirect to Another Survey”, or delete the quota.
- If question-based quotas are set up using negative logic operators like “is exactly not equal to” or “is not equal to any of”, and respondents skip the relevant questions, the condition where this question has been used will evaluate to true and the response will be added to the quota.
- Quotas are also checked for respondents that save their responses and come back later to take the survey. If the quota has already been met, the quota action is performed.