In this article
Survey Data schemas (templates) hold the meta information about target and source: Which variables are to be read from the source and updated in the target, the variable types, answer lists etc. When running a rule, with data flowing from the source to the target, some validation must be done to verify that the schemas match.
There may be different templates for source and target, and there may be differences between the templates and the survey databases when the source or target is a Forsta Plus survey database. When a rule is run, three basic validation checks are performed:
- If the source is a Forsta Plus database and a template is used, the source database is checked against the template.
- The source template is compared with the destination template.
- If the target is a Forsta Plus database, the target database is checked against the template.
These three checks are run for both the variables (questions) and the answers in the data file. As one of the main benefits of using the templates is to reduce the number of variables processed in the data processing rule to optimize performance, it is not an issue if there are more variables in the survey database than are described in the template. This process reduces the volume of data to be transferred and thereby speeds up the export/import task. However the following situations may cause issues and will be default make a data processing rule abort:
The template used in the source lists more variables than are available in the source database.
By default, if this validation returns an error, the data processing rule execution will abort. However you can override this rule and log a warning instead of aborting the task, or even switch off this validation check completely. Variables that are missing in the source database will then be available in the source template, but will be empty (no values).
The answer lists in the Survey Data template used in the source do not match the answer list in the source survey database.
By default, if this validation returns an error, the data processing rule execution will abort. However, you can override this rule and log a warning instead of aborting the task, or even switch off this validation check completely. The values stored for these variables will then be read even though they do not match the answer list in the template.
The variable type in the template used in the source does not match the variable type defined in the source database.
If this validation returns an error, the data processing rule execution will abort. This validation can not be overridden.
The template used in the target lists more variables than are in the target survey database.
By default, if this validation returns an error, the data processing rule execution will abort. However, you can override this rule and log a warning instead of aborting the task, or even switch off this validation check completely. No attempt will be made to write anything into the target database for variables that do not exist in the database.
The answer lists in the template used in the target do not match the answer list in the target survey database.
By default, if this validation returns an error, the data processing rule execution will abort. However, you can override this rule and log a warning instead of aborting the task, or even switch off this validation check completely. The rule executor will then write the values to the database even though there is a mismatch between the answer list in the template and the database.
The variable type in the template used in the target does not match the variable type defined in the target database.
If this validation returns an error, the data processing rule execution will abort. This validation can not be overridden.
For a rule with a survey database in both the source and the target, it is possible to specify different Survey Data templates for the source and the target. For other combinations, the template will either be inherited from source to target (as in Triple-S import to survey database) or inherited from target to source (as in Triple-S export from survey database). With both a template for source and a template for target, the following situations may cause issues:
The template used in the target lists more variables than the template used in the source.
By default, if this validation returns an error, the data processing rule execution will abort. However, you can override this rule and log a warning instead of aborting the task, or even switch off this validation check completely. Variables that are missing in the source template will then be empty (no values).
The answer lists in the template used in the source do not match the answer list in the target template.
By default, if this validation returns an error, the data processing rule execution will abort. However, you can override this rule and log a warning instead of aborting the task, or even switch off this validation check completely. The rule executor will then write the values to the database even though there is a mismatch between the answer list in the two templates.
The variable type in the template used in the source does not match the variable type defined in the target template.
If this validation returns an error, the data processing rule execution will abort. This validation can not be overridden.
As mentioned, for some of the validations, you have three possible choices:
a) An error is raised and the process stops (strict handling) - this is the default setting and will apply if you do not make any other selections.
b) You can switch off validation and allow the process to continue - check the Allow variables ... and/or Allow answers ... boxes (see the figure below).
c) You can choose that the process continues but also display a warning - check the Allow variables ... and/or Allow answers ... boxes and also the appropriate Show warning boxes (see the figure below).
You make these selections in the Validation setting properties on the Source, the Target and the General tabs for the Current Rule in the event the source type selected is Forsta Plus Survey Data - .
Figure 1 - The Validation Setting options
The validation process will be performed before the SSIS package (rule) is executed. If a survey does not conform to the schema and the default settings are in force, data for the survey will not be transferred and a warning will be recorded in the log.
Note: All the variables in the source and target must be of the same type. This requirement may not be overridden.