- 16 Jul 2024
- 5 minute read
- Print
- DarkLight
- PDF
Source Format Settings
- Updated 16 Jul 2024
- 5 minute read
- Print
- DarkLight
- PDF
Setting | Description |
---|---|
Status | Select "Active" to make the source format available for use in Upload Dataset. |
Name | Provide a name for the source format. |
Format | Specify the file format (for example, "Excel," "CSV," or "Fixed Width" ). This setting is used for display purposes for non-standard source formats. The formatting of the file is defined in the XML setting on the Format Definition tab. For Standard Source formats, this setting is used to match to the Slate source format. |
Select the appropriate type of import: One-Time / Differential This import will modify current record data that is included and mapped in the uploaded file. The one-time differential setting should be used for one-time imports or when the files being imported contain only new or updated records. Cumulative / Replaceable Use this type for importing cumulative files (where every record is included in a subsequent file along with any new records). This will delete the previous dataset every time a new dataset is imported. Do not select this option if you have any exports or filters that reference the data stored within the dataset (versus referencing data imported into fields). Otherwise, this is an appropriate option for most cumulative imports. | |
Remap As Of Date | Specify how far back the source format should look at data values to make them available in Remap. For new source formats, you can put the current date. |
Remap Active | When this is updated to "Active," Slate will automatically process the uploaded file using the Remap settings, which you will define in a later step. Leave this as Inactive until you are satisfied with your remap settings. Prior to activating, the Remap As of Date must be populated or Slate cannot reference the effective date. |
Scope | Select Person/Dataset Records unless importing Slate users. |
Dataset | This specifies if the dataset will be imported into Persons/Application Records or if it will go into any other datasets (such as Organizations or Volunteers). |
Unsafe | If a source format is "Unsafe," it will be allowed to update person-scoped data on student records with an active application. A source format should only be marked as "Unsafe" if the source is trusted to contain accurate and up-to-date information. You do not want an import to update information that the student recently provided you on an application with information that may be incorrect or outdated.
|
Hide |
|
Disable Update Queue | By default, records that are imported or updated from this source format will enter an update queue. This queue is used to run deferred rules, adding records to queues for data exports configured to only include records updated since the query last ran. |
Update Only | Tells Slate how to handle records in the dataset that do not match records within Slate. "Allow record creation" means that Slate will create a new record for any records in the dataset that do not match records in Slate. "Update only" means that the source format will not create new records, and the dataset will only update matching records. |
Dedupe Records | By default, Slate will create a new record for every row if "Allow record creation" is selected above, and an existing match cannot be found. With "Dedupe records" selected, Slate will evaluate the source file and dedupe records with the source based on an exact match of First Name, Last Name, Email, and Birthdate. |
Notification and Notification Email | To receive email notifications if an imported file associated with the source format generates an error, select "Failures only" and enter the email address (or addresses) where notifications should be sent. Select "Successes and Failures" to be notified of successfully imported files for the source format. |
Read Permission and Upload/Build Permission | Permission to the source format can be limited to specific permissions. |
XML | Define the file type, such as if it is delimited, contains a header row, or fixed width. |
SQL | For most data imports, the SQL text area should be left blank. |
Source Index | Unless you are working with an existing source format that requires the legacy source.index table, select "Use modern cloud storage" (the default selection). |
Paste the private key from the PGP software. | |
Each PGP private key is typically encrypted with a password. | |
Import Path/Mask | This is the field that tells Slate that you want to pull in files automatically. In this field, include the file name pattern that Slate should use to determine if the file should be imported using this source format. We recommend appending the file name with a date to avoid using the exact same file name each time. Slate does keep the files for a period of time in a Processed directory within the /incoming folder; if the file names are exactly the same, each subsequent file load will delete the existing file from the Processed directory. For example, if the "Import from FTP Mask" field is SIS_to_Slate_*.txt, Slate would automatically import any file in the /incoming folder with that file name pattern, including SIS_to_Slate_09032013.txt. |
Import Remote Server | Leave this field blank unless the data for this source format will be delivered via an API call from Slate or an external SFTP server.
|
HTTP Headers | Add a list of header in the form, for example: |
Import Order | If specifying an Import Path/Mask, you can optionally specify an Import Order if the order that Slate processes the file matters. For example, if you have an application file and a supplement file that will be delivered to the SFTP directory together, and you would want the application file to be processed before the supplement, you enter "1" for the application's source format and "2" for the supplement's source format.
Then, when Slate later scans your SFTP for new files in the incoming directory, the Import Order will be accounted for. Slate will evaluate the source formats with no Import Order specified first, followed by the source formats with the smallest Import Orders (which translates to being the highest priority). |
Import Frequency | By default, Slate sweeps the SFTP directories for new files that match the specified Import Path/Mask every 15 minutes. If you are only dropping off files once per day, you can choose to increase the frequency by specifying the number of minutes between file sweeps (for example, 1440 minutes). |