Handling Data Changes in Your Sync

Syncing the Full Data Set

To synchronize your database with Clever's data, follow these steps:

Identify Matching Records:
- Existing Records: For records in your database that correspond to Clever IDs, update them with the latest information from Clever.
- Non-Existing Records: For records without a matching Clever ID, perform a match to identify potential matches. If a match is found, merge the records; if no match is found, create a new record.
Deactivate Obsolete Records:
- For records in your database with a Clever ID not present in Clever's data (or if accessing the data returns a 404 error), deactivate these records to maintain data integrity.

Constructing "Diff" Files

Instead of completely replacing your database with Clever's dataset, you may choose to create a file that outlines the necessary changes. This approach allows for more controlled updates. Methods to achieve this include:

Events-Based Sync: Utilize Clever's Events API to receive real-time data changes. This method is efficient and recommended for applications requiring scalable synchronization.
Data Comparison: Compare your existing dataset with Clever's data to generate a file that captures the differences. This method involves analyzing both datasets and identifying discrepancies to create an accurate diff file.

Protecting Against Unintentional Changes

To prevent accidental data modifications or deletions:

Monitor Significant Changes: If a sync operation results in a substantial number of data changes or deletions, pause the sync and review the changes with the district administrator to confirm their intent.
Utilize Clever's Hold System: Clever's hold system prevents large-scale changes from propagating until approved by a district administrator. However, this system does not cover changes due to sharing modifications.
Set Deletion Thresholds: Be aware of Clever's thresholds for data changes. During the school year, if more than 10% of any record type is deleted, the sync is paused until reviewed. This threshold is relaxed during school year rollover periods when updates are more frequent.

Semester and School Year Rollover

Educational institutions may handle term transitions differently:

Term Reuse vs. New Term Creation:
- Term Reuse: Some Student Information Systems (SIS) reuse sections from term to term, updating enrollments, teachers, and terms as needed.
- New Term Creation: Other districts may create entirely new sections with new Clever IDs at the start of a new term.
Application Handling:
- Ensure your application can manage both scenarios by archiving old terms and reactivating them if necessary.
- Review your rollover processes to determine if adjustments are needed to accommodate Clever's integration.