Synchronisation session
Automatic deletion
The synchronisation framework supports automatic deletion of users or documents that have not been synchronised during a synchronisation session. This means clients of the API do not have to track what data has been added. A client just provides the most recent snapshot (users and docs) of what data should be associated with the connector, and the framework automatically deletes any previously synchronised users or docs not provided in that latest snapshot.
Automatic deletion requires a synchronisation session to be started, synchronisation to be performed, and the session to be ended. When the session ends, the framework checks for any users or docs previously synchronised with their DeletionMode set to Session. If those users or docs were not synchronised during the current session, they are deleted.
State and explicit deletion
Some data sources have more advanced mechanisms for tracking content. For example, the SharePoint native connector uses the Microsoft Graph API and its delta feature. Whenever synchronisation runs, site drive (document library) files and folders are only synchronised from the most recent delta.
The delta token is stored in state within the site drive (document library) synchronised document, persisted by the synchronisation framework. The next time that site drive is synchronised, the delta token stored in state is retrieved and a delta sync is performed.
Important: Any site drive files or folders synchronised during a delta sync must be saved in the framework with
DeletionModeset toExplicit. This means they are not automatically deleted if they are not synchronised during a following session. Instead, deletion is handled by processing the delta and explicitly deleting items via the API when the delta indicates to do so.
The synchronise state of an external system can become unknown. For example, the SharePoint Graph API documents that the delta token can become invalid at any time. When this happens, you can no longer rely on the delta sync to delete previously synchronised content, so a full resync using session deletion mode is required. At the end of that resync, a new valid delta token is available for further resyncs.
To support this, the framework can reset the synchronisation state of a document and all of its children, regardless of deletion mode. Resetting the sync status of a document and its children sets:
- Date Last Updated = null — so date comparison checks are ignored and all items are fully updated.
- Deletion Mode = Session — so that once a follow-up full resync session finishes, any documents no longer synchronised are automatically deleted.
- State = null — so that delta tokens or other state are not used, forcing a default full resync.
- Sync Session Deletion Candidate = true — so the document and its children are eligible for automatic deletion if the document is not synchronised during the session.
A follow-up full resync can then be performed. When the session completes, the connector ends up with valid sync state (a delta token) for subsequent sessions.
Automatic deletion of child documents
Automatic deletion during a synchronisation session also uses the Materialised Path and Is Parent properties to automatically delete candidate child documents of a parent (Is Parent = true) that is going to be deleted. This applies when the parent has its deletion mode set to Session and is a sync session deletion candidate (it was not synchronised during the current session).
This only applies to documents with their Is Parent and Materialised Path properties set. Child documents do not have a direct reference to parent documents; there is no parent ID property. Instead, the materialised path calculates which documents are children of the parent. This is more performant, and means parent documents do not need to be synchronised before child documents.
The client is responsible for populating the materialised paths of documents in a unique and meaningful way. The materialised path should uniquely represent the hierarchical path to the document. At its simplest, it can be a list of concatenated globally unique IDs:
1001/2002/3003/4004/5005/6006/
Here, 1001 is the ID of the root document, 6006 is the ID of the current document, and the other IDs represent the hierarchical parents in between.
For example, if the framework detects that the document with ID 3003 needs to be deleted (along with its children), the materialised path for that document is 1001/2002/3003/. All documents with a materialised path that starts with 1001/2002/3003/ are automatically deleted. For example, 1001/2002/3003/4004/5005/6006/ is deleted.