Using Adaptive Scopes with Retention Policies for SharePoint Online Sites

Exploiting the Site Property Bag and Refinable Strings

In a previous article, I explain how adaptive scopes allow Microsoft 365 retention policies to cope with dynamically changing populations of user accounts and Microsoft 365 groups. Now I want to look at using adaptive scopes with SharePoint Online.

As explained previously, an adaptive scope is a filter to select target locations for a retention policy. In the case of SharePoint Online, you can filter by site name, URL, or any of the 100 refinable strings available in an organization. Essentially, these are the equivalents of the Azure AD custom attributes used by adaptive scopes to find user accounts and Microsoft 365 groups. Within SharePoint’s search schema, refinable strings (or “refiners”) are managed properties, meaning that you can search against the refinable strings, just like you can using the Azure AD custom attributes. Because organizations dictate the values written into refinable strings, they become a valuable way of marking sites. Adaptive scopes leverage this capability by using refinable strings to find sites for retention processing.

To assign refinable strings to sites, administrators write values into the site property bag, a mechanism that has existed in SharePoint for a long time. Although that makes the property bag a well-known entity to SharePoint aficionados, I suspect that many Microsoft 365 tenant administrators have only a passing acquaintance with the finer points of SharePoint site management. If you’re in this category, be sure to read this article about how to update custom values in the site property bag.

Adaptive Scopes and Refinable Strings

Obviously, just like when you use an Exchange Online custom attribute to mark mailboxes or Microsoft 365 groups for adaptive scopes, it’s important not to choose refinable strings that might already be in use for other purposes to manage sites. I also recommend that you write down what names and values you plan to write into site property bags together with the associated managed property. Table 1 shows an example.

Crawled and Managed Properties

SharePoint Online treats the names of custom properties written into a site property bag as crawled properties. To make those properties searchable, we must update the search schema to associate each of the crawled properties with a managed property. Because the custom properties in the property bag are string values, we use refinable strings.

After updating the SharePoint search schema to associate the crawled properties with managed properties, the next time Microsoft Search processes the site, it includes the custom property bag values for the site in its index. You can accelerate this by requesting Search to reindex the site. This works for all kinds of sites, except sites belonging to Teams shared channels.

Now that we know how to mark SharePoint sites with custom properties written into site property bags and how to associate those values with searchable managed properties (the refinable strings), we can create an adaptive scope to leverage this knowledge.

Creating an Adaptive Scope for SharePoint Online Sites

When you create an adaptive scope for SharePoint Online sites, you understand very quickly why refinable strings are so important to this story. Unlike user accounts and to a lesser degree, groups, adaptive scopes have limited ways to find sites outside refinable strings. You can certainly find sites using URLs and names (for example, find all site with names starting “Critical”, or find all OneDrive for Business accounts using a scope for sites with URLs starting https://tenant-my.sharepoint.com), but it’s much more likely that you’ll use refinable strings to mark sites for special retention processing.

The flow to create an adaptive scope for SharePoint Online sites is the same as before. Go to the Information Governance or Records Management sections of the Microsoft 365 compliance center and choose Adaptive scopes. Create a new scope and give it a suitable name and description. Select Site as the type of scope to create, and then construct the query to find the target sites (Figure 1).

As with the adaptive scopes for users and groups, you can combine multiple properties in a query (for example, find sites with refinable string 80 equal to one value and refinable string 99 starting with another value).

The advanced query builder allows you to create a custom KQL query (rather than OPATH as used for accounts and groups). However, given that you want to find sites, using refinable strings is probably a more precise method because administrators control the values written into these strings. It would be nice if Microsoft added some more site properties (like sensitivity label or external sharing capabilities) to allow more flexibility in scope conditions.

Wait for the Scope to Resolve

After saving the scope, leave it alone for a few days to allow the background process to evaluate the query and find the target sites. Eventually, you should be able to view scope details to see the set of target sites. Figure 2 shows a set of sites I marked as needing attention for GDPR compliance. Note the date added column. This data captures when the background job processed the site and detected that it matched an adaptive scope.

You probably know what sites you’ve updated with appropriate values in the property bag. If you don’t (or can’t remember), you can check with SharePoint search by inputting the scope query (shown in the query summary in Figure 2) to see what sites it finds. If the two match, all is well.

However, remember the caveat about waiting for a few days to allow site property bag updates to appear in search indexes. If you rush to compare, you’re likely to see different results in the scope details and SharePoint search. Overall, it’s best to leave adaptive scopes for a couple of days before attempting to match what the scope finds with what you think it should find.

Creating a Retention Policy for SharePoint with an Adaptive Scope

After you’re happy that the adaptive scope locates the correct set of SharePoint sites, you can go ahead and create a retention policy with the scope. The steps are the same as for policies using adaptive scopes to find user accounts and Microsoft 365 groups, with the obvious difference that when we select an adaptive scope for sites, the locations to which the policy can apply are limited to SharePoint sites (Figure 3).

If the scope matches OneDrive for Business accounts, the retention policy processes these locations.

As with all retention policies, it can take some time before the policy is active for all target locations. For this reason, if you plan to replace static retention policies with adaptive policies, it’s wise to leave the static policies in place for a month or so to ensure that retention processing for the target sites is continuous. No harm comes from having both static and adaptive retention policies target the same site. Microsoft 365 must do some extra work to process the two policies, but if the same retention period and action apply to both policies, the processing won’t affect the content in the sites.

Extra Care and Attention

The need to write values into site property bags, align crawled properties with managed refinable strings, and wait for everything to settle down and work means that creating adaptive scopes for SharePoint Online is more complicated and takes longer than for user accounts and groups. However, creating an adaptive scope is a one-time operation and after a scope works properly, you’re not going to go back and tweak it on an ongoing basis. The mantra here is to take your time and wait for background jobs to run.