Restricted SharePoint Search is The Legacy of Previous Microsoft Collaboration Strategies

“I Wouldn’t Start from Here if I Were You.”

Microsoft’s announcement of Restricted SharePoint Search (for organizations with Copilot for Microsoft 365 licenses) is a stunning indictment of IT sprawl, specifically in Microsoft 365 tenants. The responsibility for the problem is shared between Microsoft and its customers. Microsoft introduced technology without suitable administrative controls and customers accepted the technology without pushback.

It’s a sad state of affairs that reminds me of the old Irish joke where a lost tourist is advised by a wise local that their starting point is not optimum. The current starting point for Copilot for Microsoft 365 is certainly not what it should be due to a collective failure in information governance over the last decade.

Office 365 Groups and a Dedication to Open Collaboration

In November 2014, Microsoft previewed Office 365 Groups as a new platform for collaboration based on email-driven “Outlook groups.” Office 365 Groups subsequently achieved general availability at the Ignite conference in Chicago the following May. Controls over group creation were limited because Microsoft pushed the line that end users should be able to create new groups when they wanted. It was all about seamless on-demand collaboration.

Few cared about the proliferation of groups because Microsoft made the resources needed to host the groups available without charge (except for the 60 MB or so consumed to spin up the SharePoint Online site for each group, which is charged against the tenant SharePoint storage quota). What did a few extra hundred or even thousand groups matter?

Eventually, a Group policy appeared to allow organizations to limit group creation to the members of a nominated group. But instead of treating the management of group creation as basic functionality, Microsoft made it a premium feature and demanded Azure AD Premium P1 licenses (now Entra ID P1). Over the years, the licensing requirement became less of an issue because organizations bought higher-end product licenses or purchased Azure AD Premium P1 for other reasons, such as to use conditional access policies. However, making restricted group creation into a premium feature was a staggering stance on the part of Microsoft. It reflected an ill-judged impulse to allow end users free rein over group creation.

The Effect of Teams

Microsoft Teams arrived in early 2017 to make the situation worse. Microsoft persisted in allowing anyone to create a group (team). Then Covid arrived and the usage of Teams exploded. Today, seven years after the launch of Teams, the product has 320 million monthly active users and the problem of group/team sprawl is bigger than ever before.

A glimmer of hope appeared in 2021 when it seemed like Microsoft was interested in taking on the problem of team sprawl disappeared without a trace. The group expiration policy helps as does the ownerless group policy, but neither makes much of a dent on the sprawl when Teams are spun up on a notion, used for a day or so, and promptly forgotten. Anything uploaded to the SharePoint sites owned by the forgotten teams also falls into the darkness of unused software. The lack of a Teams directory and the hiding of private teams from user view (recently addressed through a sensitivity label setting) contributed to the mess.

Microsoft 365 Groups are used by other apps like Viva Engage (Yammer). At times within Purview solutions (particularly), Microsoft insists on using a Microsoft 365 group where a simpler solution that doesn’t involve spinning up a SharePoint site exists, like a distribution list. But Microsoft persists on saying that Microsoft 365 Groups are the way forward for collaboration and distribution lists are antiquated. Well, I guess distribution lists have been around for a long time, and they work without contributing to group sprawl.

The net result of the last ten years is an unmanageable legacy of groups, teams, and sites where important and confidential information can remain undetected. The same is true for documents holding old, obsolete, inaccurate, and misleading information that’s now available for artificial intelligence to consume.

Just about the only good thing about the experience is that Teams helped make cloud storage the natural way to share information within Microsoft 365 and led to an explosion in usage for SharePoint Online and OneDrive for Business.

Exposing the Sins of the Past

Files buried in unused sites and messages in obsolete groups remain hidden until a new technology comes along and reveals the sins of the past and demonstrates how bad the mess really is. Office 365 Groups are better than Exchange public folders, but the public folder/distribution list combination persists in use today as a way for people to collaborate. Public folders are a glaring example of legacy technology that should have disappeared years ago. They haven’t and public folders remain a great example of how outdated and outmoded solutions can persist for years.

Copilot for Microsoft 365 is the new technology that’s come into focus recently and like a gun dog sniffing out birds, Copilot is extremely good at finding information stored in SharePoint Online and OneDrive for Business. I prefer using Copilot more than SharePoint Search when looking for documents because natural language search lends itself to finding files when you can’t remember more than a few details about something.

Figure 1 is an example of how good Copilot for Microsoft 365 is at finding information. I asked it about the Exchange hybrid configuration wizard and Copilot responded with information found in documents located in SharePoint Online. Unhappily, the documents are very old, and the information is very obsolete. I had forgotten about the documents but because Copilot finds anything that’s indexed by Microsoft Search, it hadn’t.

The good side is that Copilot for Microsoft 365 is an excellent navigator of the “abundance of information” people store in SharePoint Online, OneDrive for Business, Exchange Online, and Teams. But Copilot has no sense of the accuracy of information in one document over another, nor can it make creative judgments about which information might be more interesting and useful. The downside is that people might not always realize that what they get back from generative AI is well-written, plausible, but potentially useless text.

The Deployment Conundrum

Because Copilot for Microsoft 365 is so good at finding information, organizations have become aware that deploying the AI assistant might find and consume information when it shouldn’t. Copilot is unaware of confidentiality. It knows nothing about sensitive information or the secrets that an organization wants to keep. All it knows is that a user prompted it to answer a question based on the information available to it, and that means any information available to the signed-in account. One document seems as good as another. The fact that one document is stamped with a high-priority sensitivity label and the other is unlabelled is immaterial. If the signed-in user has the rights to access the information contained in the documents, Copilot can extract and use both.

The need to focus on curated information is why Restricted SharePoint Search exists. By limiting SharePoint Search to a list of 100 sites, organizations can limit Copilot to using the information stored in those sites (and the OneDrive for Business account for the signed-in user plus documents shared with that user). The intention is to suppress Copilot’s appetite for finding documents by limiting it to curated sites. Any other site is hidden, and unsearchable by both Copilot and end users but still accessible for eDiscovery. In a nutshell, Copilot can’t see the tangled mess of sites created by groups and teams.

A Sticking Plaster

Restricted SharePoint Search is a sticking plaster. It solves an immediate problem for organizations who want to deploy Copilot for Microsoft 365 but are worried about inadvertent data leakage. The fear is that Copilot will find and use confidential material in an unexpected location, perhaps a SharePoint site created for a team working on drafting some project ideas. The information in the site might be undated, inaccurate, or misleading, but to Copilot, a document is a document, content is content, and it’s all valid input to rummage over. In short, we have an information governance mess.

Cleaning up unused teams and their sites could be a massive task. Document libraries must be checked just in case something valuable lurks there. The clean up will require a dedicated team with input from compliance officers to ensure that anything that’s removed doesn’t conflict with business requirements to retain information for a certain period. A good start might be to make more aggressive use of retention policies to remove old information. Issuing site owners with listings of documents stored in document libraries might also encourage a clean-up.

Asking users to remove old and unwanted files from their OneDrive for Business accounts is akin to asking them to clean out the mythical Augean stables. If Hercules couldn’t complete that task, I have my doubts that the average user will remove the accumulated rubbish from their OneDrive account. They certainly don’t for mailbox contents. In both cases, life is just too short to deal with digital debris.