Some Complexities to Consider When Upgrading Scripts
Last month, I wrote about upgrading Exchange Online PowerShell scripts to use the Get-ExoMailbox cmdlet instead of its older Get-Mailbox counterpart. One of the reasons I advanced is that Get-ExoMailbox is faster at retrieving mailbox data, which led to some questions about performance in general. It’s easy (and quick) to fetch data for a few mailboxes, but once you need to deal with hundreds or thousands of mailboxes, it’s important to optimize.
Filtering Data
Which brings me to the topic of filters. It’s common to need to process just a few mailboxes of the full set available in a tenant. Filters apply conditions for PowerShell to select the right mailboxes. The better the filter, the faster you get the mailboxes you want.
Filters come in two variants: server-side and client-side. Server-side means that the data coming back from the server is filtered and ready to go. Client-side means that data comes down from the server and is filtered on the client. Generally speaking, for best performance, the golden rule is to use server-side filtering whenever possible.
Server-side filtering happens when the Filter parameter passes a query against mailbox properties to the server. Exchange PowerShell supports a wide range of filterable properties which can be used with its cmdlets. For example, this command returns mailboxes with the Office property set to Dublin.
Get-ExoMailbox -Filter {Office -eq "Dublin"}
The equivalent client-side filter fetches all mailboxes and pipes the set to a Where command to filter out the desired mailboxes:
Get-ExoMailbox | Where-Object {$_.Office -eq "Dublin"}
At first glance, you’d expect this code to work because the approach works with Get-Mailbox. On paper, it should not because the Office property is not in the default property set returned by Get-ExoMailbox. Remember, part of the reason why the REST cmdlets are faster than their Remote PowerShell counterparts is the reduction in the number of properties they return. However, for whatever reason, the code returns the set of mailboxes where Dublin is the value of the Office property even if the returned data contains blank values for the Office property. It’s just odd.
To make our code work propertly, we must tell Get-ExoMailbox to fetch the Office property.
Get-ExoMailbox -Properties Office | Where-Object {$_.Office -eq "Dublin"}
You don’t have to worry about specifying properties when you use server-side filtering because Exchange applies the filter on the server. The property restriction only applies for data returned to the client.
Remember that other parameters cause filtering to happen on the server too. Combining these parameters together creates an even more precise query. In this example, we add the RecipientTypeDetails parameter to instruct Exchange Online to return only user mailboxes:
Get-ExoMailbox -Properties Office -RecipientTypeDetails UserMailbox | Where-Object {$_.Office -eq "Dublin"}
Specifying the number of objects to be returned is also another form of filter:
Get-ExoMailbox -Properties Office -RecipientTypeDetails UserMailbox -ResultSize 100 | Where-Object {$_.Office -eq "Dublin"}
The critical point is to be as specific as possible in requesting data from the server. This will speed processing up and mean that you don’t need to discard information when it gets to the client.
Inconsistencies in Filters
A couple of inconsistencies exist in the way that server-side filtering happens for the new cmdlets. For example, you need to be careful with wildcards. Microsoft’s documentation says:
The -like and -notlike operators are limited in using wildcards (*). Specifically, you can only use wildcards at the beginning of a string value, at the end of a string value, or both.
This text makes it seem like you could take a command like this:
Get-Mailbox -Filter {DisplayName -Like "*Tony*"}
And upgrade it to use Get-ExoMailbox instead:
Get-ExoMailbox -Filter {DisplayName -Like "*Tony*"}
But the command fails with an invalid filter clause, which proves the need for testing even when a statement in Microsoft documentation gives reassurance that something should work.
Reversing Advice
For years, people have applied the golden rule to use server-side filtering to fetch mailbox data. But as we’ve just seen, the REST-based cmdlets do not work in the same way as the older cmdlets. In fact, the Exchange Online development team found that client-side filtering is faster for the new cmdlets, including Get-ExoMailbox. Microsoft says that they’re working on improving server-side filtering for the cmdlets.
After running some tests, it seems like the cause for Microsoft’s assertion is the way that Get-ExoMailbox needs to “warm up” when finding mailboxes. This includes preparing for result pagination, connecting to mailbox servers, and so on. The effect of warming up is easily seen by running the same query multiple times. The first run is always slowest and runs thereafter are much faster, and is easily verified using the Measure-Command cmdlet to run a command several times.
However, with an eye on future performance improvements, I’m not sure that I want to follow Microsoft’s recommendation to use client-side filtering with Get-ExoMailbox. The performance penalty at present doesn’t seem excessive and staying with server-side filtering avoids the need for further change in the future. However, this theory must be tested in the context of individual scripts. For some, it will be best to convert to client-side filtering, for others, the best decision is to stay as is.
I try to filter for two properties:
Get-EXOMailbox -ResultSize Unlimited -Filter “ArchiveGuid -Eq ‘00000000-0000-0000-0000-000000000000’ -And RecipientTypeDetails -Eq ‘UserMailbox'”
But that does not work. Anyone knows the right syntax for this?
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Get-ExoMailbox doesn’t support that filter. Get-Mailbox does:
Get-Mailbox -RecipientTypeDetails UserMailbox -Filter {ArchiveGuid -eq ‘00000000-0000-0000-0000-000000000000’}
Hi , can you help me with EXOv3 version for (Get-EXOMailbox -RecipientTypeDetails SharedMailbox -Identity ‘test’ -Properties ExchangeSecurityDescriptor).DiscretionaryAcl.Count
after EXOv3 this started to give 0 response . With using Get-MailboxPermission and Get-RecipientPermission i need to iterate each mailbox which is time consuming. The above command was able to identify the users with delegates and which helped me to fetch delegates only for those the count was greater than 0.
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
If something doesn’t work in V3 that used to work, you should report it to Microsoft to make sure that the problem is brought to the attention of the engineering group and is fixed. It’s entirely possible that they made a decision to deprecate the count.
When I ran this command:
(Get-EXOMailbox -RecipientTypeDetails SharedMailbox -Properties ExchangeSecurityDescriptor).DiscretionaryAcl.Count
The returned value was 12… So something is happening.
I want to insert a vairiable in the below command, Can anyone help?
Get-Recipient -PropertySet ConsoleLargeSet -ResultSize ‘1000’ -SortBy DisplayName -RecipientType ‘DynamicDistributionGroup’,’MailNonUniversalGroup’,’MailUniversalDistributionGroup’,’MailUniversalSecurityGroup’ -Filter ‘((ManagedBy -eq ”CN=UserName,OU=Users,DC=sub,DC=Domain,DC=com”))’
”CN=UserName,OU=Users,DC=sub,DC=Domain,DC=com” – This needs to be replaced with a vairiable. For eg. $xyz.
I will pass CN of the user in $xyz.
Any Help appreciated.
Got this fixed….
“ManagedBy -eq ‘$xyz'”
If anyone gets here like I did trying to extract data, the example Get-ExoMailbox -Filter {DisplayName -Like “*Tony*”}
throws a filter error like Tony mentioned, but Get-ExoMailbox -Filter {DisplayName -like “*Tony*”} returns results for me.
Just a small ‘l’ on the -like comparison operator
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Yep, it’s just one of those PowerShell things… It’s similar to the way that Graph comparisons insist on “true” rather than “True.”
Tony,
I’m trying to get daily unread email statistics for my office, I can successfully run this on our Exchange online. and it seems to work, but really I just want what’s in the Inbox for unread mail. Is there a SearchQuery attribute for folders? something like (Folder:Inbox)?
Get-Mailbox -ResultSize Unlimited -Filter {RecipientTypeDetails -eq “UserMailbox”} | Search-Mailbox -SearchQuery ‘(isread:false) AND (kind:email)’ -estimateresultonly -donotincludearchive
Sidenote, I’m shocked at how difficult this is. I tried using AdminDroid, manageEngine and a whole bunch of other pre-built apps and none of them can get this statistic for me (I asked and everything). so I’m resigned to do it myself.
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
The Exchange Online cmdlets have never delivered this kind of information (which is why EWS exists). If you want to access folder-level detail about item unread counts, you need to use a Graph query, like:
https://graph.microsoft.com/v1.0/users/{UserId}/mailFolders/Inbox/messages?$filter=isRead ne true&$count=true
Where UserId is the Azure AD identifier for the user’s account.
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Steve, I wrote this up to show how to get the count of unread messages from mailboxes:
Using the Graph API to Generate Mailbox Folder Statistics
A reader asked if it’s possible to use PowerShell to return the unread count for the Inbox folder in user mailboxes. The standard Exchange Online PowerShell cmdlets tell you a lot about mailbox folder statistics, but they can’t look inside a folder. But the Microsoft Graph APIs can, so a combination of PowerShell and the Graph deliver a solution to the problem.
https://office365itpros.com/2022/05/19/mailbox-folder-statistics/
Just for your feedback.
I thought about converting to Get-ExoMailbox for the below CMDlet but I still get a better reponse using the old filter switch. I am surprised about the limited v2 filter options.
Get-Mailbox -Filter “AuditEnabled -ne ‘$true’ -and (PersistedCapabilities -ne ‘BPOS_S_EquivioAnalytics’) -and (PersistedCapabilities -ne ‘M365Auditing’)”
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Some of the filtering capabilities are still not as good with the new cmdlets. They are built for speed in terms of fetching very large numbers of objects and this meant that some compromises had to be made in terms of how things work. We have both cmdlets available, so the rule of thumb is probably:
When you need to fetch large numbers of mailboxes, use the REST cmdlet (Get-ExoMailbox) or a Graph call.
When you need precision, use a server-side filter… and use Get-Mailbox if it is more functional than Get-ExoMailbox.
How do we apply AND OR in the filter set ? I tried many things but none works. Is there any specific way to write it ?
The Real Person!
Author Tony Redmond acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
What kind of filter do you want to apply? You might be forced to use a client-side filter as the -Filter parameter (server-side filter) has some https://docs.microsoft.com/en-us/powershell/exchange/recipient-filters?view=exchange-ps#custom-filters-using-the-filter-parameter
Hi sir,
How can we Filter mailbox databases having default quota usage set to 2Gb and then sort them based on the mailbox count they are having and then select the database having least mailbox count.
Thanks in Advance
This article is for Exchange Online. You can’t filter on mailbox databases in Exchange Online.. There are a bunch of other articles covering getting information about mailbox databases in Exchange Server. Have you looked(in this site and others) for suitable scripts?