Microsoft Guidance on Search Crawls in SharePoint Online

Tags: SharePoint Online, Search

Ok, so there’s no “official” guidance.  Generally, the message has been “you need to wait.”

As you may know, we currently have no way to initiate a full/incremental crawl of content in SharePoint Online.  Unfortunately, outside of that statement, Microsoft hasn't provided any "official" guidance on the topic.  In troubleshooting a recent issue, I opened a support ticket regarding this.  A few days later, I received a little bit of guidance, but it still can't be considered official.  So, with that, here's the Q&A you can reference if you ever need to provide a customer with something more substantial than “we can't do it” or “you’ll just have to wait.”

  1. How often do full crawls run on content?

    1. There are not full crawls that run on content

  2. How often do full crawls run on people and user profiles?

    1. There are not full crawls that run on the user profiles

  3. How often do incremental crawls run on content?

    1. 4-6 hours after previous incremental crawl is completed

  4. How often do incremental crawls run on people and user profiles?

    1. 4-8 hours after previous incremental crawl is completed

  5. If I add a new user profile property, how long must I wait until it’s available as a crawled property?

    1. Wait for next incremental crawl to be completed

  6. After mapping a crawled property to a managed property, how long must I wait until that managed property is updated to reflect that mapping?

    1. For any mapping changes, as you may be aware, new/updated content would get the changes however for the existing content a re-index needs to be done. For user profiles, I have already explained above that since there is no option given to re-index, a support ticket needs to be raised and Microsoft can re-index all user profiles in the given tenant.

  7. If I add a new managed property, how long must I wait until it’s available for querying?

    1. This should be available immediately since creating managed property has no effect on the index, however if you are querying the index then index should be updated.

  8. Will Microsoft be providing us with the ability to manually initiate search crawls like we can in on-premises environments?

    1. As of now the answer is No, however if this changes then public announcement would be made

Additionally, there’s an excellent thread out on the Office 365 Yammer Network here:  https://www.yammer.com/itpronetwork/#/Threads/show?threadId=454314774.  If you’re not already a member of that Yammer network, you’ll need to submit a request to join.  Summarizing that post, Mikael Svenson (@mikaelsvenson) has written a couple of PowerShell scripts that address this issue as well.  He’s written up an article on that here:  http://techmikael.blogspot.com/2014/12/how-to-trigger-re-indexing-of-user.html.  You can jump directly to the PowerShell scripts here:  https://github.com/wobba/SPO-Trigger-Reindex.

On May 13th, the Office 365 Yammer Network hosted a SharePoint YamJam in which I posted a question about this.  In an effort to compile all of this information in one place, here’s the link and transcript:  https://www.yammer.com/itpronetwork/#/Threads/show?threadId=535940901.

Transcript

https://mug0.assets-yammer.com/mugshot/images/48x48/xgCHJZnBqX63xv8nP3Qp5C2P3ssprz4k

May 13 at 9:25am

SharePoint YamJam

In a hybrid scenario, will we have the ability to force a full crawl of our content in SharePoint Online from SharePoint on-prem? Not being able to manually initiate full crawls in SPO has been my biggest pain point for the last 6 months. Telling customers "you need to wait" is not great. Telling customers "I have a script someone wrote that should do it" isn't great (even though the script and Mikael Svenson (MVP)) are both great). ;-)

I'd really love to see an improvement in search in SPO and it'd be great if SP2016 on-prem could help there somehow.

Translated from English

cc: Mikael Svenson (MVP)

»

https://mug0.assets-yammer.com/mugshot/images/48x48/xgCHJZnBqX63xv8nP3Qp5C2P3ssprz4k

May 13 at 9:26am

I should add that this is really related to user profile updates...in the realm of refreshing crawled and managed properties more rapidly.

https://mug0.assets-yammer.com/mugshot/images/48x48/TxZRHk8Bhn2vsMJK1979B-383J0P-FgZ

May 13 at 9:26am

Yes in setting up and in training on our environment, the wait for search to catch up was painful. It would be nice if we could periodically force it, even if on a limited basis.

like this message like this.

https://mug0.assets-yammer.com/mugshot/images/48x48/VQRwDjFM6QMFVJHHTm0n73p0dwP65sz9

May 13 at 11:30am from Windows Phone

I think the technical challenge of this is that tenants might not have their own content source and it all just runs like a train. Also if people force a lot of full crawls it could mess up the total crawl dynamic.

But it would certainly be great, and I'm sure it's possible to achieve of they want.

Regarding Hybrid I'm curious as to how I can expunge all my hybrid crawled content if I accidentally killed the onprem SSA and db?

To summarize, when dealing with search in SharePoint Online, be prepared to wait and be prepared to tell your customers to wait…even with Mikael’s scripts.  Hopefully this will be addressed in the future.