Copilot for M365: A Reality Check on Its Capabilities

Copilot for Microsoft 365 continues to be the hot topic of conversations for almost every client I talk to. Copilot for M365 has brought the power of ChatGPT to many of our Office applications including (but not limited to) PowerPoint, Word, Outlook, OneNote, and Microsoft Teams. 

Copilot for Microsoft 365 is a game changer for organizations. The insights and productivity gains are unlike anything we’ve ever seen before. Without a doubt, the feature most clients want to take full advantage of is the Teams chat experience.  Most of these clients have vivid images in their minds about all the amazing things Copilot will do to make their lives better. Most of these ideas were germinated by well-intentioned marketing efforts from Microsoft, but wow, there is a LOT of education that needs to take place about what Copilot for Microsoft 365 can and cannot do.  

So, I thought I’d create a quick blog post to help educate others with the same questions and ideas about Copilot for Microsoft 365.  

Copilot for Microsoft 365 is not proactive 

  • “I want Copilot to tell me when I have a late task” 
  • “I want Copilot to listen to a meeting I’m in and tell me if there something important I need to pay attention to” 
  • “I want Copilot to alert me to an important email that I should respond to” 

All of these sound like great ideas, BUT Copilot has to be prompted by a user before it will give you any information. Copilot for Microsoft 365 is not actively monitoring your activities. It cannot automate any tasks. Copilot for Microsoft 365 is NOT a doer, it has to have a prompt to respond to. A user must trigger an interaction with Copilot for Microsoft 365.  

We do have some degree of automation coming with the new scheduled prompts feature, but you as a user will still have to prompt Copilot for the information you want to be informed about.  

Before we move on, I just want to point out that it’s a GOOD thing that Copilot for M365 does not automatically do things for you. Remember, it’s Copilot NOT Autopilot. You should be in control of the information Copilot is sending you and you should always have human eyes on content generated by Copilot to ensure there are no hallucinations and that the information is accurate. 

Copilot for Microsoft 365 plugins are NOT conversational 

Probably one of the features that are most commonly demoed misleadingly… hopefully not on purpose… is extending Copilot for Microsoft 365 using a plugin.  

Plugins for Copilot for Microsoft 365 allow you to extend Copilot into your other applications outside of Microsoft 365.  They also allows you to have Copilot take action for your (create a ticket, create a task).  Keep in mind, out of the box, Copilot only returns information to you, it does not take action on any of that information. So, currently, plugins are the only way to get Copilot for to actually “do” something. 

That being said, as of today, plugins are NOT conversational. They are “one and done”. They are “completions”, not “chats”.  A plugin cannot ask for follow up information.  

  • “I see you want to create a support ticket, what problem are you facing?” 
  • “You want to take PTO? How many hours do you want?” 

Plugins can only execute one action and they cannot ask for follow up information.  

Copilot for Microsoft 365 is not search! 

  • “I want Copilot to return all my emails from ‘x’” 
  • “I want Copilot to list all of my late tasks”  

Don’t get me wrong, Copilot for M365 is awesome for helping you FIND things.  

  • “I’m looking for the presentation that was shared with me last week in a meeting with Susan”.   
  • “Help me find the email about my service now ticket where I was having problems with VPN”. 

But Copilot is NOT search! If you ask Copilot to return a list of information, the amount of information returned is limited. It will not return everything! What’s more, it may not return the most important items from your request. Could you spend hours crafting the perfect prompt to give you better results? Sure, but no matter how nicely you ask Copilot it won’t return everything you as asking for if there are too many results.  

Copilot should not be your go-to for search or reporting. If you need actual search, if you need to sort, filter, and page… use Microsoft Search… don’t use Copilot for M365.  

Copilot for Microsoft 365 can’t work with SharePoint metadata… yet 

  • “I want Copilot to summarize the information in my SharePoint list” 
  • “I want Copilot to tell me all the about the high priority items in my SharePoint issues list”  

This one should be a no-brainer, but it’s just not there yet. Copilot is great for files, emails, and other information within M365 but it does not yet work with your SharePoint lists or metadata. You can’t ask Copilot questions about your SharePoint list nor to return specific information from your list. Believe me, I understand the frustration on this one. You went to all the effort to tag your files and use metadata in your lists, and you can’t access this information… yet.  

But what if I NEED Copilot to do all of these things??? 

What if you need one or all of these features? Are you just completely out of luck?  

Well, that’s where custom Copilots come into play. I would say about 80% of the conversations I have with customers about features they want to do in Copilot for M365 can be solved with a custom Copilot.  

Custom Copilots give you much more control over the user experience you are trying to create. Another benefit is that you lessen the risk of hallucinations or pollution of data from oversharing. Custom Copilots are fantastic for things like Onboarding, HR tasks, and sales related conversations. Just some of the benefits of creating a custom copilot are: 

  • More targeted results – Creating a custom Copilot allows you to take full advantage of an LLM without also having it overwhelmed by all of your content in M365  
  • Take action – You can effectively gather data from a user, confirm accuracy, and take action on that data; “create a task” “submit a PTO request” etc…  
  • Work with SharePoint metadata – Using Custom Actions or Power Automate you have full access to all of your SharePoint content and metadata 
  • More secure results – With a custom Copilot you can limit the data sources that are used for responses and limit the possibility of users getting access to information that may have been overshared  
  • Use other LLM’s! – With custom Copilots you can use other LLM’s through Azure OpenAI to get even better results for your specific needs.

What’s more, creating a custom Copilot doesn’t have to be an arduous task. There are a couple of options for creating them with varying degrees of skills that are needed.  In a follow up blog post I’ll go into much more detail about these options, but for those that can’t wait, your two options are:  

Copilot Studio 

Copilot Studio is the no code / low code approach to creating custom Copilots and is part of the Power Platform.  If you can create a Power Automate Flow, you can create a Copilot with Copilot Studio.  

Declarative Copilots  

Declarative Copilots are the pro code approach to creating custom Copilots using the Teams toolkit. Soon, you will also be able to build Declarative Copilots in Copilot Studio and from SharePoint.  

We are just now scratching the surface with what Copilot can and will do someday, so it’s important to fully understand its features to get the best results and not expect it to do things it is not designed to do.  

Be the first to comment

Leave a Reply

Your email address will not be published.


*