I have been using Copilot for Microsoft 365 for two full weeks in within our organisation (no test/dev platforms) since Microsoft made this more generally available on the 14th January 2024.

Two weeks on, I wanted to share my experience of using Copilot in Microsoft 365 fairly aggressively. I am breaking this blog into a series, focusing on different aspects of the experience starting today with the apps I spend most time in (as I am sure you do too) – Microsoft Teams and Microsoft Outlook.

The other question I will try to answer is – so far, two weeks in, is it worth the £25 pupm!

Great Expectations

Despite working with organisation readiness since June with a focus on organisational data and security readiness, using a combination of labs and “closed” demo labs, this was the first time I had real hands-on exposure to the hype that is Microsoft Copilot. Like many, I was impressed by the many iterations of Microsoft’s sizzle videos, sneak peaks and on-stage demos from Microsoft and their early access programme (EAP) users and as such I was extremely excited to finally get my hands on the real thing and use it within our organisation (as part of an internal adoption trial).

One thing to note, before diving into this blog is that my personal expectations for Copilot were (and still are) extremely high.

The Copilot Onboarding Experience

This walks through the first couple of days from getting a license to being able to use Copilot in anger.

I closed my Office 365 apps (we are running on latest versions which is a pre-req) and slowly the Copilot experience begin to “light up in my apps”. First to get Copilot treatment was Microsoft Teams, followed by Microsoft Word. A few hours later, Copilot “lit up” in other apps including Loop, PowerPoint and Excel followed the next day (yes next day) in Outlook. The mobile apps also became Copilot active about the same time.

Other members of our initial internal pilot, had a similar experience but not in the same order.

One thing to note, is that if you are part of the first tranche of users within your organisation, then, Microsoft “kicks off” the Sementic index engine which runs across your tenant. This takes a few days (longer for larger organisations) and works from most recent events and content backwards. This means that things seem to “turn on” or “work” at different times initially. Users added later have a more rapid onboarding experience.

A Word on Data Preparation

Much of the technical preparation and guidance for organisation adopting Microsoft 365 Copilot is around data readiness and the need for “proper” adoption and training for users as Copilot is not like another new feature you simply turn on – well at least if you want to get the best out of it and demonstrate high returns on your £25 pupm investment. Much of this “readiness” is not a new requirement as such, but the way in which Copilot works is, and should, be a wake up call for organisations to spend time implementing a proper data governance and lifecycle policy.

Much of this preparation is just good practice, but in Copilot terms, not having the above will impact not only the user experience but accuracy and usability of the service unless you spoon feed it the data you need.

Why? This is because, what makes Copilot unique is it’s access to the Microsoft Graph and your underlying data which is powered by Microsoft 365 Search and the Semantic Index. This involves three key pillars around data;

• Understanding where your data is located and who has access
• Understand the context of your data – this includes key words, titles, versioning etc
• Understanding the data the organisation needs (and does not need) – archive, search terms, lifecycle management, retention etc.
• Access and accessibility – data privacy, security (access control), and other policies in place.

This many organisations, getting these things in shape (if not already) is not a simply task – it takes time, structure, training (especially if you are going to label and classify data) and often process change. If you don’t have these things in place, its not a show stopper for Copilot but it might mean you expose existing risks (Copilot operates under the user’s context). As such, Copilot is a good trigger point/reason to look at this – whilst ensuring good business change, adoption and training are undertaken to give users the information they need to work better with your company data

Once concern I have, is that with the entry point to Copilot now just a single license, organisations may not give this area the right level of attention which may lead to user issues, sudden changes in policy (what do users have access to) and un expected results.

That said, if you are working in Teams and Outlook where the data it is referencing is more recent, relevant to the meeting or conversation or “part” of the chat or email – the data stuff is less relevant…

I have covered this more in other articles….

Copilot for Microsoft 365 – First Impressions

#BeyondExpections and far better than using ChatGPT

Ok this is a bold statement but you need to remember that Microsoft 365 Copilot uses GPT-4 (and GPT-4 Turbo) under the hood.

Copilot in Microsoft 365 instantly adds value to my day. What makes this so so so much better that a standalone tool (and even Copilot in Edge) is the fact that it is embedded natively into your Microsoft 365 apps and services. There is then the fact that not only does it work in the context of your apps, it also operates within your organisations Microsoft 365 data and compliance boundary and the way in which it leverages the Microsoft Graph to “perform its voodoo”. Copilot in Microsoft 365 uses a sophisticated data access methodology which uses Retrieval Augmented Generation against content and context, retrieved from the Microsoft Graph (with the Semantic Index). This means that Copilot not only understands what data to use, but is also aware and understands the relationships with the data, it’s context, your meetings, emails, recent files, team members, people relations and interactions and more.

The combination of Copilot and the Graph API enables Copilot’s powerful features, and this is just the start. Microsoft also supports the increasing use of connectors and plug-ins, which allow data ingestion or connection to the Microsoft Graph. This means you can “plug-in” or “connect” third-party data sources to Copilot and extend its reach beyond the Microsoft 365 environment, while still being protected by the “trust boundary”.

In my two weeks, I have found the performance of Copilot across all the apps fast and interactive and much better than I experienced with Chat GPT and the “free” version of Copilot in Edge (Bing Chat)] The main reason for this is that Copilot using the most recent and premium versions of the Open AI Large Language Models (LLM), which use the newest GPT-4 and GPT-4 Turbo models which mean they are not only faster but can work on, and create bigger documents, process more input and leverage multiple data sources in which to form its’ response.

Hands on with Copilot in Teams

We spend a lot of time in meetings. According to Microsoft’s Work Trend Index report most of spend between forty (40) and sixty (60) percent of our week in meetings so if there is anywhere that Copilot can make a welcome impact its Teams.

Copilot in Microsoft Teams is your personal assistant before, during and after meetings. The during bit is really impactful.

Using Copilot During a Meeting

In order to use Copilot to it’s fullest, you need to make sure that meetings are transcribed (ideally recorded too). Transcribing and recording a meeting retains the transcript and recording for you to “recap” later (see Teams Intelligent Recap). As the meeting organiser, you can also choose to allow Copilot in the background without transcribing (if enabled by IT), but be aware if you do not transcribe the meeting, you will not be able to use Copilot once the meeting has finished. For best results, make sure you transcribe the meeting at least (but ensure you tell or ask people first!).

When the meeting is running, you’ll see the Copilot button in the meeting ribbon and activating it, brings up an integrated interface to the right to the meeting (just like the chat window does).

Note: Copilot prompts are only visible to you. Other participants cannot see or access your prompts or see Copilot’s analysis or results from what you ask.

Using Copilot After a Meeting

When a meeting has finished (whether you didn’t attend it, missed it, or had to leave early), you can access the meeting recap from the “recap” tab in the meeting. The recap contains the notes taken in the meeting by attendees, the recording (if recorded) and the transcript (if transcribed). If you have Teams Premium or a Copilot License, you also see see the “AI Generated” notes, which contain notes and suggested actions generated by Teams AI [this is both a Teams Premium and Copilot thing].

Just like the in-meeting experience, Copilot opens in the right hand pane, where you can interact and use your prompt (as questions) to extract the information you need.

While Copilot generates the answers, it always displays the reference time in the meeting and who said it, so you can jump to the transcript or meeting recording (assuming the recorded or transcribed). This is useful in case of course the transcription is not 100% leading to Copilot making an assumption based on the transcript. Always good to check right!!

What can Copilot do in Teams Meetings?

Using Copilot in Teams is such a game changer in meetings.

Use-case	Prompt
Summarise the meeting so far	Summarise the meeting so far. Put the information in a table clearly stating the topic discussed, key points and opinions of each people.
Discuss Pros and Cons of the topics discussed	Create a table of pros and cons for [insert topic] discussed in the meeting.
Assess the mood of the meeting.	What was the sentiment of the meeting? Which people expressed their views the most and did the participants generally agree with each other?
Assess the effectiveness of the meeting	Were there any unanswered questions in the meeting that need to be followed up.
Plan the follow up.	What was agreed in the meeting, what suggestions were made and what would the suggested next steps be? Put the results in a table and identify the most suitable owner for each action.

Using Copilot in Chat (and Channels)

Copilot is also really helpful in chats or when working in conversations within Team chat. Here I see two main use cases.

1. To catch up or summarise chat threads, missed messages or other information
2. To help you communicate better and more to the point.

You can ask Copilot to suggest next actions from the chat context, summarise the thread or create replies for you.

This available now in Chat and coming “soon” to Channels Chat in Team sites too!

Things you may choose to ask in a chat window could be.

• Show me highlights from the past x days.
• What decisions were made?
• What questions have been asked since xxxday?

Hands on with Copilot in Outlook

The second place (well mine) we all spend far to much time is Outlook. Its where many conversations (that could be an IM in Teams) happen, but also where many business to business communications and more formal communication takes place.

I have been using ChatGPT and Copilot In Edge since they first came out to help me re-write “some” emails or to help me adjust the tone of what I am writing, but Copilot in Outlook takes this to a whole other level.

Its worth noting that Copilot in Outlook is still “evolving” and is not yet (disappointingly) on par with the promotional “sizzle” videos Microsoft have been showing off (but it is coming). Today Copilot works in three ways.

1. Drafting with Copilot – where you can tell Copilot what you want to see and it will draft the email for you or you can pick from standard “templated” responses.
2. Coaching with Copilot – where Copilot works “with” you while you are writing to help you perfect/tune your email or response
3. Summary by Copilot which provides as it says an overview of a thread of emails which is really useful if you are catching up on a long email conversation.

Drafting with Copilot

This is what most will be familiar with (and expecting) if they have used Copilot in Edge, or Chat GPT to write text based stuff. The main difference with Copilot in Outlook is also that it can reference and access recent files inline. Here is an example of an email I asked Copilot to draft.

Coaching with Outlook

This is similar to drafting, but works more like Grammarly or Microsoft Editor. Instead of drafting the entire email, you start it off and then Copilot works with you on the fly to provide guidance about how to better shape and perfect your email. In this mode, Copilot doesn’t re-write the mail, it helps you to perfect it.

Summary By Copilot

This is used to bring together an email chain (the longer the better) into key points and actions. It is not replacement for manually combing through the email thread but is really useful for playing catch up.

More AI things are coming

many of the other really cool feature are yet to go live in Outlook. One of the ones I am waiting patiently for is the ability to “Follow a Meeting”. Follow will be a new meeting response (RSVP) option that goes beyond the traditional Accept, Tentative and Decline choices geared towards individuals with high meeting loads and conflicting meetings each day. Follow is the ideal RSVP option for meetings you can’t attend but still want to stay engaged and receive info about.

When you follow a meeting, you will get all the updates and insights about the meeting without having to attend it. It is expected later this year.

---

Two weeks in – Is Copilot worth the cost?

It’s still early days, but here’s my initial view. Hell yeah!

We are of course talking about just two weeks of use, in which I have “got my head around it”, educated my self (mainly out of hours) on how to write good prompts to get what I need it to do and then of course put it to practice in real life. Over that time, I:

• Have used Copilot in Teams to take notes, write up actions and also share a summary a notes to my OneNote. Across 10 meetings, I estimate it saved me ~15 mins per meeting.
• Used Copilot in Teams to record and take notes in 5 meetings I could not attend and then used Intelligent Recap (both a Teams Premium and Copilot feature) to capture meetings notes and actions. All five meetings were 50 mins in length and given the time I used (around 15 mins) to review the minutes and notes,this saved me 35 mins per meeting that I did not attend without me missing the “beat” of the meeting.
• Used Copilot in Outlook a fair amount to reply to emails quickly, redraft a few team update meetings and project progressions as well as to recap email threads. I would guess this has probably saved me about an hour or so in all over the past two weeks.

If I add these up (excluding gains in using Copilot in other apps) this has given me back:

• [15×10]+[35×5]+[1×60] = 385 mins / 6hrs 25mins over ten days
• Around 12 hrs 50 mins (over 4 work weeks)

If we assumed an average IT role of £51k p/a – then this equates to a cost of £26.15 p/h (using £51,000/52weeks/37.5hrs). A Copilot for Microsoft 365 license is £25 pupm so based the example above (using my experience over the past two weeks) then we see :

• Productivity saving of 12.83x£26.15 = £335.50 per month per person
• Return on Investment of 12.62

---

Coming next – Word, Loop and PowerPoint

In the next blog, I’ll be covering my experiences with Copilot in Microsoft Word, Loop and PowerPoint which are the next set of apps I used most (after Teams and Outlook).

I also really would love to hear your views on Copilot for Microsoft 365