Category: Data and AI

Posted on

The Future of AI Agents and shift to Agentic AI

Image of Satya Nadella

In a recent podcast episode with Bill Gurley and Brad Gerstner, Satya Nadella – CEO of Microsoft, discussed a wide range of topics related to his role at Microsoft, the state of the technology, business growth and capitalism in this new “AI Era”.

The podcast which you can watch on YouTube here covered some interesting topics including the Future of AI Agents and their potential to transform how we interact with technology. In this blog (worth a listen), Satya gives his predications/insights into the future of AI Agents and emphasises that AI agents will fundamentally change the landscape of software-as-a-service (SaaS) solutions, predicting that the traditional notion of business applications will collapse in the era of agentic AI.

What is Agentic AI?

Agentic AI refers to artificial intelligence systems that can make decisions and take actions autonomously, without direct human intervention. These systems are designed to perceive their environment, reason about the best course of action, and execute tasks independently. In short these agents are designed to function as workplace teammates, capable of handling various tasks across different applications.

As example, in e-commerce platforms, instead of static, rule based chatbots, agentic AI-driven systems can track a customer’s journey, personalised recommendations, and assist with returns seamlessly without user / supervisor input. These agents will be able to actively learn from interactions, optimising the customer journey in real time and learning about user preferences.

Agentic AI typically inhibits the following features:

  • Autonomy: Agentic AI systems can operate independently, making decisions based on input data and predefined goals.
  • Adaptability: These systems can adapt to changing circumstances and inputs, adjusting their actions to achieve their objectives.
  • Proactivity: Agentic AI can anticipate user needs and take actions without explicit instructions, making them more proactive in their behavior.
  • Collaboration: In the future, agentic AI systems will be able to work together in multi-agent networks, collaborating to handle complex tasks that a single agent cannot manage alone.

Changes Ahead for AI Agents

The video / podcast is an hour and a half (but an enjoyable and informative listen). Reading between the sections, Satya talks extensively about where he sees AI Agents evolving massively through 2025. I have summarised this below.

Increasing Sophistication and Capabilities

AI agents will become increasingly sophisticated and capable, eventually replacing traditional software applications. These agents will be able to understand and anticipate user needs, providing personalised and proactive assistance. They will leverage advanced natural language processing (NLP) and machine learning algorithms to interact with users in a more human-like manner.

Autonomous AI Agents

Through 2025. we will see autonomous AI agents handle more and more complex tasks with minimal human oversight, optimising workflows and enhancing efficiency across industries. These agents will streamline workflows, manage intricate operations, and simplify everyday activities. For example, OpenAI’s “Operator Agents” will autonomously execute multi-step processes, such as scheduling meetings or managing projects.

Multi-Agent Networks

Sayta talks abiut the “future” being a place that is not about singular agents but more about networks or systems of agents where agents can discover and collaborate with other agents. These multi-agent networks will enable agents to handle tasks that they can’t do themselves by invoking other agents (agents talking to other agents). This collaborative approach will enhance the overall capabilities and efficiency of AI agents.

Vertical AI Agents

Vertical AI agents, which are specialised for specific industries, are expected to have their moment in 2025. These agents will dominate their respective fields by offering tailored solutions that address industry-specific challenges. For example, retail AI agents will act as personal shoppers, offering personalised recommendations and optimising inventory management.

Persistent Memory and Personalisation

AI systems with persistent memory will enable highly personalized interactions, transforming AI into long-term companions that adapt to user preferences and habits. This capability will allow AI agents to provide more relevant and context-aware assistance, enhancing user experiences.

Emotional Intelligence

Future AI agents are expected to possess emotional intelligence, allowing them to understand and respond to human emotions. This will enable more empathetic and effective interactions, particularly in customer service and healthcare settings.

Integration with IoT and Personal Devices

AI agents will increasingly integrate with the Internet of Things (IoT) and personal devices. This integration will enable seamless interactions across various platforms and devices, creating a more connected and efficient ecosystem. For example, AI agents in smart homes will manage household tasks, monitor energy usage, and provide personalized recommendations.

Ethical AI and Transparency

As AI agents become more prevalent, there will be a greater emphasis on ethical AI and transparency in decision-making. Ensuring that AI agents operate responsibly and transparently will be crucial for gaining user trust and acceptance. This includes addressing issues related to data privacy, bias, and accountability.

Proactive AI Agents

Proactive AI agents will anticipate user needs and take actions without explicit instructions. For example, an AI assistant might reorganize your day based on traffic updates and weather, reschedule missed appointments, and even draft personalized messages. This proactive approach will make AI agents more valuable and indispensable in daily life.

Enhanced Communication and Collaboration Tools

AI agents will enhance communication and collaboration tools, making it easier for teams to work together. These agents will facilitate real-time collaboration, manage project timelines, and provide insights to improve productivity. They will also assist in content creation, research, and workflow automation.

Shift and the of SaaS apps?

Another interesting section to listen too is at around 31 minutes, where Satya talks about his vision of how AI agents could potentially replace traditional SaaS (Software as a Service) applications. Whilst something that will not happen over night, he talked about the shift from business apps with connectors into other apps, but in an agent to agent and agent to back-end system.

We can already have connectors into applications like SAP, Dynamics etc. A great quote he used was “when was the last time any of us really went to a business application” In the AI age, we access the data in these systems from a mesh of data sources which over time, these back-end SaaS systems would eventually become obsolete as AI agents take over multi-repository CRUD (Create, Read, Update, Delete) operations. This shift would lead to the collapse of conventional business applications, with AI agents handling the core logic.

The idea will be that you simply pull information from systems through AI Agents such as looking up customer details, updating inventories, changing a contact in NetSuite CRM for example or checking delivery status for an order.

Other examples, Sayta talked about with regards AI Agents included:

Infinite Memory: He explained that infinite memory refers to the ability of AI agents to retain and recall information over extended periods, much like a human’s long-term memory. This capability will allows AI agents to build on past interactions and experiences, making them more effective and personalized in their responses and actions.

Proactive Task Management: AI agents are envisioned to operate autonomously, handling complex tasks such as processing customer returns, managing shipping invoices, and optimizing supply chain operations. This proactive approach reduces the reliance on user-initiated interactions, further diminishing the need for traditional SaaS applications.

Automation of Business Logic: Satya explained that AI agents would be able to automate many backend business processes, creating a new tier of multi-agent orchestration. This means that business logic, which is currently hardcoded into individual applications, will be managed by AI agents across multiple apps or databases and will adapt based on useage and need.

Integration with Existing Tools: Nadella highlighted the integration of Python with Excel as an example. AI agents can use Excel’s visualisation capabilities for advanced tasks, transforming it into a more intelligent and autonomous tool. This integration demonstrates how AI agents can enhance existing applications, making them more efficient and reducing the need for traditional SaaS apps.

Posted on

The Windows Copilot app is now a “real” app

If you are not a fan of PWA (progressive web apps), the Microsoft is bringing good news. Windows Insiders are getting a new version of the Copilot app for Windows 10 and 11 which replaces the web-based application with a new native version.

The old app (or current app if you are not a Windows Insider) is a Progressive Web App which limits some of the Windows control such as quick view that is available in native Windows Apps. recently ChatGPT published their Windows App into the Microsoft Store and this latest update from Microsoft now makes the Copilot a real app too!

In the announcement, Microsoft said that

With this update, the previous Copilot progressive web app (PWA) is replaced with a native version. After installing the Copilot app update, when you run Copilot, you will see it appear in your system tray.

Microsoft Windows Insider Team

Whilst it’s hard to notice immediately differences, after installing the updated version (1.24112.123.0) Copilot on Windows is now a “proper” app rather than a WebApp.

This also means that Quick View can be used now with Copilot which lets you move the quick view window and resize it to suit your workflow. By default, the Copilot app in Windows uses the RegisterHotKey function and sets Alt + Space keyboard shortcut to open Copilot in Quick View mode which can be used to open and close Copilot’s quick view whenever you need it.

If you need to switch / flip back to the main Copilot app window, then this can be done by clicking the icon at the top left corner of the quick view window.

Devices with the dedicated Copilot key will open the Copilot app up the main window.

Posted on

Streamlining Copilot Adoption: Reducing Data Oversharing in Microsoft 365

One of the concerns I often talk to organisations about, is the fear that Copilot might surface sensitive information that it should not have access to due to IT/Compliance teams not really knowing who has access to what. The phrase “Security through obscurity” is often what we heard being used.

The primary cause of this is the over-permissioning and sharing of files, which is a growing concern for organisations and one of the “blockers” often cited in Copilot Adoption.

The over-sharing problem

The ability to reason over employee data and shared organisational data is one of Microsoft 365 Copilot’s strengths over other Gen AI tools (that need feeding). These responses Copilot gives and the content it creates rely on access to data that the user already has access to across their organisation’s Microsoft 365 environment. And here often lies the problem. If an organisation has low levels of data governance, no data classification and labelling, combined with high levels of over-sharing can create real concerns for IT and Data Compliance teams.

One of the reasons that Copilot often has access to data that it “perhaps” shouldn’t have is not due to security flaw or issue across Copilot or Microsoft 365, but because files or sites have been shared too widely and have no (or the wrong) privacy and sensitivity set. Addressing this is no small task since many organisations will have million of files and tens of thousands of SharePoint and Teams sites.

Organisations and even teams within organisations often operate at various levels of maturity in governing SharePoint data. While some orgaanisations strictly monitor permissions and oversharing of content, others do not. The situation is further complicated because many people, teams and organisations have “legitimate” reasons to share “some” data widely within the organisation. This can mean users in your organisation may make choices that result in the oversharing of SharePoint content. As an example

  • Users may save critical files in locations accessible to a wider audience than intended.
  • Users may prefer sharing content with large groups rather than specific individuals.
  • Users might not pay close attention to permissions when uploading files.
  • Users may not understand how to use sensitivity labelling (if enabled) to control access.

Services such as Microsoft SharePoint and Microsoft Copilot for Microsoft 365 utilise all data to which individual users have at least View permissions, which might include broadly shared files that the user is unaware of. As a result, users might see these applications as exposing content that was overshared. Oversharing can lead to sensitive information being exposed to unintended recipients. Users, while well intentioned, might not always grasp the implications of their sharing choices. They might overlook permissions or opt for convenience over security.

As a result, it’s important to use the permission models in SharePoint to ensure the right users or groups have the right access to the right content within your organisation. The following sections describe the key steps that administrators can implement to configure their SharePoint permissions model to help prevent data oversharing.

Dealing with Oversharing

The good news is that Microsoft is adding new features to SharePoint and Purview to make it easier to see, understand and control over sharing across Microsoft 365 with a hope to help adoption efforts and wider roll out of Microsoft 365 Copilot. This includes new Data Security Posture Management (DSPM) and enhancements for Data Loss Prevention policies in Microsoft 365 Copilot, and SharePoint Advanced Management. These can help automate site access reviews at scale and add controls to restrict access to sites if they contain highly sensitive information.

Microsoft have also released a blueprint guide for organisations planning to or deploying Copilot. These are nicely tailored to adjust to those with mainly Microsoft 365 E3 and E5 licenses respectively.

These new tools IMO are going to be vital to help organisation understand and address oversharing so they feel more feel confident in their employees adopting AI tools like Microsoft 365 Copilot.

AI is really good at finding information, and it can surface more information than you would have expected. This is why it’s really important to address oversharing. Typically, these issues are a by-product of good collaboration, particularly across Teams, SharePoint sites and OneDrive.

Alex Pozin | Director of Product Marketing | Microsoft

From early 2025, Microsoft will make access to SharePoint Advanced Management (SAM) available at no extra cost to Microsoft 365 Copilot subscriptions. Outside of this, SharePoint premium (which includes SAM ) will be available at a cost of around $3 per user each month.)

New Capabilities in SharePoint Advanced Management

There are also new features for SAM that Microsoft says will provide greater control over access to SharePoint files. 

  • New permission state reports (available now) can identify “overshared” SharePoint sites. The site access review feature can then provide a easy way to ask site owners to review and address permissions.
  • Restricted Content Discovery – which should start to roll out this month in public preview (December 2024), will allow IT admins to prevent Copilot from searching and processing data in specific sites for content and result generation. This does not prevent direct access to the site meaning that users can access the content directly as normal. This feature builds on the SharePoint Restricted Access Control, which was released last year, and lets IT admins restrict site access to specific sites to just “site owners” only, while also preventing Copilot from indexing and summarising files in these sites.

One of the use cases for this, are for where there are data locations containing information that needs to be contained to a set of people – such as financial reports, M&A planning, amnd other secret stuff. IT need to be confident that these locations and files will not show up in SharePoint searches and will be well out the reach of Copilot or other AI tools, essentially making sure that nobody can accidently or unintentionally be aware of, see or access the content. This is where Restricted Content Discovery comes in – locking down and hiding this information from plain site and from Copilot’s retrieval augmentation and indexing.

New Capabilities in Microsoft Purview

Microsoft are also adding new capabilities in Purview too. Purview is available as standalone or is part of Microsoft 365 E5.

/

Microsoft Purview is a centralised hub within Microsoft 365 that helps organisations meet regulatory and compliance requirements. It helps organisations manage their compliance obligations, protect sensitive data, and mitigate risks within their Microsoft 365 environment.

Here, there are new tools to help identify “overshared files” that can be accessed by Copilot. These includes oversharing assessments for Microsoft 365 Copilot in the Data Security Posture Management (DPSM) tool which is now in Public Preview (from December 2024) and can be accessed via the newly revamped Purview portal.

DSPM Portal in Microsoft Purview

The oversharing assessments are designed to highlight data that may present exposure risk by scanning files for sensitive data and identifying data repositories such as SharePoint and Teams sites where access permissions appear to be too wide and broad. The tool will also provide recommendations to admins and site owners for ways to mitigate oversharing risk, such as adding sensitivity labels or restricting access from SharePoint.

For example, DSPM can detect and help you deal with controlling ethical behaviour in AI (example demo environment below). For all the recommendation, Microsoft provides a simple step by step “wizard” to help IT and Compliance add policies.


Microsoft Purview Data Loss Prevention for Microsoft 365 Copilot, also in public preview, enables IT and security admins to create data loss prevention (DLP) policies to exclude certain documents from being processed by Copilot based on a the file or sites sensitivity label. This applies to files held in SharePoint and OneDrive, but can be configured at other levels, such as group, site, and user, to provide more flexibility around who can access what.

Insider Risk Management has also been updated to detect “risky AI usage.” This even includes user prompts that contain sensitive information and attempts by users to access unauthorised sensitive information. What’s key to note here is that this feature is not just limited to Microsoft 365 Copilot and also also covers Copilot Studio, and ChatGPT Enterprise.

Oversharing Blue Prints

I really like this. Microsoft’s new blueprint resource pages on Microsoft Learn provide recommended approaches and guidance for organisations to help them understand, mitigate and manage oversharing during what they define as the three main stages of Microsoft 365 Copilot deployment.

  • Pilot [Pilot]
  • Wider Deployment [Deploy at Scale]
  • Organisational Rollout [Operate]

Microsoft provide two blueprint designs. A “foundational path” and what they call an “optimised path” that uses some of the more Microsoft 365 advanced data security and governance tools found in Microsoft 365 E5 subscriptions.

Is there funding available to help?

It depends – but most likely!

Microsoft have a Cyber Security Investment Program open to select/specialist partners like Cisilion. These provide funded workshops, assessments and proof of value deployments across key Security workloads including Microsoft Purview as well as structured Copilot pilot deployments, vision and value

Organisations should speak to their Microsoft Solutions Partner for more information. You can contact Cisilion here should you need to.

Conclusion

In many of the discussions I and my team at Cisilion have with customers, we see that almost all of the organisations we work still have concerns over data governance in the realm of AI access. Of these most expect Microsoft to help them address these whilst some have already invested in third party tools to help them get a “grip” on their data and sharing.

We have seen a plethora of customers invest/upgrade to high-tier Microsoft 365 plans (including E5 Security and Compliance) or full Microsoft 365 E5 in order to gain access to Microsoft Purview. Some argue these tools should be provided as part of their Copilot investment, so it is great to see Microsoft meeting customers in the middle and at least providing some of these tools as part of this license investment.

The issue is not Copilot per-say, but it is that Copilot with it’s ability to access compnay data is causing more organisations to double down and look at the existing issues they have of too many SharePoint Sites, too much over sharing, orphaned data (data with no owner) inadequate data classification and labeling.

By addressing security and data governance and levering the new tools available, this at least should solve one of the blockers to AI adoption.

The second is Adoption and Change Management – more on that in the next blog post!

Useful links.

Posted on

Microsoft’s Recall hits preview on Qualcomm, Intel and AMD AI and Copilot+ PCs

Microsoft has recently expanded the testing of its innovative Recall AI feature to Intel- and AMD-powered AI and Copilot Plus PCs. Initially available on Qualcomm-powered devices only, this feature is now accessible to a broader range of devices for testing.

What is Recall?

Recall was the keynote Windows feature announced when Microsoft unleashed the Copilot+ PC  when they were released in September this year.

Initially recalled due to privacy concerns this is now in Public Preview for Windows Insiders on the Dev Channel.

Recall works by taking screenshots of almost everything you do on your  Copilot+ PC, (these are devices with dedicated NPUs that run at 45 Trillion Operations per Second (TOPS) or more). Recall makes it easy to search and recall past activities such as “the train route I was looking at on Tuesday” rather then scanning back through Internet search history.

Recall on Copilot+ PCs

This feature is entirely optional to use, but when enabled enabled, helps users find previous work, content or Internet data through natural language search or an interactive scrollable timeline.

As the user, you are completely in control of what snapshots are saved and how long for, and have the ability to delete them as needed, ensuring upmost privacy and security. Snaps shots require TPM, secure boot and Windows Hello to be active on the device and Microsoft has not access to the data which is encrypted on your device.

The power of Edge AI

Unlike services like Copilot, Recall and many of the newer Copilot+ PC features leverage local LLM models on the device as well as the NPU’s present on Copilot+ PC devices like the Surface Laptop 7 and Pro 11 range. As such when you install the #WindowsInsider Dev builds, you’ll also notice that Windows Updates installs a number of processing services as well as the Phi Silica LLM.

Recalls’ enhanced security and privacy

Microsoft has implemented many new security updates and controls to address initial concerns raised by security folk and early testers.

As I mentioned, accessing snapshots now requires Windows Hello for authentication, and the feature mandates the use of BitLocker and Secure Boot. Additionally, Recall can now automatically detects and excludes sensitive information like credit card details and passwords from being saved.

Click-to-Do and more AI features

Alongside Recall, Microsoft is also allowing Insiders on Copilot+ PCs to test out Click to Do feature, which recognise text and images in snapshots and content in screen allowing users to perform actions like copying text, invoking Copilot, saving and editing images and more. This functionality extends beyond Recall, enabling users to take actions on images and text with a simple Windows + Q key or Windows Key + mouse click.

In Paint, the new Cocreator top lets you create art and images by simply typing in text prompts. The Photos app has also been updated with new tools including Image Creator, which lets users make images from text prompts, and Restyle Image, which lets users add different artistic styles to their existing photos. You also get powerful generative erase tools which can be accessed directly from the app or from Click-To-Do.

These tools use local AI and analysis models on the Copilot+ PCs to work efficiently on the device itself through the use of the NPU.

Conclusion

Microsoft initially only made these features available for Snapdragon (ARM based) Copilot+ PCs but with this update they are continuing to u lease the new AI features in Windows 11 to more devices. The expansion of Recall to Intel and AMD Copilot+ PCs marks a step forward in enhancing user experience and productivity on this next generarion of devices.

What do you think of Recall and Click-to-Do?

Posted on

Facilitator agent: Live AI notes in Teams meetings & chat

Microsoft announced at Ignite, the new Facilitator agent – an update to the AI notes in Teams that works inside your meetings and chat and is designed to enhance collaboration and streamline the way teams work. It works similar to the AI generated notes after a meeting, but this works live alongside you and all participants can see it working live in the meeting.

How Facilitator works in Teams Meetings

Facilitator will take real-time notes during Teams meetings (not currently adhoc meetings or Meet Now), enabling everyone to co-author and collaborate seamlessly. This allows meeting participants to focus and engage more deeply in meetings, while ensuring alignment before the meeting concludes.

To enable this feature and use it a meeting, organisers can toggle AI-generated notes setting on or off when setting up a meeting in the Teams calendar or enable it during the meeting via the Notes section in the meeting.

Once enabled, a notification appears in the meeting chat to inform all participants. This also activates meeting transcription, with a notification to users… During the meeting, participants can click on Notes to open a pane where the AI generated live notes are created every few minutes, organised by topics and follow-up tasks.

What is nice about this is that participants can edit the notes inline or assign tasks to users, with attributions indicating whether the content is AI-generated or user-edited making these Co authored notes by humans and AI!

After the meeting ends, notes continue to be accessible in the Recap tab and are stored in the OneDrive of the user who enabled real-time notes. These notes are contextual to the meeting transcript, ensuring relevance and accuracy.

Future Capabilities in Meetings

As the Facilitator agent gets developed futrther, Microsoft say that it will be able to take on more tasks to enhance meeting effectiveness. Soon, it will also manage meetings from end-to-end, including managing agendas, moderating discussions, and handling action items automatically or semi-automatically

In early 2025, the real-time note-taking experience will also expand to Microsoft Teams Rooms. Employees will be able to invite a Teams Room to a meeting, allowing all participants to see real-time notes however, they have joined the meeting. This feature will also be available for ad-hoc meetings, enabling in-office discussions to be captured seamlessly.

How Facilitator works in Teams Chats

As of now (November 2024), the Facilitator Agent creates and maintains up-to-date summaries of what it considers valuable information within Teams chats. This includes key decisions, action items, and open questions, helping groups stay focused, align faster, and resolve issues efficiently.

AI-generated notes are automatically enabled when creating a new chat. For existing chats, users can toggle it on via the Notes icon which is shown at the top right of the chat window as shown below.

When notes are enabled, a notification appears in the group chat to inform everyone that notes are being taken in real time.

To access the notes users simply click on the Notes icon in the top right corner of the chat to show a summary of the chat thread, organised by topics with corresponding decisions, action items, and unanswered questions.

These are continuously updated as the chat conversation progresses.

Availability and access

Facilitator is already in public preview now for desktop (Windows/Mac), web, and iOS/Android. To access the public preview of the new Facilitator agent, meeting hosts need a Microsoft 365 Copilot license.

Facilitator will only be available to users that have app permission policy for Microsoft apps set to “Allow all apps”. The Facilitator App will become available soon for Admins to see and manage in Teams admin center. For more information about app permission policies, see Manage app permission policies in Microsoft Teams – Microsoft Teams | Microsoft Learn

External users cannot access AI-generated notes

Let me know if you find this helpful

Posted on

Enhanced Data Protection and Governance coming SharePoint and Purview to remove AI adoption blockers.

This week at Microsoft Ignite 2024, Microsoft unveiled new features and controls for SharePoint and Purview, aimed at empowering IT teams and ensuring robust data protection and governance. These enhancements are part of Microsoft’s ongoing commitment to providing intelligent solutions that respect an organisations controls while driving productivity and efficiency. Oversharing of data is one of the most common causes of rogue data and poor data governance and one of the biggest blockers to wider AI adoption.

Microsoft offers two powerful tools to address this concern of oversharing: SharePoint Advanced Management for site management and content governance capabilities, and Microsoft Purview for security, compliance, and governance across data and files. Both have new capabilities and availability following announcements at Ignite in Chicago.

Advanced Management for SharePoint for Copilot.

To give IT teams even more control, Microsoft have said that SharePoint Advanced Management will be included at no additional charge for Copilot customers. There’s also need and updated features coming.

  • Restricted Content Discovery (RCD) to help identify and manage content that is restricted or sensitive, ensuring that such content is not overshared within the organisation. This works by allowing SharePoint Administrators to restrict specific SharePoint sites from participating in organisation wide search and Microsoft 365 Chat. Once configured, all content from the site will be hidden from tenant-wide search and Microsoft 365 Chat by default for all users in the tenant, even if a user has site access permissions. While child content will be hidden by default, users will still be able to search for content they have recently interacted with. This includes recently accessed and modified files, even if RCD is applied to the parent site. Searches originating from a site context will not be impacted.This will be available in Public Preview in December 2024 and Generally Available from March 2025
  • Restricted Access Control (RAC) feature allows administrators to control and restrict access to specific sites or content within SharePoint to helps in preventing unauthorised access and oversharing of sensitive information. New AI content governance controls and insights will be available in early 2025.
  • Deployment Blueprint is a new feature that will offers a structured approach to deploying Microsoft 365 Copilot while addressing the risks of information oversharing. It will include best practices, guidelines, and tools to ensure that sensitive information is protected during the deployment process.

SharePoint Advanced Management will be Generally available in Q1 2025 and will allow IT to better govern access and usage of Copilot and agents, including controls over which users can use Copilot and agents, along with visibility into agent status and life cycle.

SharePoint Advanced Management will be included as standard for organisations with Microsoft 365 Copilot licenses.

Purview

  • Data Loss Prevention – To provide addition protection, Microsoft Purview Data Loss Prevention (DLP) will soon be extended to support Microsoft 365 Copilot. DLP policies will be able to identify sensitive documents based on sensitivity labels and exclude processing for Copilot interactions in Microsoft 365 Copilot Business Chat. In preview from December.
  • Data Security Posture Management (DSPM) provides insights into the security posture of data within the organisation. It helps in identifying potential risks and vulnerabilities related to data oversharing and provides recommendations to mitigate these risks.
  • Risky AI Usage Detection is a new feature which can detect and alert admins about potentially risky usage of AI within the organization. It helps in preventing data leakage and unauthorised access by monitoring AI activities. This is now in public preview.
  • GenAI Risk Detections is another feature which focuses on detecting risks associated with the use of generative AI. It ensures that AI-generated content does not lead to oversharing of sensitive information or unauthorised access.

Measurement and Reporting with Copilot Analytics

To help IT and business leaders track adoption patterns and return on investment from the use of Copilot and agents, Microsoft is introducing Copilot Analytics.

This new feature includes out-of-the-box experiences to measure Copilot adoption and business impact, customisable reporting for deeper analysis.

The new Copilot Analytics. Microsoft Viva Insights will be included in Copilot at no additional charge as part of this new analytics suite.

How Microsoft partners can help

Cisilion, as your Copilot Jumpstart partner, will be incorporating these new features and controls into our guidance and briefing and expect Microsoft will rapidly be updating their official documentation and guidance.

The Copilot Pilot programme, entwines technical readiness with business guidance and comprehensive adoption and change management to ensures that your organisation receives the most up-to-date and comprehensive support in leveraging these advancements for optimal data protection and governance whilst putting these into practice for a smooth and measurable pilot.

Conclusion

These new controls and features are designed to provide IT teams with the tools they need to govern access, usage, and reporting while ensuring data protection and governance. Microsoft is committed to helping organisations leverage the power of AI to drive productivity, efficiency, and security.

You can read the official Microsoft Blogs here.

Posted on

Microsoft’s Copilot AI Agents enter Public Preview

TL;DR

Microsoft has introduced autonomous Copilot AI agents in public preview. These agents can learn, adapt, and make decisions, aiming to assist employees with various tasks and improve productivity. While AI has the potential to displace some jobs, it also creates new opportunities and enhances productivity.

Microsoft’s wave of Autonomous agents are here

Microsoft has unveiled new tools designed to help businesses create software agents powered by foundation models, referred to as autonomous Copilot AI agents. These agents are currently available in public preview.

Copilot is Microsoft’s generic term for all their AI-driven productivity workloads. Copilot is built upon the advanced GPT-4 series of large language models by OpenAI and offers a chatbot interface where users can input text, images, or audio prompts to receive responses tailored to their needs. Microsoft 365 Copilot also seamlessly integrates with Microsoft Office applications like Word, Teams, and Excel. It can generate documents, analyse extensive Excel spreadsheets, summarise meetings content, rewrite documents, create entire PowerPoint presentations and even reason over your inbox and company information you have access too….., and much, much more.

The next step in Microsoft 365 Copilot’s advancement is through what are termed AI-Agents, which are chat bots that can not only respond but can also perform a series of linked tasks (actions) based on user instructions. This new wave went into public preview this week at Microsoft Ignite in Chicago.

What are Microsoft 365 Copilot Agents?

This first stage of the next phase of evolution comes with Microsoft introducing a set of Microsoft 365 Copilot agents with predefined roles. These include:

  • Agents in SharePoint. These can be customers with a personalised name and certain behaviours, and can be shared across emails, meetings and chats, with users being able to ask the agents questions and getting real-time responses. These are grounded just on the SharePoint sites and files you specify. One created, employees can ask the agents questions about data across your files. These agents can even be shared or published in Teams for simple access.
  • The Employee Self-Service Agent in Microsoft 365 Copilot Business Chat (this currently in private preview), will be able to respond to specific HR and IT questions. It can retrieve employee benefits and even things like payroll info and holiday information, or request help from IT such as a new mouse, password reset etc.
  • The Facilitator agent (in public preview), works like a assistant in meetings and goes beyond the current AI notes that Teams Premium offers. It can take notes, curate actions and even pull up information or execute instructions such as “see if Bob is free and invite him to the meeting”. It will also be able to summarise the conversations based on the role of the participants.
  • The Interpreter agent (due in preview in early  2025) promises real-time interpretation in Teams meetings in up to nine languages. It will also be able to sample and then simulate their personal voice for a more inclusive experience as part of the translation, essentially using the sound of your voice in the language of the other participants. It was great to see this in action at ignite in a live demo!
  • The Project Manager agent, will be able to act and work like a PM with the ability to automate project management, from planning to execution using Microsoft (and later other) project tools like Planner.

For organisations that need more control or different templates to build on and use, Microsoft Copilot Studio provides a way to customise or create your own AI agent behaviour.

Agents in Copilot Studio

Agents built in Copilot Studio can operate independently, dynamically planning and learning from processes, adapting to changing conditions, and making decisions without the need for constant human intervention,These autonomous agents can be triggered by data changes, events, and other background tasks – and not just through chat.

Copilot Studio bundles many templates for common agent scenarios that can serve as the basis for a customised version. It will also shortly support voice-enabled agents, image uploading (for analysis by GPT-4o), and knowledge tuning with the added ability automatically add new sources of knowledge to help agents respond to questions.

Devs can use the Agent SDK to access services from Azure AI, Semantic Kernel, and Copilot Studio. There’s also an Azure AI Foundry (also launched at Ignite) integration that links Copilot Studio to facilitate connection to services like Azure AI Search and the Azure AI model catalog.

Finally, a public preview of agent builder in Power Apps was also announced at Ignite.

What about Responsible AI?

Sarah Bird, chief product officer for Responsible AI, wrote in a blog post this week that extra safety considerations arise with autonomous agents and that Microsoft is focused on ensuring that they behave and hand over to human before taking unexpected actions which can have big impacts and that extra guard rails and protections will be put in place.

The blog post talks about examples of such measures including the vital need for a human-in-the-loop check to make sure autonomous decision-making doesn’t do things it’s not expected too. Nothing demonstrates confidence in automation more than a human approval process.

Microsoft also suggest that anyone looking to get a sense of AI agents in a real role, can try out the  Linked In Hiring Assistant which is designed to help HR hiring teams speed up the process of dealing with the Admin involved in reviewing  job applications.

Key Benefits

The key Benefits these new adaptions to Copilot. Agents should bring to users and organisations includes:

  • Learning and Adaptation: The Copilot AI agents can learn from their environment and adapt to new information and tasks.
  • Decision-Making: These agents are capable of making decisions to assist users in their daily work.
  • Productivity Enhancement: The primary goal is to empower employees by reducing workload and improving efficiency in tasks such as managing meetings, emails, and creating presentations.
  • Automation of some tasks connected to regular and recurring inquiries or asks.

Human Impact – what about jobs!

The introduction of AI and automation, including Microsoft’s Copilot AI agents, has the potential to impact the roles of people in jobs.

  • Job Displacement: People naturally worry that AI has the potential to replace certain jobs, particularly those involving repetitive and manual tasks. According to a report by Goldman Sachs, AI could replace the equivalent of 300 million full-time jobs….. But.
  • Job Creation: On the other hand, AI also creates new job opportunities. It can lead to the emergence of new roles that require advanced technical skills and the ability to work with AI systems
  • Economic Impact: AI is expected to contribute significantly to global economic growth. McKinsey Global Institute estimates that AI could deliver additional global economic activity of around $13 trillion by 2030
  • Skill Demand: The demand for skills will shift towards more advanced and technical capabilities. Employees will need to upskill and reskill to stay relevant in the evolving job market. AI skills will be similar requirement to the “Internet skills” we saw on CVs in the 1990s!

Conclusion

Microsoft’s autonomous Copilot AI agents represent a significant step towards integrating advanced AI into everyday business operations. By enhancing productivity and reducing routine workload, these agents have the potential to transform how employees manage their tasks.

These will be in public preview very soon as these often take a few weeks to rollout across the globe.

Source: Conversation with Copilot, 22/11/2024
(1) How Will Artificial Intelligence Affect Jobs 2024-2030. https://www.nexford.edu/insights/how-will-ai-affect-jobs.

(3) The impact of AI on jobs – GOV.UK. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1023590/impact-of-ai-on-jobs.pdf.

Posted on

AI agents are transforming Customer interactions.

AI and human Agent

In our recent fireside chat, we delved into the transformative potential of AI agents across multiple industries. These business areas include customer service, IT support, and internal business support. The discussion, titled “Rise of the AI Agents,” brought together industry experts across several fields. These included transportation, public sector, legal, media, and executive search. The panel explored how AI is reshaping customer and consumer interactions and discussed enhancing efficiency and driving more inclusive interactions.

Introduction to AI Agents

We kicked off the session our fireside chat by setting the scene. We highlighted the traditional challenges faced in contact centers. These include long hold times and inefficient call transfers between chat bots and human agents. Here we agreed on these and but also the importance of not just jumping on “injecting ChatGPT” into workflows, but instead discussed the advent and value of generative AI and human-like conversation across chat and AI-Voice and how these rapid technology advancements have the potential to revolutionise these experiences.

AI agents, leveraging large language models like, are now capable of understanding context, handling a wide range of queries, and providing personalized responses and we are seeing Contact Center solutions such as Cisco Webex, starting to infuse this technology to assist end-to-end in the Human-to AI, Human-to-AI-to-Human, and Human-to-Human conversation.

AI agents leverage advanced technologies like large language models (LLMs) and machine learning to provide more dynamic and context-aware interactions. AI agents can understand and generate natural language. This ability allows them to handle a wider range of queries. They also provide more personalised responses. They can learn from data and feedback, improving their performance over time without needing manual updates.

AI agents can also integrate with various data sources and systems, enabling them to provide more comprehensive and accurate information.

  • Autonomous Agents can operate entirely independently, without human intervention. They can handle multi-step tasks, make decisions based on pre-programmed logic, and adapt to new situations using advanced AI techniques like reinforcement learning. These agents are ideal for environments where human input is minimal or impractical.
  • Semi-Autonomous Agents on the other hand, still involve a “human in the loop.” While they can perform many tasks independently, they require human input for certain decisions or actions. This hybrid approach combines the efficiency of automation with the oversight of human expertise, ensuring accuracy and reliability.

The example below is a recent marketing video from Cisco introducing their new AI agent in Webex Contact Centre.

AI Agent example in Cisco Webex Contact Centre

AI Agent – Use Cases

Through the discussion, the panel agreed on several key areas in which AI assisted agents could add value.

  • Customer Support: Investing (or extending existing platform) in AI agents can help multiple lines of business better and more efficiently handle routine customer inquiries, such as changing addresses, booking or changing appointments, and freeing up human agents to work on less trivial customer requests or more complex issues. 
  • Sales Assistance: Another area discussed, was where AI can assist human agents (for example in sales or customer service), by providing real-time information and suggestions during human customer interactions, improving the chances of successful sales conversations, such as overcoming objections or asking for more technical information about a product or service.
  • Customer Service and Complaints: helping agents improve their interaction with their customers, such as making agents aware of similar problems, outages or similar calls that led to successful outcomes or helping explain something better or in a different way to their customer.
  • Training and Development: AI can be used to train new agents by simulating customer interactions and providing feedback, helping them improve their skills more quickly. This can be used for onboarding fresh staff, running different customer scenarios or reviewing previous calls for improvement
  • Sentiment Analysis: Using AI to analyse customer sentiment during interactions, allowing agents to adjust their approach and improve customer satisfaction as well as flagging to supervisors early where interaction or training may be needed.

AI Agent Value and Applications

Driving efficiency and improving satisfaction

Darren Everden (London Borough of Hillingdon) shared his insights on how local authorities are looking at utilising AI to improve resident interactions. David emphasised the importance of channel shift and transformation in the public sector, driven by funding reductions and the need for more cost-effective solutions that also improve the resident experience and resolution rate. Darren highlighted the evolution of chatbots, which can now use natural language processing to understand and respond to resident queries more effectively. He also discussed the potential of integrating AI into voice channels, enhancing accessibility and providing a more natural interaction experience making it almost impossible to differentiate from human voice. Interactions are far more natural than ever, and this continues to evolve and improve with models such as ChatGPT-4o.

Inclusivity and Accessibility

Ken Dickie from Leathwaite Executive Search, discussed the role of AI in promoting and improving inclusivity and accessibility. He pointed out that AI agents are far better at being able to adapt to the needs of users with disabilities, such as dyslexia, by adjusting text spacing or providing audio responses something human only operated agents simply cannot easily do This real-time adaptability empowers individuals to engage with systems more effectively. Ken also mentioned the global reach of AI, enabling organisations to provide support in multiple languages, thus breaking down communication barriers.

Enhancing Agent Efficiency

Aidan Shanahan from Govia Thameslink Railway discussed the benefits of AI in assisting human agents. He discussed his view on where AI can provide real-time guidance and sentiment analysis, helping agents handle customer interactions more effectively. The panel here discussed the role AI as an human assistant (An Agent to the Agent) being particularly valuable in high-stress situations, such as handling complaints, where AI can suggest appropriate responses based on the customer’s tone. Aidan also highlighted the potential for AI to improve internal processes, such as IT support, by automating routine tasks and reducing response times, replacing laboreous processes with natural language requests.

Jas Bassi from Gately highlighted the potential applications of AI in the legal sector. While acknowledging the generational differences in adopting new technologies, Jas emphasized the need for a multi-channel approach that includes both human and AI interactions. He pointed out that AI can deliver efficiency gains in transactional activities, ensuring faster and more consistent service delivery. However, he also raised concerns about biases in AI training and the risk of deep fakes, underscoring the importance of maintaining a balance between automation and human oversight.

Low cost of entry and ease of Proof of Concept

Alex Taylor from Awin shared his experience with implementing AI agents internally at Awin. He mentioned that this is no longer about one off business cases and specific dictated expensive systems. He shared that he is seeing huge interest in the use of AI agents across various departments, such as InfoSec and marketing, in leveraging AI to not only automate and ease customer interactions but also going beyond this and automating processes and improving efficiency. He emphasised the importance of extending this value by connecting backend systems (which also involved in many cases minimising diserpate vendrs) and ensuring they are “compatible” to maximize their effectiveness with automation and semi-automatic interactions. He realised examples, of automatically logging tickets, providing simple answers to issues and even liaising with other systems or processes.

Finally Alex and Ken agreed that the bar to entry is much lower, with a similar approach,. bring able to serve multiple departments, handle thousands of enquiries and not only reduce the cost, but truely delivery faster, more inclusive and international support even for organisations that don’t have global offices.

Conclusion: The Value and Opportunities of AI Agents

Our fireside chat concluded that there were several key value points when it comes to the use and exploration of AI agents across customer and employee focused formal contact centers but also across more adhoc and internal communications within and across business, from website chat to internal IT support.

  • Enhanced Customer Interaction: AI agents can provide more efficient and personalized customer service, reducing wait times and improving satisfaction
  • Cost-Effective Solutions: The low cost of entry and easy of deployment (compared to the previous laborious process of programming conversational paths), enables organisations to handle a higher volume of interactions without significantly increasing costs, making it a viable option for sectors with budget constraints and most importantly without a huge development and support burden.
  • Inclusivity and Accessibility: AI agents can adapt to the needs of diverse user groups, promoting inclusivity and breaking down language barriers both with end customers and with human-agents.
  • Supporting the Human Agents: AI assists can act as a huge support for human agents by providing real-time guidance, sentiment analysis, and automating routine tasks, enhancing overall efficiency and can even help handle delicate situations, detect agent stress and suggest rest-bites and training to supervisors based on AI assisted analysis.
  • Internal Process Optimisation: Used effectively, this can extend way beyond the conversation, streamlining internal workflows, reducing response times and improving productivity across various departments.

Missed the fireside chat? Catch up on demand here

Posted on

Creating a Copilot Agent from a SharePoint Library

The new Agent Builder in SharePoint is designed to help people use and share Copilot Agents to query sibsets of data within your organistion using a simple click, point, create and tweak approach. Out of the box every SharePoint site (assuming you have a Copilot license) brings a Copilot sidebar allowing you to ask questions about the content, but you can also replace this with a custom Copilot Agent which we will walk through here.

The goal is to enable business users to easily empower their employees use Copilot to reason over specific information sources or across discrete repositories. Microsoft provide a handful of “use cases” as why a Copilot agent might be useful and what’s great is that “anyone” can create one!

Image – Microsoft Copilot Adoption Hub

Once created and tested, these custom Copilot Agents can be easily shared via a simple hyperlink that can be embedded in SharePoint pages or used in Teams.

In this how to blog, I walk you through the setup and customisation of a Copilot Agent using Agent Builder in SharePoint, customising of the agent, and sharing of the agent. Free to follow along and create your own agent.

Copilot agents are specialised AI assistants designed to enhance the capabilities of Microsoft 365 Copilot by connecting to your organisation’s knowledge and data sources. They are custom tools embedded in Copilot Extensions, providing additional functionalities tailored to specific needs. In SharePoint, Copilot agents are natural language AI assistants that give trusted, precise answers and insights. Agents are expert systems that operate autonomously on behalf of a process or company.

Building your First Copilot Agent

Step 1 – Choose your starting point.

First, you need to navigate to a SharePoint site, library or document library you want to create an “agent” from. You will of couse need to have access to that Library and also need a Microsoft 365 Copilot license to create the agent.

From here, you can select the three dots and choose “Create a Copilot agent“

Step 2 – Click and you are done!

Done (well – you will probably want to customise it and test it), but once you do this, your Copilot Agent is created for you. Click “Edit” to make changes, such as change the name, and then of course test it out.

Step 3 – Edit and Customise

Here I have clicked “Edit” to take me to the customisation pages. From here you can toggle across different options to customise your Copilot agent.

The customisation pages are split into three sections – Idenitity, Sources and Bebaviour – each of these allow you to tweak the way the agent works. There’s also the ability to edit for advanced customisation through Copilot Studio but this feature is not available at time of writing…

In the Identity Section – you can change the name, icon and description (who the agent introduces itself to the user)

In the Sources Section – you can modify the sources that the Copilot Agent uses. You can add additonal SharePoint sites, individual files or extenal sources such as websites.

In most cases, I suspect you will want to use a single library or a discrete set of files, but you can add up to 20 different information sources. These 20 information sources can be mean sites, libraries, folders, or documents. What’s more, you can have a combination of these as long as the total is 20 sources – for example, you could add 20 sites or 20 documents, or 3 sites, 5 document libraries, 2 libraries and 10 descrete files as long the total sources totals 20.

Note: You of course need to ensure that the intended users of the agent have access to the sources your specify as agents run under the security context of the user using the agent.

In the Behaviour Section, you can customise the welcome message which will help your users to understand the purpose of this Copilot agent and can also edit or change the starter prompts to help users get some tips on some of the things the agent can do for them. You can also give the agent specific instructions on how it should respond and behave based on the user input.

As you update the behviour, you will see the changes in real-time.

Testing your Copilot Agent

Once you are ready, you can test your agent, simply writing a prompt in the chat dialog as you would with any other Copilot – feel free to try one of the templates or create your own.

Be sure to test a few things, you might find you need to update the user instructions and review the sources before you share it with other people to test further.

Once you are happy with your agent, click save. The agent is saved a “file” with a .copilot extension in the root of the SharePoint folder you started creating your agent in.

Using your Copilot agent

Once saved, your new Copilot Agent launches automatically for any user accessing the SharePoint library that has a Microsoft 365 Copilot license. This replaces the default copilot interface that opens when you visit a SharePoint library.

Sharing your Copilot agent

Since the Agent is encapsulated as a manifest “.copilot” file, you can simply share the file like you would any other file, or click the three dots and select share.

Once shared, they click on the file and open it and it displays like a standalone app or can of course access it from the SharePoint library directly.

[Current] Limitations

  1. Currently Custom agents do not appear on the main Copilot Business Chat pages, though this is coming I beleive. On the FAQ on Microsoft’s support page it clearly states that “You can access a Copilot agent from a SharePoint site, page, or document library. You can also use it in Teams if added. We plan to make it available across Microsoft 365, including Microsoft Copilot.” https://support.microsoft.com/en-us/office/get-started-with-copilot-agents-in-sharepoint-69e2faf9-2c1e-4baa-8305-23e625021bcf.
  2. Advanced editing with Copilot Studio is not currently available, but is also coming soon.
  3. It’s not possible to “hide” the .copilot file (that I can see anyway), so make sure to change permissions on the file.

Let me know how you get on….

Posted on

Copilot Vision: A New Era of AI Assistance or a step too far?

Microsoft is about to add more capabilities to the consumber version of Copilot including a new way we interact with it Copilot through its latest feature, Copilot Vision.

Is this a privacy step to far? or is this simply leveraging the power of vision to bring new experience and assistance to users.

Designed to enhance the capabilities of Microsoft’s Copilot AI assistant by allowing it to see and understand the same content as the the user is seeing on scene is said to bring a new dimension to how we work with AI tools.

Copilot Vision – image (c) Microsoft

What is Copilot Vision?

Copilot Vision is an extension of Microsoft’s Copilot AI assistant that enables it to visually perceive the content on your screen. Whether you’re browsing websites, reading documents, or viewing images, Copilot Vision can interpret and respond to the visual data, making it a more intuitive and helpful assistant.

Examples of Copilot Vision in Action

  • Travel Planning: Instead of manually searching for travel options, Copilot Vision can provide recommendations and answer questions based on the travel websites you’re viewing.
  • Recipe Adjustments: If you’re looking at a recipe online and want to make substitutions, Copilot Vision can suggest alternatives and cooking tips without needing to switch tabs or open a new search.
  • Document Assistance: While working on a document, Copilot Vision can offer suggestions, corrections, and additional information relevant to the content you’re editing.

All this happens without you having to copy and paste stuff to the Copilot chat interface so it’s really like having eyes on what you are doing (once turned on of course). It’s like having your assistant working and viewing with you so it can see exactly what you see on your screen, whether it’s a website or a document.

Copilot Vision.

That’s what Copilot Vision does. It can read both typed and handwritten text displayed on your screen, and it responds to what it sees, offering answers to your questions and suggesting next steps.

Privacy Consideration

Microsoft has emphased that privacy is a top priority with Copilot Vision. The data processed by Copilot Vision is not saved or used beyond the current session (like clearing the cache). Microsoft Copilot Vision will initially also be limited to certain popular websites that meet Microsoft’s security standards, ensuring a safe and secure user experience. It also won’t work on sites that contain sensitive data such as banking sites.

Remember, this is something you can choose to use or not, so you can turn it off!

Rollout Timeline

Microsoft say that after a successful trial period with a select group of users in the Copilot Labs experiment hub, Copilot Vision is “now ready” for a broader rollout.

The feature will soon be integrated into the Microsoft Edge browser, accessible via a screen-like icon. This phased rollout will allow Microsoft to gather more feedback and ensure a smooth user experience.

Posted on

Recall “Recalled” Again: Microsoft’s Copilot+PC Flagship Feature Faces Further Delays

Recall Recalled again feature image

In a move that has surprised few, Microsoft has once again delayed the rollout of its controversial Recall feature for Copilot AI PCs. Initially planned for a June release to coincide with the new Copilot+PCs launch, Recall was then postponed to October while Microsoft addressed initial concerns around privacy and security.

This week however, Microsoft has yet again delayed this again with testing for Windows Insiders coming (so we are told) in December, which unfortunately falls after Ignite.

Microsoft Recall….

Security Concerns and Refinements

Recall’s primary value is to create a timeline of screenshots that users can scroll through and search. However, early testing revealed by security researchers discovered that the core database storing these screenshots and tagging was not encrypted, posing a massive security risk.

Microsoft have since addressed this by fully encrypting the database and requiring Windows Hello authentication for access.

Microsoft have also confirmed that Recall will now be an opt-in feature, allowing users to completely uninstall it if they choose.

Microsoft’s Cautious Approach

Brandon LeBlanc, senior product manager of Windows, enforced Microsoft’s commitment to delivering a secure and trusted experience with Recall.

Microsoft need to get this right in order to maintain trust with its customer base. He stated that the additional time is necessary to refine the feature before previewing it with Windows Insiders. Despite these assurances, social media shows huge skepticism about whether Microsoft will meet the new December deadline and even if they might scrap the feature all together. This will be a shame, as the value around it looks. Promising and is really ( currently) the one killer reason consumers were looking at when looking at investing in Copilot+ PCs outside of the huge battery life that these Qualcomm Snapdragon Powered devices deliver.

The verge covered this in an exclusive interview

Wider AI privacy concerns

The repeated delays and security issues surrounding Recall highlight broader concerns within the AI industry.

There is a growing perception that companies are rushing to release new features without fully considering the potential consequences.

Microsoft’s cautious approach with Recall is a step in the right direction, but it also underscores the need for more rigorous testing and security measures in AI development.

Will Recall still be exclusive to Copilot+ PCs?

That’s a good question.

When Microsoft announced the Copilot+PC back in June, Recall was the flagship feature and it was unique to the device’s (and kinda stole the show).

Since then AMD and Intel have released their new AI PC chipsets offering similar NPU performance to the Snapdragon chips in Copilot+ PCs like Surface Pro 11 and Surface Laptop 7.

We now have NPU turbocharged PCs with Snapdragon® X Series, AMD Ryzen™ AI 300 Series and Intel® Core™ Ultra 200V Series devices after all.

We don’t know if this will remain an exclusive (I don’t see why it would) and if all the “exclusive AI features” that are part of Windows 11 24H2 will soon be lit up in any decide with a dedicated NPU.

From what I can… It will be supported… But some features are limited to Snapdragon, so we will have to wait and see….

Coming soon then… Or will it?

While the future of Recall still remains uncertain, Microsoft’s efforts to address security concerns and refine the feature are commendable, I just hope they haven’t missed the boat. We’ve already seen Apple quietly move forward with Apple Intelligence (clever) and it’s now embedded in MacOS. Microsoft need to move quick and innovate here to regain confidence and innovative is their mission to empower every person on the planet to achieve more (with their technology)!

Consumers, IT professionals, industry experts and social media will be keenly observing whether the Recall gets the release and value reputation it received back in June, with  a secure and functional version of Recall to define what AI can really do in Windows.

I hope succeeds and brings life to the new AI PCs and Copilot+PCs or of it quietly gets canceled as skeptics seem to think…

Posted on

Cisco Webex One 2024 Keynote: “Experiences Amplified”

Webex One 24 Logo

The Webex One 24 opening keynote was a showcase of Cisco’s commitment to revolutionising the way organisations collaborate and connect. It highlighted several new announcements, renewed partnerships and AI innovation across their portfolio which will continue to innovate the “future of work”.

This blog summarises my key highlights from Cisco’s fifth annual Webex One event. This year’s theme focussed on the transformative power of AI and human connection. Senior Cisco execs, including Aruna Ravichandran and Jeetu Patel, opened by discussing the latest AI innovations and their impact on customer and employee experiences and of course used the event to showcase new technologies and products from Cisco as well as new and extended partnerships with key vendors including Apple and Microsoft.

Cisco emphasised the integration of AI into (all) their products, the importance of secure and smart workplaces, and the future of seamless collaboration. They also introduced new technologies like the autonomous Webex AI agent and Cisco spatial meetings for Apple Vision Pro, showcasing their commitment to enhancing productivity and connectivity.

“Don’t worry about AI taking away your jobs, but worry about people who use AI effectively” | Jitu Patel

Aruna Ravichandran, Cisco’s Senior Vice President emphasised the importance of human connection and creativity in the transformative world of AI, highlighting the event’s focus on groundbreaking AI innovations and customer experience.

Jeetu Patel, Cisco’s Executive Vice President, talked about the “seismic shift in AI“, noting that while AI has brought significant momentum and pockets of impactful change, for the most, our lives have not yet materially changed. He said that he sees a dramatic change in the next decade, emphasizing the potential of agent-based technologies to redefine and reshape jobs and workflows fundamentally.

Let’s look at a few of these in more detail.

Cisco’s AI Strategy

Cisco’s AI strategy is focussed on integrating AI natively into their products, providing AI infrastructure, and ensuring AI security. Jeetu Patel talked passionately about the importance of AI-ready data centers, modern networks and AI aware security as well as what he called “future-proof workplaces”, and resilience in operations, driven by AI and data. Cisco unveiled new AI-driven features across their Webex collaboration suite, designed to make virtual meetings more intuitive and productive. These enhancements included real-time language translation, advanced noise cancellation, and AI-powered meeting summaries, ensuring that every participant can stay engaged and informed as well as work with Apple and Microsoft integration and connectivity.

Building Future Proofed Workspaces

Cisco talked about the importance of seamless integration across various platforms. They said they are introducing new APIs and partnerships that will better enable Webex and their meeting spaces technologies to integrate more effortlessly with other tools and applications, providing a unified and streamlined user experience, including with Microsoft Teams. Cisco aims to continue to raise the standard of what the workplace with secure campus and branch networking, smart building technology, workplace security, and seamless collaboration should look like something they see themselves having a unique industry advantage of.

“Our goal is to make every meeting as productive and engaging as possible, no matter where you are in the world.” | Jeetu Patel | Cisco VP

Cisco said the focus is on creating productive, automated, and secure environments for employees to work from anywhere. Cisco also showcases its advancements in spatial meetings, emphasizing the importance of human connection in AI-driven interactions – with a differentiated focus on creating immersive and engaging meeting experiences that feel as intimate and effective as in-person interactions.

Autonomous Agents in Webex Contact Centre

Cisco also introduced their upcoming autonomous Webex AI agent, designed to enhance self-service in contact centers. This AI agent combines conversational intelligence with generative AI and integrates with back-office systems to deliver personalized outcomes, reducing the need for human agents. The demo showed some advanced autonomous (AI Agents) enhancing customer self-service – combining conversational AI with generative AI along with integrates with back-office systems and processes like HR and Finance.

Extended Partnerships

Cisco discussed extended partnerships with both Apple and Microsoft, emphasizing their collaborative efforts to enhance technology and user experience and to meet customers where they are. These partnerships emphasize Cisco’s commitment to interoperability and enhancing user experiences across different (in some cases, competitive) platforms.

Cisco said that their Cisco Room devices for Microsoft Teams have become the fastest-growing Microsoft Teams Room solution in the world with over 3,000 customers now leveraging Cisco technology to power their Microsoft Teams investment.

Cisco’s long-term relationship with Apple was discussed and references made to Cisco technology being integral to Apple’s product development. The keynote highlighted the collaboration between the two including the development of the Apple Vision Pro, which integrates Cisco’s Webex for immersive 3D meetings

An AI powered Sustainable Future

High on the agenda was Cisco’s commitment to sustainability which was a key theme through the keynote. Cisco introduced new features aimed at reducing the environmental impact of virtual meetings, such as energy-efficient data centres and tools to measure and offset carbon footprints.

Did you attend Webex One or watch it remotely, what did you find of interest and what did you hope you’d see but didn’t?

Posted on

Understanding the EU AI Act and Microsoft’s Commitment to Compliance..

TL:DR

The EU AI Act, effective from August 2024, regulates AI systems within theEU, categorizing them into prohibited, high-risk, and limited or minimal risk. Microsoft is committed to compliance through tools like Purview Compliance Manager, continuous monitoring, data privacy measures, bias mitigation, and transparency initiatives.

Understanding the EU AI Act

The EU AI Act, effective from August 2024, is a comprehensive regulation designed to govern the development, deployment, and use of AI systems within the European Union. It categorises AI systems into three risk levels: prohibited, high-risk, and limited or minimal risk.

  • Prohibited AI Systems: These are AI applications that pose unacceptable risks, such as those that manipulate human behavior or exploit vulnerabilities of specific groups. Organisations must decommission such systems by February 2025.
  • High-Risk AI Systems: These include applications used in biometric identification, critical infrastructure, education, and law enforcement. High-risk systems are permitted but must undergo stringent compliance checks, including conformity assessments by accredited third parties or through self-assessment.
  • Limited or Minimal Risk AI Systems: These cover applications like chatbots and AI-generated content, which are generally permitted but require transparency and informed consent from users.

Key Challenges in AI Compliance

Organisations will likely face several challenges in navigating AI compliance:

  • Ensuring Continuous Compliance: AI regulations are dynamic, and organisations must continuously update their systems to remain compliant. This involves tracking regulatory changes and implementing necessary updates promptly.
  • Managing Data and Privacy: AI systems often process vast amounts of data, including sensitive information. Ensuring that AI applications do not inadvertently access or misuse sensitive data is a significant concern.
  • Addressing Bias and Inaccuracy: AI systems must be trained on diverse and representative data sets to avoid biases. Inaccurate or biased AI outputs can lead to ethical and legal issues.
  • Maintaining Transparency: Organisations must ensure that their AI systems operate transparently, providing clear information on how data is used and decisions are made.

Microsoft’s Commitment to AI Compliance

Microsoft is at the forefront of ensuring AI compliance and ethical use. Here are some key initiatives and tools that demonstrate Microsoft’s commitment:

  • Purview Compliance Manager: Part of the Microsoft Purview family, this tool helps organizations manage compliance with various regulations, including the EU AI Act. It offers templates for different regulatory requirements, enabling organizations to streamline their compliance processes.
  • Continuous Monitoring and Updates: Microsoft ensures that its AI applications, such as Microsoft 365 Copilot, are continuously monitored and updated to comply with evolving regulations. This proactive approach helps organisations stay ahead of compliance requirements.
  • Data Privacy and Security: Microsoft emphasizes robust data privacy and security measures. AI applications are designed to prevent unauthorised access to sensitive data, and tools like Data Loss Prevention (DLP) policies help safeguard information.
  • Bias Mitigation: Microsoft is committed to reducing bias in AI systems. By using diverse data sets and implementing rigorous testing protocols, Microsoft aims to ensure that its AI applications provide fair and accurate results.
  • Transparency and Accountability: Microsoft promotes transparency in AI operations. Users are informed about how their data is used, and AI systems are designed to provide clear explanations for their decisions.

Conclusion

The EU AI Act represents a significant step towards ensuring the ethical and responsible use of AI. As organisations navigate this complex regulatory landscape, Microsoft’s tools and initiatives provide valuable support in achieving compliance. By prioritising continuous monitoring, data privacy, bias mitigation, and transparency, Microsoft is helping organisations harness the power of AI while adhering to the highest standards of ethical conduct.

What organisations can do

As we move forward in this AI-driven future, it’s crucial for every organisation large and small, private and public to stay informed and proactive about regulatory compliance in this space.

If you are invested in Microsoft Technology, be that Microsoft 365 or Azure, ensure to further explore Microsoft’s extensive and comprehensive suite of tools and resources to ensure your organisation and AI connected systems are not only compliant but also ethical and transparent.

1. You can check out Microsoft AI Compliance Hub

2. Check out their YouTube video https://youtu.be/briI9LdiZuc

3. Speak to your Microsoft Partner.

Posted on

Hands-on with Copilot Voice: An almost human conversation

Copilot Voice Cover


The recent Copilot update is a Game-Changer in AI Voice Technology. In the recent announcements, Microsoft unveiled a new version of its Copilot app for iPhone and Android. The update brings a fresh look and new features. It also includes an impressive voice mode that rivals OpenAI’s ChatGPT Advanced Voice – especially since Microsoft make this available for free – yes free

I have tested both recently. I can confidently say that the new Copilot is a significant upgrade. What’s more, it is totally free to use. This is best read while/after you have watched my hands-on video below.

Hands on with Copilot Voice

User-Friendly Interface and Enhanced Voice Mode

The updated Copilot app boasts a more “consumer-friendly” interface. I do wish they would bring some of the advanced customisations back. The standout feature in this update is most definitely the new voice mode, which on first look (a few app updates before it worked), I thought would be a bit of a fad – but it is absolutely brilliant.

Voice mode offers speech-to-speech functionality, allowing for more natural and engaging conversations. While it may not interrupt as fluidly as OpenAI’s offering (though it’s still in early stages), it feels more casual and less stilted, making interactions feel more like chatting with a friend.

A Conversation That Feels Real

During my testing, I found myself deeply starting to actually forget that I was talking to an AI as the conversation felt natural and real (there was the odd delay. In my hands-on example (see the video below), I participated in a discussion. We talked about “if and when AI could ever become self-aware”. We also considered what the implications might be. Unlike a text-based discussion, this level of engagement goes to show just how fast and how rapid the advancement of natural conversation is becoming.

Copilot appears to adapt its vocal tones and pace during conversations. It emphasizes certain words as we speak.

Perhaps the biggest (pleasant) surprise I found was how Copilot adapted to use slang terms the more I used them too. If I swore or spoke more loudly, it also seemed to detect the change in my tone and adjust its output. I’ll be testing this more to see just how far it can go.

Spoiler: I did find the occasional limitation as the conversation continued, such as occasional delays when I interrupted it and seconds of silence.

Customisation and Accessibility

Copilot offers four voice options: Grove, Canyon, Wave, and Meadow. Unlike ChatGPT, you can modify the speed and tone of these voices, making them sound more natural and suited to your preferences. This feature, combined with the app’s inclination to use slang and short-hand words, makes it easy to forget you’re interacting with a machine. I’m not a fan of all the voices though and they are not currently that localised – with most very American (which is fine for now).

Gemini Live (yes, all the chat bots are discovering their voice) currently gives users a choice of 10, but Microsoft say more voice options will be coming “soon”.

What I also like is that you can customise the speed at which each of the voices speaks. Personally, I find the standard setting is too slow and find that a speed of 1.1x sounds most natural. I also discovered that you can also ask Copilot to speak differently by explaining how you want it to sound – for example, applying a slightly different accent, changing its tone of voice or to be more empathetic but I’d like to think eventually Copilot will do this natively without me asking (after all it’s unlikely you’d ask a human to speak in a different tone!).

Copilot Voice is free

One of the most significant advantages of Copilot is that it’s free to use. Today, OpenAI’s ChatGPT Advanced Voice feature currently requires a $20 monthly subscription, whilst Microsoft makes this feature available to all Copilot users, regardless of their subscription status.

Conclusion

Copilot is now under the leadership of Mustafa Suleyman [Microsoft CEO for AI]. It seems poised to make a significant impact in the AI voice technology market. It builds on its partner OpenAI. Its user-friendly design, natural voice interactions, and accessibility make it a strong competitor against other AI voice models.

The best thing – this is totally free

Try this out. Let me know how in depth you feel during a conversation is and can be with Copilot. How “close” do you think this is in becoming a natural, almost human conversation.

Posted on

Expanded Controls for Managing Transcription and Copilot in Teams Meetings

Microsoft is rolling out new controls to better manage the availability of Copilot in Teams meetings. These expanded controls offer greater flexibility and customisation for managing Copilot and transcription settings in meetings. This ensures it aligns with the specific needs and policies of organisations and meeting organisers.

IT admins can find the new setting in the Teams Admin Center and have a few options when it comes to the control Scope with options to set  Copilot in meetings to be ‘Off’ at different levels including “per Tenant” Per User” and “Both”

Meeting Organisers also get controls which are accessible in the meeting options under “Allow Copilot“. From here there are three choices around how Copilot can be used in “their” meetings:

  •   Only during the meeting: Copilot is available only while the meeting is in progress.
  •   During and after the meeting: Copilot is available during and post-meeting.
  •   Off: Completely disables Copilot for the meeting.

Meeting organisers can choose who has access to the recording. They can also decide who can see the meeting transcript after a meeting. This adds a huge amount of control and privacy.

Image (c) Microsoft

Auto meeting Summaries

After a meeting ends, users can open the meeting event in Microsoft Outlook. This allows them to view a summary of everything that happened in the meeting. Here you will see a summary. It includes a link to the recap page, transcript, recording, notes, shared files, whiteboard, and meeting details.

Auto meeting Summarisation in Outlook.

Users with a Copilot License or Teams Premium license will also see highlights. These include the number of speakers. They will see the number of times you were mentioned. Additionally, they will see the number of AI-generated tasks.

Rollout Timeline

Both features are being rolled out this month (October 2024).

Posted on

Click-to-Do and new “AI Search” coming to Copilot+ PCs

Microsoft (off the back of Windows 11 24H2 release this week) have unveiled a series of updates to their AI infused Copilot+ PCs, aimed at enhancing the user experience with innovative features.

Copilot+PCs are a new category of PCs released this year by leading OEMs including and debuted by Microsoft and are equipped with NPUs. These devices come in Intel, AMD, and Qualcomm-powered Windows on Arm configurations. Microsoft introduced several new features for both types of these advanced PCs. The newly launched Windows 11 2024 Update (version 24H2) brings new functionalities for all Windows 11 users, with some exclusive enhancements specifically for Copilot+ PC owners.

These updates further demonstrates Microsoft’s commitment to baking AI assistance into more and more manual or semi manual tasks, and are designed to make everyday tasks more intuitive and efficient, leveraging the power of AI to transform how we interact with our new shiny devices.

New AI experiences in Copilot Plus PCs

These new features are summarised in an extensive Copilot / Windows Blog but the key ones I have summarised below.

Enhanced Search: Find stuff faster and easier.

One of the standout features in this update is the new Enhanced Search. Here, Microsoft has integrated AI-driven improvements deep into Windows search, making it significantly more powerful and user-friendly.

With these enhancements, users can now find files, images, and emails with ease, even if they don’t remember the exact names. Better than a simple search index (you know, that thing that used to grind your PC to a halt), this AI search understands context and can retrieve relevant results based on partial information or related keywords.

Image (c) Microsoft.

For instance, if you’re looking for a presentation you worked on last month but can’t recall the title, simply typing in related terms like “presentation” and “last month” will bring up the correct file. This feature is huge change for how we find and retrieve information and is more human connected in terms of how we think and ask. Plus, if you are anything like me, it should reduce the time spent searching for documents and allowing users to focus on their work.

“AI-powered search makes it dramatically easier to find virtually anything,” says Yusuf Mehdi, executive vice president and consumer chief marketing officer at Microsoft. “You no longer need to remember file names and document locations, nor even specific names of words. Windows will better understand your intent and match the right document, image, file, or email”.

Yusuf Mehdi |Executive VP and CMO | Microsoft.

For images this will work extremely well as you won’t need to know the file name and can search for pictures using words, even if the search word isn’t part of the file name!

The improved Windows search will first show up in File Explorer on Copilot Plus PCs from November.

This improved search will also be available “in the coming months” in the main Windows search interface and through the search box that appears in the Settings interface. You can type things like “duplicate my screen” into the Settings search box and it will help you find the right settings.

Click to Do: Interacting directly with your screen.

This feature really caught my attention and is definitely one I think I will use.

Click to Do will allows users to interact directly with images and text on their screen to perform quick actions. As an example you might be viewing a webpage or a pdf and see a phone number. You’ll be able to click on it to initiate a call, or clicking on an address to open it in your maps application. This feature streamlines workflows by reducing the number of steps needed to complete tasks.

Image (c) Microsoft

Click to Do works by understanding everything you’ve seen on your screen and enabling useful shortcuts to actions to help you more quickly search, learn, edit, shop, or act on those items… It works on any windows, document, image, or even video”

Yusuf Mehdi |Executive VP and CMO | Microsoft.

As an example, you could use Click to Do to perform a visual search on an item that appears in a YouTube video you’re watching or a page you are viewing in a browser. Click to Do is also context-aware, assisting with text-related actions like rewriting, summarising documents, explaining text, and sending emails.

Microsoft have said Click to Do will begin testing with Windows Insiders on Copilot Plus PCs in October, with a gradual rollout planned for November.

The previously announced Recall feature will also be available to Windows Insiders in October on Qualcomm-powered devices, before being rolled out to Windows Insiders on Intel- or AMD-powered Copilot Plus PCs in November.

I think this could be a real time saver for me and for anyone who does a lot of research, works with high volumes of information and has lots of open windows on their screen(s). By enabling direct interaction with on-screen content, it should help minimise interruptions and keeps the focus on what your are doing without having to open other tools.

Generative AI in Paint: Bringing modern tech to a staple app.

Microsoft Paint is also getting another significant upgrade (which has been in testing with Windows Insiders for a few months) with the introduction of new generative AI tools. These new tools ncluding generative fill, image generation (using Designer) and generative erase, that bring capabilities similar to those found in advanced photo editing software like Photoshop. Layers was also introduced to Paint earlier this year.

Copilot UI : The human touch.

Finally, Microsoft are revamping the general Copilot experience. This is not just about adding new features (though there are some) but are more about a total overhaul and part of their vision to make AI a more human-centric experience.

Their goal is to differentiate Copilot from other AI tools like ChatGPT and Gemin, by focusing on usability and user satisfaction. This approach ensures that the technology adapts to the user’s needs, rather than the other way around.

I covered this in more detail in another blog post below.

Posted on

Microsoft release huge Copilot (consumer) update…

In a blog post yesterday, Microsoft unveiled a huge overhaul. They also introduced new features which are rolling out “now” to the consumer version of Microsoft Copilot. These updates set out to help them differentiate Copilot from ChatGPT. They aim to bring a more human-centric approach to the user experience.

These experiences are coming to Apple iOS, Android, the Copilot web experience at copilot.microsoft.com and in Copilot in Windows. Microsoft have also announced that Copilot is coming to WhatsApp. It will “help users there experience natural and engaging interactions with Copilot”. Interesting.

This is not available for me yet (I have a Copilot Pro license). Microsoft says that until this rolls out, users will see a message like below. This message pre-advertises the new Copilot experience that is coming “soon”.

In this blog, I’ll cover the key changes, the reasons for change and some of the coolest new features. I’ll also share my take on this and what they mean for users.

New Microsoft Copilot Video

Oh yes, there’s also a Microsoft “sizzle” video below if you don’t want to read my ramblings:

The Human Side of Gen AI?

At the heart of Microsoft’s vision is the belief that technology should enhance human well-being and support our unique qualities. Microsoft say that the revamped Copilot has been designed with this philosophy in mind. It ensures that every interaction is intuitive, personalised, and ultimately beneficial to the user.

Now, this is kind of the part of Copilot that was missing in my view. Other Gen AI tools now do this. They are essentially giving Copilot (for consumer – not Microsoft 365 Copilot) to “get to know you” – this is how they describe it:

“Copilot will be there for you, in your corner, by your side and always strongly aligned with your interests. It understands the context of your life, while safeguarding your privacy, data and security, remembering the details that are most helpful in any situation. It gives you access to a universe of knowledge, simplifying and decluttering the daily barrage of information, and offering support and encouragement when you want it. …..Over time it’ll adapt to your mannerisms and develop capabilities built around your preferences and needs. We are not creating a static tool so much as establishing a dynamic, emergent and evolving interaction. It will provide you with unwavering support to help you show up the way you really want in your everyday life, a new means of facilitating human connections and accomplishments alike.”
Mustafa Suleyman, Executive Vice President and CEO of Microsoft AI

New Copilot Features

The new Copilot experience will be far more than an UI facelift when it arrives. (I don’t have it yet.) It will come with brand new features that promise to re-innovate the ways in which we can use Copilot in our daily lives. The new features include:

  • Copilot Voice: This allows users to interact with their devices using natural language, making technology more accessible and user-friendly. This means you will be able ro engage in natural conversations with Copilot. The current method is more just speech to text and doesn’t really feel like a conversation. This is rolling out “now” for initially available in English in Australia, Canada, New Zealand, the United Kingdom and the US with more regions and languages coming soon.
  • Copilot Daily: Delivered as “cards”, this will provide personalised insights and recommendations to help users manage their day-to-day tasks more efficiently. Microsoft say this content will only be served by trusted news sources – the cite: Reuters, Axel Springer, Hearst Magazines, USA TODAY Network and Financial Times, with more “local sources” coming soon. As Copilot gets to know you, it will bring personalsed feeds and also allow users to have controls over what is served up. This starts rolling out now in the US and UK with other countries following soon.
  • Copilot Vision: which looks so cool, will leverage Microsoft’s vision and seeing AI tools to offer enhanced visual recognition capabilities, making it easier to organise and find information. Microsoft say that this work on all websites and documents. They have taken steps to put “boundaries” on the types of websites Copilot Vision can engage. It also won’t work on pay-walled content initially. It won’t work on sensitive content initially either. This is to protect “users’ and creators’ interests” and copyright. This will be a Copilot Pro Feature and will roll out to US customers first…
  • Think Deeper: This is similar to Deep Search in Bing, whereby Copilot will be able to reason through more complex questions. As such it will take longer to respond and may ask for clarifications before responding. This is really designed to allow Copilot to go beyond basic responses. It will help with more in-depth discussions. It will also aid in challenging questions and research. This is in experimental phase and will more details will come.

Wrap up

The new and enhanced Copilot user experience is all about making Copilot easier to use. It feels more natural (less techy). It brings some new capabilities and personalisation. This makes the experience far more “personal”. It’s great to see more work being done to create a more seamless and up-to-date experience. This allows them to compete better with Google, Meta and Apple in this fast paced Gen AI development.

This is great to see and needed as update and adoption of Copilot for consumers is nowhere near the adoption of Chat GPT. Part of Microsoft’s “problem” is they are not great at consumer marketing. Copilot (IMO) offers far more value than ChatGPT and includes many features (as standard) that you need to pay for in ChatGPT.

Will this make you revisit and retry Copilot? Let me know your thoughts?


Posted on

Windows Recall: Enhanced Security& Privacy ahead of October Launch

As Windows 11 24H2 gets ready to release, Microsoft has announced major security and privacy upgrades to its (yet to be released) Windows Recall feature which was the flagship new AI powered feature announced on the back of the Copilot Plus PC announcements back in May.

Recall, which was initially met with scepticism and concerns around security and privacy, has now be significantly updated with stronger and more granular protections for privacy, encryption and identity access to make it ready for its “delayed” release next month.

What is Recall?

Recall takes screenshots of active windows on your PC every few seconds, analyses them on-device using a Neural Processing Unit (NPU) and an AI model, and adds the information to an SQLite database. You can later search for this data using natural language to prompt Windows Recall to retrieve relevant screenshots.

I covered Recall in more detail in an earlier blog

Responding to Customer Feedback


Microsoft’s decision to enhance Windows Recall comes after considerable feedback from customers and privacy advocates. The company initially delayed the public release of Recall, opting to make it available for preview with Windows Insiders. This cautious approach allowed Microsoft to refine the feature based on real-world usage and analysis of how it was used whilst they review ways to address the concerns and enhance security and privacy controls.

Putting Users in control of Recall

One of the most significant changes is that Windows Recall is now strictly opt-in. Users must proactively activate the feature, ensuring that no screenshots are taken or saved without explicit consent. This addresses a major concern about privacy and data security. Additionally, users can remove Recall entirely through the optional features settings in Windows, providing total control over its usage.

Advanced Security and Privacy Measures

Windows Recall now includes several advanced security measures designed to protect sensitive information after feedback, development and testing since it was announced in May this year.

Firstly – Recall is an optional feature and can also be removed entirely from the Windows Installation. If a user doesn’t proactively choose to turn it on, recall is off by default, meaning snapshots will not be taken or saved.

If Recall is enabled, all snapshots and associated data are encrypted, with encryption keys protected by the device’s Trusted Platform Module (TPM). This module is tied to the user’s Windows Hello login information and biometric identity, ensuring that no data leaves the computer without the user’s explicit request. Recall snapshots are also only accessible after users authenticate using Windows Hello credentials. This enhanced sign-in security ensures privacy and actively authenticates users before allowing access to their data

Recall also comes with a sensitive information filter designed to protect confidential data, like passwords, credit card numbers, and personal identification details, by automatically applying filters over this content. This uses AI and pattern matching to look for such information adding another level of privacy protection.

Additionally, Recall includes malware protection features like rate-limiting and anti-hammering measures, which prevent unauthorized access attempts.

Recall: Four Principles of Operation

Microsoft has redesigned Recall to function on four core principles:

  1. User control
  2. Encryption of sensitive data
  3. Isolation of services, and
  4. Intentional use.

Recall is always opt-in. Snapshots are not saved unless you choose to use Recall, and everything is stored locally……Recall does not share snapshots or data with Microsoft or third parties, nor between different Windows users on the same device. Windows will ask for permission before saving any snapshots.

David Weston – VP Enterprise and OS Security, Microsoft.

These principles will ensure that users can keep complete control over their data, with the ability to remove snapshots, pause them, or turn them off at any time. Any future choice to share data will need fully informed and explicit action by the user.

    Image (c) Microsoft.

    Conclusion

    In short, the enhanced Windows Recall feature represents a significant step forward in balancing functionality with security and privacy which show Microsoft are licensing to their users and take privacy and security seriously. By addressing key concerns and implementing robust protections, I believe Microsoft has demonstrated its commitment to user trust and data security. As Recall rolls out to Insiders with Copilot+ PCs in October, it will be interesting to see how these enhancements are received and what further innovations Microsoft will bring to the table. What do you think? Will you be trying this out or uninstalling it?

    When will Windows Recall be released?

    Microsoft announced last month that Recall will start rolling out to Insiders with Copilot+ PCs in October as part of the 24H2 release of Windows 11.

    Posted on

    What are Copilot Pages?

    Microsoft yesterday, announced the next stage of the evolution of Microsoft 365 Copilot with Wave 2. Amongst the many new features was the launch of Copilot Pages. This innovative feature is the first step into the new evolution of Copilot which is set transforming how employees interact with Microsoft AI in a new collaborative environment.

    What are Copilot Pages?

    Copilot Pages is a dynamic, persistent canvas integrated into Copilot chat, designed to facilitate what Microsoft call “multiplayer AI collaboration”. It allows users to turn insightful Copilot responses into durable, editable content that can be shared with teams for further collaboration.

    “This is an entirely new work pattern – multiplayer, human to AI to human collaboration”.
    | Jared Spataro | VP of AI at Work | Microsoft.

    Copilot Pages -Key Features

    • Dynamic Collaboration: With Copilot Pages, employees can work directly with Copilot on a shared page, prompting and refining responses together as a team rather than individually in silos.
    • Persistent Canvas: The pages are persistent, meaning collaborative efforts are saved and can be revisited and edited at any time by anyone.
    • Team Learning: This feature encourages learning from each other’s prompts, enhancing the overall quality and depth of the information gathered.

    Getting Started with Copilot Pages in 5 Steps

    1. Access Copilot Pages: Open your Copilot chat and look for the new “Pages” tab. Click on it to create a new page or access existing ones.
    2. Create a New Page: Click on “New Page” to start a fresh canvas. You can name your page to keep your projects organized.
    3. Collaborate with Your Team: Invite team members to your page by sharing the link. Everyone can contribute by adding prompts, refining responses, and editing content.
    4. Save and Revisit: Your pages are automatically saved. You can revisit and edit them anytime, ensuring your collaborative efforts are always up-to-date.
    5. Share and Export: Once your page is finished, you can share it with others outside your team or export it for presentations, reports, or further analysis.

    Check out the Microsoft Video for more.

    Where are Copilot Pages Stored?

    Copilot Pages are .loop files stored in a new user-owned SharePoint Embedded container. IT Admins can manage these files using Loop admin switches and other governance tools. The feature supports various compliance and manageability capabilities, including GDPR compliance, Intune device management, and data loss prevention. IT admins manage these .loop files just like any other files (.docx, .pptx, .xlsx, etc.). They support all the features of the SharePoint file system, including everything detailed here. 

    Additional capabilities, such as programmatic API access for third-party tools, are expected in Q4 CY2024.

    Read more in the Copilot Admin Support Pages:

    Availability

    Copilot Pages is rolling out “later this month” for Microsoft 365 Copilot subscribers and will soon be available to all Microsoft 365 subscribers. Loop must be enabled in your environment.

    Do you have it in your Tennant yet?

    Posted on

    Announcing Copilot Wave 2: Exciting New Features and Enhancements

    This afternoon (16th September 2024), Microsoft passionately announced (almost 9 months after Copilot was officially available to any organisation) the latest updates coming to Microsoft 365 Copilot as part of what they are calling “Wave 2”. The 30-minute-long session, hosted by Microsoft CEO Satya Nadella and Jared- Spataro, unveiled a heap of new features, capabilities and performance enhancements were announced across the entire experience.

    Microsoft 365 Copilot (THIS IS NOW IT’S NEW OFFICIAL NAME) provides enterprise data protection, ensuring the same level of security and compliance protection as other services like files, SharePoint, and emails. Microsoft said that significant improvements have also been made to PowerPoint and Excel based on feedback, including the introduction of Python integration in Excel.

    Some of these things are “generally available” from today and others are coming in the next few weeks and months. – See the end.

    Evolving the purpose and role of Microsoft 365 Copilot

    Microsoft told today, how Copilot is evolving from an individual productivity assistant to a collaborative partner at work. Copilot can utilise various content sources such as files, chats, calendar invites, and emails to generate rich outputs based on the needs of the user and teams of people. You will have seen in the various demos how the product demos now showcase how Copilot facilitates collaboration and achieves outcomes, making both personal and teamwork more efficient and effective.

    They have also focussed lots on performance and stability enhancements with huge investments in their Azure Data Centres.

    Microsoft also wanted to ensure organisations know just how much they focus on providing the same level of enterprise data protection to Copilot as they have with files, email, SharePoint etc.

    So, here’s my pick of what’s new and most cool!

    1. Copilot Agents

    Microsoft is also broadening the definition of “agents“, ranging from personal AI assistants to fully autonomous agents. These agents span across a spectrum, from human-in-the-loop to fully autonomous.

    This is the top announcement for me, I think. Agents (aka Custom GPTs,) have been a very popular discussion with my customers. When Microsoft talks about agents, they use it in its broadest term with an agent being anything from an AI assistant helping you retrieve information right (a foundational agent capability), all the way to autonomous support agent, meaning the agent does not need a human to intervene for it or for it to be able to do its work. Wow Right!

    You will see Microsoft focusing on agents across the entire spectrum – giving organisations the ability to create agents using natural language to “support whatever custom business process you are trying to automate“. This will be going into public preview later this month Copilot Agent Builder will be part of Copilot Studio as I understand it.

    As an example, say an issue out in the field required further research and follow-up and the team keeps all their customer records from deployment info to maintenance reports on a SharePoint site. Like many organisations, there’s loads of valuable information stored here, but it takes ages to sift through it and find what is needed. Now with Copilot Agents, an agent can be built from any SharePoint site library or folder. It’s possible to create an agent with a single clip and in just a few seconds, your agent is ready to be used and shared with your team and it can be simply added to the department’s team’s chat.

    Once created, created agents can be customised and things like topics and knowledge can be enhanced as well as the actions it can take. These can be connected to third party apps such as Dynamics 365 Supply Chain Management.

    2. Copilot Pages – Powered by Loop

    This is the second biggest announcement and combines two of my favourite parts of Microsoft 365 – Copilot and Loop.

    Accessed through BizChat, Copilot Pages, allows users to create side-by-side pages for real-time collaboration. Built on Loop components, these pages enable multiple users to work together and update data simultaneously, enhancing teamwork and productivity.

    This can be used when you are working in BizChat – pulling Copilot’s response into a sharable canvas (Loop spaces) built for real-time multiplayer collaboration with Al. Copilot can then be used to improve and expand upon subject matter, build visualizations, and bring in additional content.

    Copilot Pages.

    Biz Chat will be the place where you can start and finish most of your work before you transition it to your format of choice. Within Biz Chat, you’ll also soon see a button to move the output to the final experience – says move to Outlook, move to Word, move to PowerPoint etc., but you can do all of the pre-work in Bizchat which Microsoft’s customers say is where they spend “most” of their time.

    3. Biz Chat Enhancements

    Copilot Biz Chat, will, as I discussed above have the ability to send output to the final app for you to finish your work. You’ll see a send to button making workflow faster and just feel smoother and more integrated.

    4. Utilising Email and Bizchat for Business Insights

    Copilot can now use email and Bizchat to identify patterns and information, recall similar situations, and access relevant content from various sources. This feature is designed to help discover strategies such as price reduction, promotional offers, and ad campaigns as example by reasoning over email and chat threads to help solve business challenges.

    5. Copilot in PowerPoint Improvements

    Copilot democratizes advanced features in Excel by allowing users to use natural language to access them. PowerPoint improvements include the ability to easily build custom narratives, sections, and flow within presentations. Branding options ensure consistency with company logos, fonts, colours, and styles. PowerPoint’s new narrative builder, with creative control over the flow, reordering topics, deleting unwanted ones, and adding new ones.

    Copilot helps create presentation outlines quickly, offering image suggestions from approved sources or AI-generated options. It designs slides in company branding with various layout choices.

    PowerPoint’s new features include adding picture notes to slides, built-in slide transitions, and animated text. These features help users create professional-looking presentations quickly and easily, using their own corporate-branded content.

    Leveraging corporate templates requires that marketing teams integrate their organisations branded assets into a SharePoint OAL (Organization Asset Library) in order to be able to create presentations with organisational images. This is scheduled for release in Q4 2024.

    6. Copilot in Word

    Copilot in Word has new features including (finally) allowing it to reason over more document sources including emails, chat, meeting content and files etc making this much easier to get documents created while referencing multiple sources of information.

    7. Copilot Enhancements in Teams

    Improvements are coming to Teams based on user feedback. Copilot can now reason over chat in addition to meeting transcriptions. This entered public preview today.

    8. Copilot in Outlook Improvements

    Outlook’s new Copilot feature, ‘Prioritise my inbox’, organises emails based on topics, keywords, and important people. It identifies important contacts like your boss and their boss, enhancing email organisation and efficiency.

    Since Copilot can reference emails, meetings and attachments as well as knowing who your colleagues are when you are drafting it can save a huge amount of time as it understands the context in which you are working. This new feature is rolling out now, with early access for some insider rings.

    Summary

    As I am sure you are, I am excited about these new features (and there’s still more to come) and improvements and look forward to your feedback on the announcements.

    In summary, here’s what was announced again and when it will be available (according to Microsoft).

    Image/Table – (C) Microsoft.

    Are there things you were expecting to hear about but didn’t?
    Oh and if I missed anything, let me know!!!

    Proudly powered by WordPress