At work (not a tech company), there's an ongoing, slow, 365 rollout. The people who participate in the rollout are not technical in any way, but they all love it because they're not regular ChatGPT/Claude users, either. In a way, the restricted feature-set of Copilot compared to ChatGPT helps them, they're overwhelmed enough by Copilot. IT loves 365 because it's so risk-averse. No big jumps, no surprises, clearly defined data and risk policies.
I think if they drill down into the boring, slow, predictable, they will capture the market of risk-averse non-tech companies, not people.
I've tried building agents using 365 for our internal documents and they're OK for basic stuff (what's in what document where - max 20 documents only!!), but langchain/RAG/whatever are a million times more powerful.
Umm no, it's the opposite. It's super high-risk right now for us. Microsoft is constantly shifting stuff around leading us to have to constantly change our processes, documentation etc. Often with zero heads-up and often defaults to on. Some incidents:
- They suddenly started a "free promotion" with Sharepoint agents. We don't want to offer that to our users but it just popped in one day and the admin setting defaulted to on so people were already using it before we turned it off. This was a big deal for us.
- Constant rebranding of their product names leading to confusion among users and zero-value documentation and process rewrites for us. Also constantly fiddling with the URLs is sooo annoying.
- Constant changes in features leading to impact to our DPIA. For example copilot chat didn't have history at first. So no data was kept (they also promise they don't store any for training). Suddenly they added that one day, so we had to redo our entire DPIA because it now does suddenly store personal info which it didn't before so a whole lot of overhead comes into scope (data lifecycle, privacy regulations, security, data loss prevention etc). This is exhausting and there is no way to delay these features until we have approved them. Also, it caused our DPIA team to be highly critical after this incident. Because of course: If they did this before, what guarantees that they won't change something worse next month?
- Limited granularity of access controls - a lot is very high-scope on/off style controls. Meaning that if we want to block something we often block unintended features as well.
A lot of these things are definitely 'big jumps' and 'surprises'.
And the first product that lets a cell phone control keyboard and mouse sending camera to ChatGPT, having ChatGPT do all the work is not far off either.
Presumably both of these would violate your ... policies something awful. I should say will violate all your policies, because we both know this is going to happen.
You have no way of preventing both of these from occurring with IT policy.
Only marginally in general office use. Maybe is has better coding capabilities, but there is no way its the same sized model used on ChatGPT or via. OpenAI API.
Is it raking in cash right now? No. But that's just now. MS can and I feel is thinking longer term. Everyone is on a "Crash Dammit" Economist front-cover mode right, trying to will a crash into existence. Microsoft can think longer term about it - after all, we see younger generations diving in to make use of related technologies. Older demographics will, and are, slowly finding out what to make of the tech that is free with an OS install, and MS will see what kind of business to make out of that over time.
The problem with this broader topic is that people are restricting the time frame of the conversation to Right Now. MS doesn't need to worry about that so much, in the grand scheme of things.
That little copilot icon is sitting there in the task tray, clickable and usable. I think people will, to whatever degree makes sense for them, get used to making some use of the basic free tier. Monetization / restrictions requiring some degree of limitations to upsell will eventually come for the currently half-billion or so Win11 users.
I get the point that few are currently paying for it, but there isn't all that much pressure to do so right now. That will come, later.
It’s “installed” at my company but basically refuses to interact with basic company data like files in sharepoint.
Is it the IT department’s fault for not enabling something? I don’t know, but does that matter?
If the normal search system has access to the data why doesn’t copilot? Surely it would be trivial for Microsoft to run copilot in a way that keeps company data private so that they could actually turn it on and make it useful by default.
on my work computer - there's a sep. 365Copilot app that is tied into Teams,Sharepoint, outlook, and I believe our engineering wiki. Probably other stuff I'm not aware of.
I'm honestly shocked how often I use it now.
If I get a random Pipeline failure; I'll copy/paste it into the o365 Copilot app - and it points me to an email I didn't notice ~3 months ago about a new policy change, and then points me to discussion thread I wasn't on ~2 weeks ago about how to get in compliance with direct links to EngWiki 'how to fix..' documentation, and an Teams link to join the breaking teams Office Hours.
Just off a single ~1 sentence prompt and a stack trace
It's kind of amazing.
We make heavy utilization of Copilot Studio lite and full. Lite has quick access to SharePoint/Teams data. Full has access to _any_ data that has a Power Platform connector, REST API, MCP server, or Copilot [Graph] connector, all of which you can build or buy yourself. SAP, SQL, Databricks, you name it, Copilot Studio full can consume it.
It sounds like you don't have an M365 Copilot license but are instead using Copilot 365 Chat (the naming is horrid, absolutely).
I disagree, and I think you'll see what I mean when you shift the frame of reference from "person working at a large company with Office 365 installed" to a couple hundred million average Joe's having access to it at home.
Webs search is now a horrifying wasteland, and people know it. Remember the conversations we all had just a few years ago: LLM will replace web search. That's the key point here - not "replace web search" for the subset of people who have office jobs, but web search for the vastly greater number of people who just have it at home on their computer.
The tech - the products - are good enough for your average person at home who wants a starting point and a structure to work through for something they know nothing about. I think that's actually one of the strengths of the tech as it exists for the winder audience: you don't really need ultra accurate, super precise, info and checklists and guides when you just want to know what to look into to do some decorative tiling on the top of an old table you bought; a way to make sense of and work through a type of pop media you have become interested in; to give you a starting point to work through some new problem you have encountered in day to day life.
That "80 percent vaguely accurate-ish" threshold that LLMs can broadly deliver for a novice is actually good enough for that vast majority of things people deal with that aren't really super-critical. Are you idly curious about some ways to think about how to replant and re-do your back yard greenery? Curious about how to make sense of all the competing numbers and criteria and features when looking to buy your first air-conditioner? Want to take a vague, repetitive, not very well put together response to something your neighbor starting holding forth on on Nextdoor and make it tighter and better expressed?
That little Copilot icon that comes default in Windows legitimately can help you there.
If you have a tab selector "Work / Web" at the top, you do have the paid version. Select Work to make it able to find stuff in sharepoint, look at your calendar and emails etc.
If you don't have this selector you only have the free version.
> If the normal search system has access to the data why doesn’t copilot? Surely it would be trivial for Microsoft to run copilot in a way that keeps company data private so that they could actually turn it on and make it useful by default.
Upselling of course. Finding stuff in the gigantic trash heap that is Sharepoint is its main added value for me.
Also, RAG will cost a lot more compute.
I'm using Win 11 and I didn't even notice Copilot on my task bar until you mentioned it...
I think that is a wakeup call for me to finally flash my pc to linux
The list goes on and on, and we have to take time into consideration: people may have default strong feelings against the tech in question now, but often that's just a default stance. Over time, people will dip their toes in, and make use of it to whatever degree makes sense for them. Don't mistake current vocal criticism for the end all, be all, stance that will last forever: people get used to stuff, use it a bit, or more than a bit, time passes, and the tech slowly melds itself into people's lives in some manner and in some form: MS will wait and observer, and evolve the product to suit that slow change over time.
Yes that is inevitable. But it's exactly why I won't use it. It feels like a scam, the typical drug dealer model. Get them hooked on free stuff, and then milk their addiction.
I almost only use local LLMs now. They're not as good, no. But at least they are mine and nobody will raise the price suddenly. Some pro AI models I just use with API, for the rare instance where local just doesn't cut it. It costs me a few bucks per month that way, not 20 or 30.
A thing that gets in the way is not usable, is annoying.
I personally thinks this just pisses people off. It certainly does me. I don't want to be told what to do, especially by some vendor. If I don't want to use it I don't want to be herded into using it. If they try anyway I will just push back and start hating on it just out of annoyance.
However this has always been Microsoft's adoption model. Very pushy. Unlike for example Apple's where they make things look, feel and work really good so people actually want to use it.
It also disregards a big chunk of people not using Windows, where the little Copilot icon is not built in.
I have a paid 365 account and couldn't determine from logging in or account info screens if I was on paid or just the freemium version with my 365 plan
In testing out what I did have access to with Copilot, it was incredibly bad compared to ChatGPT or Claude, so I decided not to pay for Copilot whenever I see an ad for it.
If only MS could have asked Copilot whether their naming strategy makes sense.
It's easy to pick an arbitrary date and point out the lack of profit. But why don't we pick a date a year from now? Or three years. Or five years.
I think over time users will come to assume that their computing device has, as part of the many tools on it, a tool that they can ask about stuff and get general answers.
Is it profitable right exactly now? No. But I suspect it will become completely commonplace in a few years, and not even worthy of note or comment for the average user.
MS isnt some cash-strapped startup that needs to post instant profits right this moment.
Hmm no, but they are clearly under pressure. Integrating Copilot Chat in office was a pretty big move, and the constant rebranding of everything also feels like they have zero vision and are losing control and scrambling to fix it.
I wonder if Copilot Chat in office will really convince users though. As it doesn't have access to Graph it will lose some of the most valuable 'intelligence'. And thus users will try it, get convinced it's pretty useless and be more hesitant to pay for a license in the future.
I guess they spend their money on sales and licensing and not on developing good products now?
They make their money because they can go to the largest corporations & governments, talk to the CTO and tell them they have a product for everything that checks all their regulatory requirements, and all these products kind of sort of work together.
Who else can offer that?
For Microsoft, engineering is just about checking boxes on feature lists. Quality doesn't matter and engineering is a cost to be minimized. The people who make the purchasing decisions aren't the ones who have to use their stuff.
And yeah Teams, I can't stand it. Outlook too, in particular the "new" one that doesn't do half the things the old one could do and requires all your email to be in MS cloud even if you have another provider.
You either get A) hallucinations of "I created the calendar entry now" (it didn't) or B) an .ics file you need to download, then import to Outlook manually.
More tests about this scenario in here: https://www.perspectives.plus/p/assistants-without-hands
A year is a long time in the tech industry, let alone LLM (or Copilot) history.
Changing the name of the entire Office suite into Copilot didn't make things better. Now you have no idea of knowing what you are really getting. Is Office Copilot? Is Copilot Office? Is Office AI? Is AI Copilot? Is time real or just a concept? Who knows? The strongest product in Microsoft's portfolio with an established name is now gone.
Give it a year and it will be renamed again to Microsoft Teams 365 Copilot Enterprise with Office subsystem for Windows Business. Just to cover all bases.
They changed how that key works at a low level so it cannot be cleanly remapped back to right-CTRL. This is because, unlike the CTRL, ALT, Shift and Windows keys, the now-CoPilot key no longer behaves like a modifier key. Now when you press the CoPilot key down it generates both key down and key up events - even when you keep it pressed down. You can work around this somewhat with clever key remapping in tools like AutoHotKey but it is literally impossible to fully restore that key back so it will behave like a true modifier key such as right-CTRL in all contexts. There are a limited number of true modifier keys built into a laptop. Stealing one of them to upsell a monetized service is shitty but intentionally preventing anyone from being able to restore it goes beyond shitty to just maliciously evil.
However, I've had no issue remapping it to the Context Menu key on the Chinese Lenovo Xiao Xin laptop I bought recently. It shows up in the PowerToys keyboard remapper tool as an F23 key. (Yes, there are more than 12 F key codes! I believe there is 24 in Windows.)
Other OSes will not even be able to say things like "Press your Copilot key to open FEDORA SEARCH" or whatever without being cease'n'desisted to high heaven.
Terrible.
That's probably because you're not using it as a modifier so the OP's complaint of the automatic key-up event doesn't impact you (it's not necessary to have a keep-pressed state in this type of use).
So I thought, maybe the Copilot GTM is probably following the same strategy as another enterprise tool that was pushed onto its existing customer base: Teams. So I went looking for data on the rate of Teams adoption.
Interestingly, the sources I found suggest that Teams also had 8 million users in 2019, two years after release!
https://www.demandsage.com/microsoft-teams-statistics/
https://www.businessofapps.com/data/microsoft-teams-statisti...
Note, this was before Teams got a massive boost in early 2020 during the Covid push to remote. And for all the hate MS Teams gets, we see where it is today.
So maybe this is just following the normal curve of adoption of Microsoft products. Maybe it's even doing better than Teams, considering Copilot is not free, as Teams was at launch.
Put the LLM on answering (or at least triaging) your support tickets faster.
Plus for 10$ a month (~8.50 EUR last month) I get more-than-enough-for-me access to Gemini, Claude and OpenAI models from my code editor. I wouldn't pay double for Claude Code or Cursor, so I feel like it's a good enough deal.
Copilot doesn't run on Microsoft's own LLMs, they're all third party. ChatGPT in office copilot, in github you can choose various other ones too.
Microsoft is really just a reseller there.
Personally I think Claude Code (with Opus in particular) is far better. Unfortunately they are very aware of this and thus it is eyewateringly expensive.
Copilot is often wordy, wrong, and seems like it can't do a lot of the same things I can accomplish without logging in on ChatGPT
If only asking from Copilot in a chat, the results are less spectacular or reliable. That just shows how a chat UI isn't the best solution for everything. Yet taking the M365 Search terms as input for the LLM to translate into a query that doesn't require the user to know the exact keyword to search - yes, that's a clear step forward.
And Graph grounding remains one of the features behind the premium license. So, only ~2% of M365 users will currently see this benefit. Unless the pay-as-you-go pricing option has huge adoption that we don't have any leaked figures on yet.
Besides, I just recently saw a review of the newest Copilot for Excel by a serious spreadsheet jockey, and he was truly blown away. Basically, it knows everything Excel can do, how to do it, and it now truly understands numbers as numbers and when to use formulas, etc. etc. Basically, it went from smart automation to full on AI.
If Microsoft doesn't fall for short term thinking, the product will eventually succeed.
Possibly the Office Agent can become a V2 of what M365 Copilot was originally sold as. It's not at all surprising that it takes a few years to figure out what LLMs can really be used for in Office tools. Whereas asking for businesses to fund all these experiments with a $30pupm license is not the best move to create early adoption and fans...
I also use it to play around with data and roughly graph it to understand trends without having to roll out a Jupyter notebook.
I find it quite useful.
What I like:
- Meeting catch-up (I don't use meeting minutes much)
- Finding stuff in my messy uncategorised mailbox
- Finding stuff in sharepoint
Things that were a heavy disappointment:
- Everything in excel
- Rewriting text (it's always too noticeable that it's AI, it should be able to learn and match my style)
- Powerpoint presentation generation. It could not modify for a long time and even now it's just not useful
Tbh I only use it a few times per month where it's actually useful. For other things I've tried but it needs so much explaining to get it to do exactly what I want that it's more efficient to just do it myself.
You’re a company whose main product has a brand recognition that ranks right up there with Coca-Cola and Apple, and you rename it after a new product that is a pile of hot garbage. Absolutely mind-boggling.
Traditional search works fine.
If you see a work and web tab at the top, click work. Then it should be able to. If you don't see this, your company is not paying and it just won't work.
But only for paid users. Free users can't have access (and also there is zero custom integration on the free version).
Are there any companies solving LLM-based interactions with arbitrary messy spreadsheets?
Not the nice demo ones with a single consistent table and maybe some charts/PivotTables - the ones with 20 sheets each of which has 3-4 different data tables laid out side by side, which should be linked to each other but likely aren't?
I've long thought that one could build a probabilistic graph representation, annotated with text, of "this is the likely meaning of this area in the sheet, this is how it relates to other columns in other areas, formulas are inconsistent in this way," allow an LLM to ask questions of a tool that can traverse this graph until it determines an optimal plan, and allow it to output and iterate on e.g. xlwings code to execute that plan.
I'm frankly surprised Microsoft didn't create an entire "skunkworks" for this problem - is anyone else doing so?
If corps ever stop needing that, MS will be in trouble. Until then, few worries. They can play at following fads (not that I think AI is a fad) and not worry about execution.
Microsoft is actively killing of Active Directory and they're replacing it with Entra ID, which is "just" OAuth and hence is relatively easily replaced by a competing product.
https://learn.microsoft.com/en-us/windows-server/get-started...
Of course all the people asking for these were already very interested in AI so they are not a very representative group. We did take that into account.
Even the "productivity" suggestions in teams are ridiculous - one of them is "roast me by my calendar"...wtf is that? How is that related to productivity at all?
something, something, clown world
But also, 30$ per month is a lot. For around 300 users you're already talking 100k per year! It's also many times the price that companies pay for their existing M365 licenses including office apps, hosting etc. So it means a completely rebudgeting of their entire IT operations. That kind of money leads to highly critical evaluations that reach the very top. They don't do much volume discount because it's all compute heavy.
Another thing I've been seeing is that users who get Copilot often don't use it at all. From the users buying it, I've heard that in our org only 5-10% actively use it. Others might tap it once a month or use only one or two specific features like the teams meeting summaries. Only the most serious AI bros use it for everything.
I'm in that group myself too. I use it for asking questions about meetings (like "i was distracted for a bit, what did I miss") and for finding stuff in SharePoint which always ends up being a huge mess.
I don't use the email rewriting because Copilot can't match my style which makes it very obvious that I've used it which automatically leads people to devalue the message ("he didn't even bother to write it"). Something the end of this article also alludes to when it refers to an all hands email from Nadella being made with copilot.
Also the pushiness of Microsoft is exhausting. Since that ignite 2022 it's's constant Copilot this Copilot that. For me that has the opposite effect. I just end up rolling my eyes when they rebrand entire product groups and keep rolling their product names. It makes it feel like they have no idea what they are doing and just scrambling to keep up.
The dumbing down is also a problem. When 'researcher' was released first it would spend half an hour on my queries making really great and lengthy well-researched results. It was really powerful and comprehensive. Now all I get is a couple of minutes. It's already lost its value. I'm sure they do this to tune the compute costs but it makes the value prop even worse.
I don't know where this is going but our company is not buying much of it for now. And tbh I can see why.
