I'm currently tracking exactly two numeric metrics: total MAUs (to track the aforementioned), and total DAUs (to gauge adoption and rightsize seat-licensed contracts.)
If the benefit is there people will use it or get left behind, there's no sense having a mandate that people resentfully try the new tooling.
Imagine you had a developer who writes Java using vim. It sounds insane but they are just as productive as everyone else. Then you mandate they have to try IntelliJ every quarter, just to see if maybe they like it now. You're just going to piss them off and reduce their productivity by mandating their workflow.
FWIW in the face of these kind of mandates I have been using tokens but ignoring the output. So it's costing my employer money and they have a warped metric of whether the tool is actually useful.
"If the colleges were better, if they really had it, you would need to get the police at the gates to keep order in the inrushing multitude. See in college how we thwart the natural love of learning by leaving the natural method of teaching what each wishes to learn, and insisting that you shall learn what you have no taste or capacity for. The college, which should be a place of delightful labor, is made odious and unhealthy, and the young men are tempted to frivolous amusements to rally their jaded spirits. I would have the studies elective. Scholarship is to be created not by compulsion, but by awakening a pure interest in knowledge. The wise instructor accomplishes this by opening to his pupils precisely the attractions the study has for himself. The marking is a system for schools, not for the college; for boys, not for men; and it is an ungracious work to put on a professor."
-- Ralph Waldo Emerson
If AI makes an employee 10X more productive they get a slight pay raise maybe, but the company makes substantially more money or gets substantially more output. So there is a large difference in incentives.
What you're actually doing here, from my POV, is incentivizing your employer to use more invasive metrics when they tried to stay hands-off and mandate the absolute bare minimum of "uh, give it a shot and see if you think it's useful right now."
The analytics that Claude Enterprise exposes are far more intrusive than I would want to be subjected to as an engineer, so I rolled out a compromise. I don't even track who the active users are, currently.
But maybe you're right, and there are enough people sabotaging the metrics out of spite, that there's a reason they provide the other data.
I hope that the engineers in my org are more mature than that, and would be willing to just say "I'm not currently using it", but thanks for giving me something to think about.
That’s not the bare minimum, though. The bare minimum is: “if you are meeting or exceeding your job expectations, great work, keep using the tools that are working for you.”
To a productive employee, merely saying “just try out AI, it might help” feels like the boss saying “just try out astrology or visit a psychic for a reading. You might find it interesting.”
Go ahead and spend more time collecting more granular metrics spying on your employees. Apparently there aren't more valuable things for you to do than micromanaging individual developers.
Where I work there as certainly been that kind of discussions, "we need to use AI for this, because no offense but you are simply not fast enough". And this from people who do not understand software development and has never worked with it. They have only read the online stuff about 20X speeds and FOMO. (And my workplace is generally quite laid back and reasonable. I am sure many other places are much more aggressively steered.)
If you have accurate metrics to gauge developer productivity then use them.
But you don’t because if you did you’d be a billionaire.
What you have is metrics that can measure developer busyness. If you use those metrics all you’ll do is run your good devs off and keep the ones who can’t find new jobs.
So you’ll have to do what anyone who manages software teams has always done and trust your line managers to manage your devs.
When it comes to people wasting tokens, most people aren’t gonna to do it with the intent to fuck your metrics. But if you tell people you are measuring something they will find a way in increase that metric whether it results in anything productive or not.
Re: some of them being upset about it- probably. Some people are also upset about being required to use Jira. I personally dislike using Okta.
I engaged in the thread in good faith, and am transparent about what I'm doing and why. I also clarified that part of the job in my org is experimenting with these tools.
Not necessarily. A carpenter has a job to make things. Not to use specific tools and keep up with the latest tools used for repairs. It can be suggested, but telling a caprenter what tool to definitely falls under micromanagement.
>Some people are also upset about being required to use Jira
Jira's job is to report metrics to management. That implicit to the job. Telling people how to perform their tickets is micromanagement. The whole goal of a non-junior employee is to trust they can estimate and accomplish their task.
