Token maxing is a cost line, not a result.
Meta made token consumption a performance metric, engineers gamed it, and this week Meta dropped it. The firms publishing numbers worth reading did something harder first: they kept a baseline from before the deployment.
Hi folks,
Plenty of firms published AI numbers this week. Very few published a result.
The one thing.
Token maxing is a cost line, not a productivity measure.
Token maxing became a norm this year: firms measuring performance on how many tokens their people were burning. Meta ran it as a criterion in engineer performance reviews, and engineers responded by burning tokens in bulk to climb the internal leaderboard. This week Meta dropped the metric in favour of the quality, speed and complexity of the work. (The Decoder)
The lesson is clear enough. A token count is a cost line. It is not a productivity measurement.
More recently we are seeing firms move the other way, setting a baseline before deployment and measuring against it after, to show true ROI. Three from this week:
- Banner Life, a US life insurer, took instant underwriting decisions per underwriter from 67 a month in 2022 to 109 now, and cut cost per application 27% over three years. Its CEO stresses that humans still write and check the rules. (Digital Insurance)
- IAG took HR query deflection from 30% to 67% in the first month, against an 80% target. Underneath the number sit about 730 knowledge articles being rewritten so AI can retrieve them. (iTnews)
- OpenAI put its own research org at 3.1 agent-workdays for every human workday, at over US$600 a day of inference for the median researcher. (OpenAI)
What it means for a fund:
This is consistent with what we have been hearing in conversations with funds. The strategy has mostly been a broad "let's use AI", rather than a view on the outcome wanted and the before-and-after that would prove it.
When you are implementing AI, and especially when you are doing it workflow by workflow, we suggest starting by defining the current state. Then define the future state, and assess the improvement against those two.
In the mix.
- ASIC says it cannot see the sector clearly enough (ASIC)
- Chair Sarah Court told the parliamentary joint committee on 4 September there is "a lack of information and insight into wholesale private credit funds", and that Australian data collection falls short of the US and UK. She pointed to Bathla, whose administrators put liabilities past A$3.4 billion the same day.
- Why it matters: APRA is naturally putting more scrutiny on the credit industry, and ASIC and others are asking questions. It will be interesting to see how this one evolves.
- APRA made ING Australia hold an extra A$50 million over a reporting failure (APRA)
- ING Bank Australia reported a liquidity coverage ratio of about 160% when at times it was below the 100% minimum. APRA is making it hold an extra A$50 million of capital until the reporting is fixed and independently reviewed.
- Why it matters: your numbers need to make sense and be accurate, and if they are not there are consequences. Cash flow and liquidity monitoring is something we hear about a lot from the funds we work with.
- Australia's biggest funds are building their own AI platforms (Financial Standard)
- UniSuper's digital adviser now gets a member through a statement of advice in 7 to 14 minutes. AustralianSuper, with A$430 billion under management, is building its own digital advice platform with Ignition Advice.
- Why it matters: large financial companies are using AI to bring costs down and improve the customer experience. The same is becoming the case in credit.
Levercon builds the AI operating system for credit funds: we connect your data, deploy agents across origination and monitoring, and run the repetitive work end to end.
Book a strategy call.
See where AI fits in your firm.
A short conversation about where AI could help your firm, what to build first, and what it would take.