I went with quantized Gemma
Well, was it quantized in a way that iphone 16 supports?
Often it’s the quantization where things break down, and the hardware needs to support the quantization, can’t run FP16 on int8 hardware… And sometimes the act of quantization can cause problems too.
And yeah, LLMs are likely going to be very hit or miss anyway.
The core functionality is simple:
Automatically, upon each payment, add the expense to my app
Update an Apple Watch complication with the % of my monthly budget spent
Categorize the purchase for later analysisCan someone enlighten me? I don’t understand why you need AI for this in the first place.
Reading the payment and turning it into structured data
Dozens of apps and services do that already and and have for years before AI existed.
Yea, but those are all using heaps of proprietary heuristics.
The beauty of LLMs and one of their most useful tasks is taking unstructured natural language content and converting it into structured machine readable content.
The core transformer architecture was original designed for translation, and this is basically just a subset of translation.
This is basically an optimal use case for LLMs.
Quite obviously not the optimal use case. “The tensor outputs on the 16 show numerical values an order of magnitude wrong.”
Grok says it’s right so it must be 🤤
tl;dr: because of LLM’s.
Certainly his use of LLM was stupidly egregious, but he found that even by those standards the math results underpinning the LLM were way off.
I wonder if this hardware issue is specific to HIS iphone 16 pro max, or ALL iphone 16 (pro, max)s
Given he apparently found a bunch of forum posts of people complaining about erratic behaviour so it may be more widespread
Might be helpful to have a reproducible test case for it.
