This is a very large part of the problem. This and the fact that, by design, the output of AI is, despite its faults, increasingly difficult to distinguish from good work. Accountants’ spreadsheets and traditional software systems can be audited but AI output can’t: there’s no auditable process. The output doesn’t come out of nowhere, but the process is fundamentally resistant to inspection and validation. The only choices then are to run a parallel auditable system, audit it and compare the results, or run without quality control. It’s a crazy risk, but how many companies will spend the money to mitigate it? How many can survive the short term consequences of doing so?
Did they actually check if the generated stuff was correct? I’m betting it isn’t
Nobody ever does. All AI demos are just: look at this mountain of generated text.
This is a very large part of the problem. This and the fact that, by design, the output of AI is, despite its faults, increasingly difficult to distinguish from good work. Accountants’ spreadsheets and traditional software systems can be audited but AI output can’t: there’s no auditable process. The output doesn’t come out of nowhere, but the process is fundamentally resistant to inspection and validation. The only choices then are to run a parallel auditable system, audit it and compare the results, or run without quality control. It’s a crazy risk, but how many companies will spend the money to mitigate it? How many can survive the short term consequences of doing so?