Pointing at the error: compiler-style diagnostics in uutils coreutils
67 by ingve · · uutils.org · 8 comments
Handling errors in compilers has become the most interesting part of my job. If you do the math, most calls to compilers are error-yielding calls. So those calls really need to produce the best errors possible! (Maybe this is less true in LLM land, but certainly it was right before.) Tracking spans, doing extra algorithmic work, and exploring why the expectation failed in order to give a user the most-informative error possible is extra work on the main compute path. But it pays dividends: easily-diagnosable bugs and quick fixes present in the code itself. Tracking spans to point back to origin definitions, etc., always feel like plumbing work until you see the error that explains exactly what went sideways. And once you do, the errors that don't feel like a frustration you are trying to cast your net over.
>Maybe this is less true in LLM land, but certainly it was right before
I disagree:
I was tasked with rewriting a Python application in ucode for use on OpenWRT. It's a low-resource and dynamic language, and sufficiently JavaScript-like that Opus 4.8 couldn't write code without errors that were obvious with my LSP. The interpreter errors were often reported with the the correct line and column numbers, but with the incorrect code and a vague error message. I had to ask it to use my LSP and fix the warnings and errors, which led to it fixing a few dozen bugs.
Just the other day, Fable 5.1 still wrote a lot of code in Rust that had compiler errors. While I'm sure it could have fixed these errors if the Rust compiler also reported incorrect line numbers, columns, code, with a vague error message to boot, I saved a substantial amount of time and tokens because its error reporting is so comprehensive.
Specific and accurate error message reporting will always have the same high value that it does today. Error messages are far cheaper to generate than tokens.
Can I ask which Compiler you are working on?
Please run all of these through a TTS engine.
Yes, it is very helpful to have actually directed information. But the need to use a TUI to do it, bugs me. Not coincidentally, I use JAWS most of the time.
While this is using ariadne for rendering, for annotate-snippets (the renderer for rustc) there is active work for a screen reader mode. This is being tracked at https://github.com/rust-lang/annotate-snippets-rs/issues/351
Cool innovation (actually is good and cool).
Do it after you have actually made the rust version work the same as the C version.
Insane to be adding complexity, breaking compatibility and introducing new bugs to software that needs to function correctly.
> Insane to be adding complexity, breaking compatibility and introducing new bugs to software that needs to function correctly.
On one hand yes. On the other hand maybe adding this detailed error reporting will prove so useful to LLMs that it helps uutils adoption grow significantly, instead of uutils fading into obscurity while the world keeps running on the original coreutils.
AI would do better with a wordier textual description, rather than a 2d layout.
[comment dead]
[comment dead]