From Newsgroup: comp.compilers

In article <25-05-016@comp.compilers>,
<anton@mips.complang.tuwien.ac.at> wrote:

I am just surprised that I read and hear so much about work based on LLMs, which
seems to be a dubious technology for doing things where correctness is >important. What am I missing?

The fact that AI is "hot" right now? "Sexy"? "Good for getting
startup capital"? Who cares about correctness?

Pardon my cynicism.

Arnold
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.compilers

On 24/05/2025 06:26, [anton@mips.complang.tuwien.ac.at](mailto:anton@mips.complang.tuwien.ac.at) wrote:

I expect that I am not the first one with this idea, and that there are papers about it already, but I have not kept up with optimization literature, so I
am not aware of that. Maybe someone knows of such work?

A recent paper (of which I am one of the authors) is not specifically directed at optimisation, but at program understanding. Since a smaller program is often a more efficient progam there is a lot of overlap. Also, the simplicity metric developed in the paper could be refined to an efficiency metric which aims to reduce the amount of code and also replaces less efficient constructs by more efficient but semantically equivalent constructs:

"Climbing The Hill to Understand The Code"
Doni Pracner, Martin Ward, Natasa Sukur and Zoran Budimac
IEEE Access April 2024, Print ISSN: 2169-3536, Online ISSN: 2169-3536 doi:10.1109/ACCESS.2024.3389500

<http://www.gkc.org.uk/martin/papers/Climbing_The_Hill_to_Understand_The_Code.pdf>


\--
Martin

Dr Martin Ward | Email: [martin@gkc.org.uk](mailto:martin@gkc.org.uk) | <http://www.gkc.org.uk>
G.K.Chesterton site: <http://www.gkc.org.uk/gkc> | Erdos number: 4
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.compilers

arnold@freefriends.org wrote:

In article <25-05-016@comp.compilers>,
<anton@mips.complang.tuwien.ac.at> wrote:

I am just surprised that I read and hear so much about work based on LLMs, which
seems to be a dubious technology for doing things where correctness is >>important. What am I missing?

The fact that AI is "hot" right now? "Sexy"? "Good for getting
startup capital"? Who cares about correctness?

Pardon my cynicism.

I samewhat different spirit: LLM folks have a lot of money for research.
They try to apply LLMs to various things, it does not matter if those
trials make senss or not. In the process they improve their machinery
and may get some possible applications.

Concerning correctness, one possible variation is to treat LLMs as
heuristic search, which may find some gems but also may produce
garbage. For me it would be natural to combine that with more
reliable technology. Possible combionations:
- optimization hints for classic compilers (in some cases that
may involve inventing more precise types), if hints are wrong
optimization would not fire or would not lead to improvement
in runtime.
- LLM produced proofs. Compiler would check the proofs and
reject wrong ones.

Of course we will see companies applying LLMs techniques with
little checks, but this is due to how companies works, if not
LLMs than something else would be abused.

--
Waldek Hebisch
[See the two papers I recently sent out, which use LLMs to do C
to Rust conversion but have feedback to say whether the conversion
was correct. -John]
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.compilers

On 2025-05-25, antispam@fricas.org <antispam@fricas.org> wrote:

Of course we will see companies applying LLMs techniques with
little checks, but this is due to how companies works, if not
LLMs than something else would be abused.

LLM-generated code cannot just be checked, because checking equivalence
of two pieces of code is as hard as the Halting Problem.

At best you can do it for very short program fragments, (like basic
blocks of instructions, or small trees) that can be executed to see if
they have the right effect on all the temporary registers. (How genetic programming works, more or less.) If the fragments don't contain any
looping or recursion, you don't have to worry about nontermination.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
[General program equivalence is the halting problem, but there are subsets where
you can say these two programs are equivalent or those two are not. The question
is whether there are enough of those to be useful. I have no idea. -John]
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.compilers

On Sun, 25 May 2025 03:52:17 +0000, arnold@freefriends.org wrote:

In article <25-05-016@comp.compilers>,
<anton@mips.complang.tuwien.ac.at> wrote:

I am just surprised that I read and hear so much about work based on LLMs, which
seems to be a dubious technology for doing things where correctness is >>important. What am I missing?

The fact that AI is "hot" right now? "Sexy"? "Good for getting
startup capital"? Who cares about correctness?

ANNs can be made into excellent pattern matchers. The probem IMO is
that too many "applications" are not simply matching patterns and
acting upon them, but rather are matching pattern /prefixes/ or
subsets of the complete pattern, and then acting /as if/ the complete
pattern exists.

This is what "generative" LLMs are doing: e.g., the AI sees the words
"brown" and "jumped", assumes "The quick brown fox jumped over the
lazy dogs", and bases its responses on that sentence.

If it turns out that, in fact, the user was seeking information on "Encyclopedia Brown"[1] and the pickup truck rather than on different
species of canines interacting ... well, too bad.

Pardon my cynicism.

Neural AI has been around since the 1950s ... I've been following it
since the late 1980s. It just wasn't practical to train even small
ANNs until (relatively) low cost SIMD became available.

Now ANNs have grown to scales where, even having cheap SIMD, once
again the computational costs have become an issue for training (and
retraining when relevant).
[The costs can be an issue even for deployment, though less so.]

ANNs are best used as high speed probablistic pattern matchers - which
includes use as associative memories. The problems are in relevant scoring/comparison of multiple possible matches [stability], dealing
with low scoring matches [guessing], trusting that the ANN
usually/always will be correct [non-technical users], and how the
responses are being used [again, non-technical users].

I think a rule based logic system using ANN for input matching is a
good combination with a lot of potential. You don't need to resort to
ANN logic - rule based systems can account for probabilities ["fuzzy"
logic]. I remain unsold on ANN logic because - at least currently -
it neither can be verified, nor easily changed. Having to retrain
your ANN logic every time some conditional in the problem set changes
is a non-starter for me.

YMMV.

Arnold

George


[1] https://en.wikipedia.org/wiki/Encyclopedia_Brown
--- Synchronet 3.21b-Linux NewsLink 1.2