Leveraging LLMs.

March 3, 2025 (1 year ago)

It's really hard to evaluate any reasonable new technology when there is a giant hype cycle. The promises that are made add to the challenge, embodied by this comment on HN about the GPT-4.5 release:

Early testing shows that interacting with GPT‑4.5 feels more natural. Its broader knowledge base, improved ability to follow user intent, and greater “EQ” make it useful for tasks like improving writing, programming, and solving practical problems. We also expect it to hallucinate less.

Early testing doesn't show that it hallucinates less, but we expect that putting that sentence nearby will lead you to draw a connection there yourself

It is undeniable that LLMs are helping everything from collecting research to accelerating software development. I love that Claude can get me mostly correct answers much faster than I can summarize. And I'm not going to trust Claude or ChatGPT more than a random editor on Wikipedia, but the benefit of Wikipedia is there is an editorial consensus. It may be biased, externally influenced, or even wrong but there is a consensus and that adds value.

I remain a cryptocurrency skeptic, although stablecoins definitely have utility. I started out by being an LLM skeptic, and probably I still am, but in a very different way.

It feels I'm being told that by using an LLM or Agent, I will be able to do things I couldn't do before. At some point over the last year my stance is it's purely leverage.

Using an LLM to assist in tasks reduces the time or cognitive load. With some tasks, reducing cognition is actually damaging because having context is valuable in the ongoing involvement in a subject.