ChatGPT

The world is split about ChatGPT, and I’ll throw in my two pennies. Firstly: I am something of a hater, for technological and philosophical reasons. The idea that just taking a statistical next-word maximiser and scaling the hell out of it leads to eerily good results doesn’t make sense, and ChatGPT is better than it has, in my mind, any right to be. That of course does make me doubt myself and wonder what I am missing. I am by temperament a late adopter and dismissive of hype. But I have to acknowledge that it is possible that ChatGPT will completely change the world and/or make me and all knowledge workers obsolete and/or destroy humanity. I just don’t currently think it likely. (I don’t worry about this in any case, because this is an “everyone problem” so I rely on the other people who are busy worrying about it.)

For a while I refused to try it (or CoPilot, etc). It seemed like a gimmick and I didn’t think it would save me time. But under some pressure from colleagues, and an interest in challenging my prejudices, I have been giving it a shot lately.

Often the output is useless. For example, I needed to write an SQL query which filters WHERE ID IN (long list of ids), where the list is much longer than the maximum character limit allowed by the SQL implementation I’m querying. So I wanted to write a Python function to take in a query, list of ids, and character limit, and return a list of queries below the maximum length and containing each id, only once. This seems trivial but annoying so I considered it a good task for ChatGPT. It gave me a solution that had a few obvious errors, but once I fixed them I discovered that there was also at least one non-obvious error, so at that point I gave up and just did it myself.

After a couple of experiences like that, I did finally find something that ChatGPT was helpful with. I wanted an SQLite function that was the equivalent of lstrip in Python. I couldn’t find it by searching the web, although I did find something which worked for removing a single character (combining INSTR and SUBSTR). After prodding ChatGPT I discovered LTRIM, which I hadn’t been able to find through Google-fu, so one point for ChatGPT.

So I think that it can be useful. I was going to say that it is like a better version of Google - surfacing information fairly effectively and with some added polish - but it is also worse in some ways than Google, because since Google shows you the source of the information, you can assess its quality, whereas ChatGPT lies with equal confidence. I think it is better to say that it is a generative Google - think of it as generating something in the same general zip code of the truth. It can be helpful for brainstorming or to get you unstuck, but its output is more poetry than science.

People who learned to program in previous generations did it with heavy dead-tree books, which they read end-to-end. Search engines and Stack Overflow changed the way people learn, for good and bad. There are things I do often in a language I know well which I still search for, because I don’t bother to remember it since it’s so easy to reference online. (Example: the syntax to flatten a nested list in Python - [item for sublist in l for item in sublist], which I always look up here.) There is value to the old way - a deeper knowledge of the abilities of the tool you are using, knowing to use capabilities you wouldn’t have guessed to search for, the ability to program where you don’t have an internet connection - but there is also probably some value in the new ways, too. Tools like IDEs and autocomplete free up the human brain for working faster and more creatively, and it is possible that AI assistants will generate snippets together with a user in the future, widening the circle of entry-level coders, and improving the productivity of experienced coders. Or maybe this is an energy-burning hype trap that has already hit its limits and will soon fade away. Or maybe it will wipe us all out. Only time will tell.