Every day a new tools pops out claiming “Throw the data analysts and data scientists of your company away, you don’t need to write SQL anymore, everyone can use data with our groundbreaking ’talk to your data’ tool”, and Benn discusses this:
There are thousands of computational devils in details like how to handle nulls. For analysts, describing these specifics in English is inefficient and inexact. For everyone else, they wouldn’t know they need to describe them at all.
LLMs as a better interface to data models and semantic layers are great. LLMs as a Eucharistic for transubstantiating sketches of queries into real ones are great. But LLMs as an analyst, turning novel business questions into full queries? Seems unwise.