Data Analysis

LLMs are being experimented with to do so many things today, and one of the use cases that sound compelling is getting their help to generate insights from data. What if you could find the answer to your data question without begging a data analyst in your company? But this is easier said than done. To perform this task properly, LLMs need to know about your datasets, the tables, their schemas, and values stored in them. You can provide this information in the prompt itself if your dataset is tiny, but this is not possible in most real life scenarios, since the information will be huge and either it won’t fit the LLM’s context knowledge or it will be very expensive and not feasible. ...

Them: Can you just quickly pull this data for me? Me: Sure, let me just: SELECT * FROM some_ideal_clean_and_pristine.table_that_you_think_exists source

Every day a new tools pops out claiming “Throw the data analysts and data scientists of your company away, you don’t need to write SQL anymore, everyone can use data with our groundbreaking ’talk to your data’ tool”, and Benn discusses this: There are thousands of computational devils in details like how to handle nulls. For analysts, describing these specifics in English is inefficient and inexact. For everyone else, they wouldn’t know they need to describe them at all. ...

I came across this Linkedin post from a Google engineer, on a new (in preview) and very interesting BigQuery syntax: GROUP BY ALL. This will save time when writing and specially modifying complex SQL queries on BigQuery. The GROUP BY ALL clause groups rows by inferring grouping keys from the SELECT items. It will exclude expressions with aggregate and window functions, constants, and query parameters for a smart GROUP BY. So instead of GROUP BY name, city, device, browser, date or GROUP BY 1, 2, 3, 4, 5 you would use GROUP BY ALL. ...