sqlite-utils

Second this recommendation for the terminal. For my CLI toolbox, VisiData is my favorite. I find VisiData is great for quickly exploring and querying data from the CLI. It can handle many types of files (SQLite, CSV, TSV, Excel, JSON, YAML, etc). Visidata loads all the data into memory, and so is very responsive when exploring the data. It allows you to quickly do all sorts of of adhoc queries interactively, without having to write a valid SQL query. I haven't used Q. When I first heard of it, I liked the idea that Q allowed you to run random queries on CSV and TSV files. However, it seemed like it would be slow if you wanted to do follow up queries, since it had to repopulate the in memory SQLite file for each query. Though it looks like the latest version has a way to cache the generated sqlite file. So that seems like it could

I've been using GitHub Issues for this for the past few years, and I absolutely love it. Each of my projects has issues. If I'm fixing a bug or working on a feature I open an issue for it, then I post a stream of comments as I figure out the details, make decisions, find examples of code that I need to use and so on. I have my own private notes repository which I just use for this for things that don't relate to a current project and that I don't want to publish yet. I add a "research" label to my repos, and open research issues for ideas or for things that I want to explore that are relevant to the project in some way. Recent example: https://github.com/simonw/sqlite-utils/issues/344 - researching STRICT mode that was just added to SQLite, and seeing if it would make sense to support it in my sqlite-utils library. GitHub's issues API is

I've been looking at calling ".import" in SQLite directly as an optimization for my sqlite-utils tool - I'm seeing a big speed up in comparison to parsing the CSV in Python: https://github.com/simonw/sqlite-utils/issues/297

sqlite-utils isn't (yet) a dependency of Datasette. The goal with sqlite-utils isn't to build an ORM - that .rows_where() method is definitely the most ORM-like piece of it, and it's purely there as a SQL-builder - it was a natural extension of the .rows property, which grew extra features (like order_by) over time as they were requested by users, eg https://github.com/simonw/sqlite-utils/issues/76 The vast majority of the library is aimed at making getting data IN to SQLite as easy as possible. Datasette is mainly about executing SELECT queries, and I found myself writing a lot of code to populate those database files - which grew into a combination library and CLI tool. So yeah - I don't use the rows_where() feature much in my own code - I tend to use db.query() directly - but I've evolved the method over time based on user feedback. I

Good point, I should add that to the docs. You can see the schema in a couple of ways: sqlite-utils memory blah.json "select sql from sqlite_master" Or sqlite-utils memory blah.json --dump The second option will dump out both the SQL schema and the INSERT statements, so you probably want to | head that. You've inspired me to add a --schema option which does the dump without also including the inserted rows. Issue here: https://github.com/simonw/sqlite-utils/issues/288

I use documentation systems that publish the documentation from the repo to a website. Most of my projects use Sphinx and reStructuredText for this, but I recently tried MyST (Markdown for Sphinx) and I like that a lot. Some examples: - https://docs.datasette.io serves documentation from https://github.com/simonw/datasette/tree/main/docs - which has documentation unit tests here: https://github.com/simonw/datasette/blob/main/tests/test_doc... - https://sqlite-utils.datasette.io/ serves from https://github.com/simonw/sqlite-utils/tree/main/docs - unit tests here: https://github.com/simonw/sqlite-utils/blob/main/tests/test_... - https://django-sql-dashboard.datasette.io/ serves from markdown in https://github.com/simonw/django-sql-dashboard/tree/main/doc... - I don't have documentation unit tests for that yet Those three are all hosted on h