datastation

I use Go heavily cross-platform developing DataStation [0] and dsq [1]. I am not an expert. And I don't have proof for it but on some rudimentary benchmarks the Linux-specific file idioms in the Go standard library definitely don't seem to translate well to even macOS let alone Windows. For example some good streaming techniques for reading large files on Linux that work really well there seemed to be pretty bad on macOS. I think Amos has presented more proof than I can on the topic of just how Linux-influenced Go is. And I think it is fine for the majority of Go users because the majority users of Go are building server apps or Linux CLIs. Amos has spent some time building cross-platform desktop systems with Go for itch.io and I think I'm seeing some of the same things they are in that scenario. I think this is a reasonable article. I di

Thanks for including dsq! Its focus is always going to be flexibility and UX over performance since it's just a thin wrapper over the internals of a GUI data integration tool [0]. For example many other tools operate on only one or two file types and have optimized for querying on those types. Some like octosql and spyql IIRC have implemented their own sql parsing layer and engine. So if full sql support is necessary you may not be able to use them. In contrast dsq supports excel, avro, parquet, json, csv/tsv, nginx logs, apache logs, json newline, concatenated json, etc. And this list keeps growing. Dsq stores data in sqlite under the hood so you have access to the full sqlite dialect. Textql and q also do it this way. That said, despite performance not being the major I goal I think it will get better over time. There's already an OSS c

I don't sign the build at all for Windows or Mac for DataStation [0]. I'm planning to sign the mac build because it's cheap and not insane. Windows signing is expensive and insane. [0] https://github.com/multiprocessio/datastation

This looks awesome! I've already played around with pyodide and coldbrew doing the same thing for CPython. I use it for an in-memory playground [0] of an open-source desktop app I build [1]. I've been waiting for Ruby, Julia, and R support to add them in too. That said, I am not seeing a link in here about how to actually use this code. Is there a good tutorial/example somewhere? [0] app.datastation.multiprocess.io [1] github.com/multiprocessio/datastation

I've got some good first projects if you're interested in OSS data tools and have some Go experience. Check out: https://github.com/multiprocessio/datastation/blob/main/GOOD...