awesome-bigdata

postgres is adequately efficient technology from last millennium, but at the level of execution efficiency it's pretty much the same as oracle, maybe a bit worse more recent systems like impala, spark, kafka, cassandra, leveldb, influxdb, rocksdb, duckdb, monetdb, lmdb, sqlite, parquet, hbase, etc., are a lot more efficient at execution efficiency, though in many cases that's only 'up to such-and-such a scale' or 'above such-and-such a scale' or 'for such-and-such use cases'. there's a perhaps too comprehensive overview of the scene at https://github.com/newTendermint/awesome-bigdata but the big business inefficiency with oracle is not technical, it's the human problem of not having control over your core business processes or the ability to continue carrying them out if, when oracle says 'jump', you say anything but 'how high?'

One of the few times I wish we had dms here. :) I do not have many groups to offer, but for educational resources, here you go... Dig around on GitHub for Awesome Lists. There is so much out there: - https://github.com/sindresorhus/awesome#learn - https://github.com/mike-north/awesome-learn-to-code - https://github.com/lauragift21/awesome-learning-resources - https://github.com/afontcu/awesome-learning - https://github.com/yrgo/awesome-educational-games - https://github.com/notpresident35/awesome-learn-gamedev - https://github.com/RahulBirCodes/awesome-steam - https://github.com/Ghassen-Chaabouni/machine_learning_games - https://github.com/microsoft/Web-Dev-For-Beginners - https://github.com/microsoft/ML-For-Beginners Also check out this list Pitt puts out every year: - https://github.com/SimplifyJobs/Summer2024-Internships and make sure

From "Ask HN: Learning about distributed systems?" https://news.ycombinator.com/item?id=23932271 : > Papers-we-love > Distributed Systems: https://github.com/papers-we-love/papers-we-love/tree/master... > awesome-distributed-systems also has many links to theory: https://github.com/theanalyst/awesome-distributed-systems And links to more lists of distributed systems papers under "Meta Lists": https://github.com/theanalyst/awesome-distributed-systems#me... In reviewing this awesome list, today I learned about this playlist: "MIT 6.824 Distributed Systems (Spring 2020)" https://youtube.com/playlist?list=PLrw6a1wE39_tb2fErI4-WkMbs... > awesome-bigdata lists a number of tools: https://github.com/onurakpolat/awesome-bigdata

From a previous question re: "Ask HN: CS papers for software architecture and design?" (https://news.ycombinator.com/item?id=15778396 and distributed systems we eventually realize were needed in the first place: > Bulk Synchronous Parallel: https://en.wikipedia.org/wiki/Bulk_synchronous_parallel . Many/most (?) distributed systems can be described in terms of BSP primitives. > Paxos: https://en.wikipedia.org/wiki/Paxos_(computer_science) . > Raft: https://en.wikipedia.org/wiki/Raft_(computer_science) #Safety > CAP theorem: https://en.wikipedia.org/wiki/CAP_theorem . Papers-we-love > Distributed Systems: https://github.com/papers-we-love/papers-we-love/tree/master... awesome-distributed-systems also has many links to theory: https://github.com/theanalyst/awesome-distributed-systems - Byzantine fault: https://en.wikipedia.org/wiki/Byzantine

1) Design the reports you want. Pay special attention to interactive elements like filters and drilldowns. List all dimensions and metrics you need. Think about privacy. 2) Find your visualisation tool of choice. This is more important than any architecture choice for the tracking because this makes your data useable. [1] 3) Select your main data storage that is compatible with your visualisation tool, data size, budget, servers, security, ... SQL is always better because it has a schema the vis tools can work with. For a low amount of data you might just want to use your existing database (if you have one) and not build up new infrastructure that has to be maintained. 4) If you need higher availability on the data ingress than your db can provide use a high availability streaming ingress [2] to buffer the data. 5) Design a schema to conn

GitHub has a great list of big data stuff: https://github.com/onurakpolat/awesome-bigdata