awesome-public-datasets

One of the few times I wish we had dms here. :) I do not have many groups to offer, but for educational resources, here you go... Dig around on GitHub for Awesome Lists. There is so much out there: - https://github.com/sindresorhus/awesome#learn - https://github.com/mike-north/awesome-learn-to-code - https://github.com/lauragift21/awesome-learning-resources - https://github.com/afontcu/awesome-learning - https://github.com/yrgo/awesome-educational-games - https://github.com/notpresident35/awesome-learn-gamedev - https://github.com/RahulBirCodes/awesome-steam - https://github.com/Ghassen-Chaabouni/machine_learning_games - https://github.com/microsoft/Web-Dev-For-Beginners - https://github.com/microsoft/ML-For-Beginners Also check out this list Pitt puts out every year: - https://github.com/SimplifyJobs/Summer2024-Internships and make sure

Merry Christmas buddy. You'll find a ton of public datasets on GitHub [1]. Maven Analytics offers a monthly data analytics challenge [2] that you can enter for free. See their past competitions for some interesting datasets. As I'm based in Ireland I'll also recommend the Irish Data Portal [3]. [1] https://github.com/awesomedata/awesome-public-datasets [2] https://mavenanalytics.io/challenges [3] https://data.gov.ie/

https://datasets.reddit.com https://opendata.reddit.com https://archive.ics.uci.edu/ml/datasets.php https://lod-cloud.net/ https://www.data.gov https://data.un.org/ https://data.worldbank.org/ https://fred.stlouisfed.org/ https://data.oecd.org/ https://www.nber.org/research/data?page=1&perPage=50 https://github.com/awesomedata/awesome-public-datasets https://github.com/datasets https://opendata.cern.ch/ https://data.nasa.gov/ https://data.world/datasets/machine-learning https://data.noaa.gov/datasetsearch/ https://www.usgs.gov/products/data https://www.fema.gov/about/openfema/data-sets etc... And of course don't ignore the data you can collect yourself one way or another. A few cheap Arduino Nano or Rpi Pico boards, some sensors, and you can build quite a variety of distributed data collection systems. Use solar panels for power in remote

You are exactly right! There does not seem to be much out there. I usually do hash or otherwise transform real-world (e.g., https://github.com/awesomedata/awesome-public-datasets) data to retain basic properties like skewness utf8 well-formedness, cardinality, and such. Like many others, I often use the Enron eMail dataset, NYC Taxi, or Twitter data (for coherent use of Unicode). And then scale it either by combining attributes randomly or otherwise augmenting it. Earlier in my career, I wrote a regular expression inversion data generator; it quickly spiraled out of control complexity-wise ;-).

> As part of my learning in data science, I need/want to gather data. Also not web scraping, but a few other public data set sources to check. https://registry.opendata.aws https://github.com/awesomedata/awesome-public-datasets

This is a good list: https://github.com/awesomedata/awesome-public-datasets

> What is the open source alternative of this? There are many public-domain datasets: * https://aws.amazon.com/opendata/public-datasets/ * https://cloud.google.com/public-datasets/ * https://github.com/awesomedata/awesome-public-datasets > Data is so cheap that it should be actually free I feel like you haven't had the opportunity to work with data that has value.

Crawlers and anti-crawler wasted too much resource, data shared in open database is a great idea. BTW, if someone can maintain all data listed here, that will be really helpful. https://github.com/awesomedata/awesome-public-datasets

The link at the bottom should be emphasized: https://github.com/awesomedata/awesome-public-datasets It is a very expansive collection of datasets, some well-prepped for ML and most not (which is part of the fun of it, anyways).