dolly

I'm going through the dataset with your datasette tool and it looks like it might be a good idea to clean things up a bit. There are many duplicates[1], creepypastas[2] and other strange things in there. [1] https://lite.datasette.io/?json=https%3A%2F%2Fraw.githubuser... [2] https://lite.datasette.io/?json=https://github.com/databrick... EDIT: Maybe I'm passing link wrong, the query I'm using is select count(instruction), instruction, group_concat(context, ' ============= ') as c, group_concat(response, ' ============= ') as r, group_concat(category, ' ============= ') as cat from [databricks-dolly-15k] group by instruction having count(instruction)>1 order by count(instruction)desc limit 100 [databricks-dolly-15k] should be the name of dataset, first column is the number of instruction duplicates Creepypastas are responses to instructio

Can someone help me to understand why categories for these two differ? row #51 "Think of some family rules to promote a healthy family relationship" - brainstorsming [1] row #68 "What is the future for human?" - general_qa [2] In nature they both are brainstorming to me - does the question mark is what assigned the #68 as _qa? [1] https://lite.datasette.io/?json=https://github.com/databrick... [2] https://lite.datasette.io/?json=https://github.com/databrick...

Here's a link to open up and explore that training data in Datasette Lite: https://lite.datasette.io/?json=https://github.com/databrick...

https://github.com/databrickslabs/dolly it’s now available on GitHub

Full source code is up here now: https://github.com/databrickslabs/dolly Sorry it took us a day to get the external repo setup.