MLmlboss·Sep 27, 2019Submission
2 points · Open discussiongithub
CodeSearchNet
Jupyter Notebook2019–2020
github / CodeSearchNet · Archived
CodeSearchNet
Datasets, tools, and benchmarks for representation learning of code.
AI & Machine LearningScience & EngineeringSystems & InfrastructureLanguages & CompilersLearning & ReferenceTesting & QualityNetworking
4Distinct people
6Mentions
0Explicit recommenders
2.4KGitHub stars
What to know before exploring
What it is
Datasets, tools, and benchmarks for representation learning of code.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 6:01:08 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by month6 dated mentions
Observed mentions: 2019-09-27 07:12:16 UTC to 2020-05-14 13:34:21 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Sep 2019 | 2019-09-01 00:00:00 UTC | 2019-10-01 00:00:00 UTC | 5 | 1 | 4 |
| Oct 2019 | 2019-10-01 00:00:00 UTC | 2019-11-01 00:00:00 UTC | 0 | 0 | 0 |
| Nov 2019 | 2019-11-01 00:00:00 UTC | 2019-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2019 | 2019-12-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2020 | 2020-01-01 00:00:00 UTC | 2020-02-01 00:00:00 UTC | 0 | 0 | 0 |
| Feb 2020 | 2020-02-01 00:00:00 UTC | 2020-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2020 | 2020-03-01 00:00:00 UTC | 2020-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2020 | 2020-04-01 00:00:00 UTC | 2020-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2020 | 2020-05-01 00:00:00 UTC | 2020-06-01 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
5 sourcesYou would need to extend the data-processing pipeline for the new language, which in the best case only requires to adapt the standard wrapper around the Tree Sitter parser. The wrapper needs to take care of language-specific details (e.g., where to find the documentation for a function, what methods should be filtered out, etc.) but can be fairly small. See https://github.com/github/CodeSearchNet/blob/master/function... for examples.
In GitHub releases an ImageNet for code and a CodeSearchNet challengeSince the CodeSearchNet Corpus contains metadata such as owner/repository, it would be nice to create a search tool for the data set itself. That way you could check if, by chance, some of your open source code is part of the corpus.
The data set is apparently ~20GB [0], so a cheap VPS instance might do the job of hosting the data in a searchable format.
[0] https://github.com/github/CodeSearchNet#downloading-data-fro...
In GitHub releases an ImageNet for code and a CodeSearchNet challenge[I'm one of the Microsoft Research people who worked on this]
We used the repo-level license information from github to filter to free non-copyleft licenses, and the license files are actually stored next to the extracted data corpus (see https://github.com/github/CodeSearchNet#Licenses).
In GitHub releases an ImageNet for code and a CodeSearchNet challenge[I'm one of the Microsoft Research people who worked on this]
All code is from public repositories. We only used publicly available APIs to obtain the data, so that others can reproduce the results / build on top of the data extraction pipeline we built: https://github.com/github/CodeSearchNet/tree/master/function...
In GitHub releases an ImageNet for code and a CodeSearchNet challenge