textteaser

There is also TextTeaser (https://github.com/MojoJolo/textteaser). There is a version in Python and Scala. I believe is behind the article summarization bots on reddit. I explored using it to generate talk summaries for https://www.findlectures.com and have been reasonably happy with it, although it's tough to get a summary that is succinct enough for search engine results.

TextTeaser (https://github.com/MojoJolo/textteaser). It's an automatic summarization algorithm I created.

In my opinion, automatic summarization by abstraction. There are lots of automatic summarization algorithm that does extraction like TextTeaser (https://github.com/MojoJolo/textteaser). I think they do the job well but it doesn't have a realistic feel in it. To achieve it, I think automatic summarization by abstraction is the way to go. It almost mimics how humans summarized text. But I think there is still no algorithm that can really perform abstraction well. I'm looking to try it in the near future. With a good training data, I'm looking into using genetic algorithm eventually evolving the text into an almost human summary. I'm excited for this!

An opensource bot - https://github.com/MojoJolo/textteaser/

Curiously, I find easier to read the scala code than the commented pseudo code. I've been experiencing this a lot lately. It' seems I am loosing my ability to reason about code that loops explicitly. One minor nitpick that can be of help when dealing with tuples: A partial function ( {case xxx => yyy} ) is a Function1 so you can use it with map and filter. This way you can deconstruct tuples into names and avoid using _1, _2, etc. { case (name, value) => blah } https://github.com/MojoJolo/textteaser/blob/master/src/main/... could be made more readable by giving names to the tuple elements. Thanks for publishing this code. It yields impressive results.

It's more of statistical NLP and a bit of machine learning. The algorithm can be found here: https://github.com/MojoJolo/textteaser/blob/master/src/main/...

It's mostly statistical (simple scoring). But as you can see in this lines of code: https://github.com/MojoJolo/textteaser/blob/master/src/main/... I keep track of the keywords used by the blog and category before. Through it, TextTeaser employs a little bit of machine learning to improve the quality of the results.

https://github.com/MojoJolo/textteaser/blob/master/src/main/... you post non-idiomatic(?) scala in a comment to explain what you are doing, i think? not a criticism of textteaser (which was behind this excellent project https://news.ycombinator.com/item?id=6498625), but seems to raise questions about the language...

Yes, I will open source it soon. Here's a teaser: https://github.com/MojoJolo/textteaser