elasticsearch-sql

You can see the OSS and Free offerings at https://www.elastic.co/subscriptions. Real question is what are the Open Distro is offering: * Alerting - you can use ElasticAlert * Security - Search Guard * SQL - https://github.com/NLPchina/elasticsearch-sql There very little reason (IMO) for users to a choose "Open Distro" except that it comes as an AWS image.

https://github.com/NLPchina/elasticsearch-sql/ https://www.elastic.co/products/stack/elasticsearch-sql

As someone who is involved in the ecosystem and maintains some open-source projects based on ElasticSearch, I am similarly curious about the move to not build upon existing projects (there could be a good reason, but haven't heard it yet). Why wasn't ElastAlert used / built upon for alerting? https://github.com/yelp/elastalert Or why wasn't ElasticSearch SQL used / built upon for SQL? https://github.com/NLPchina/elasticsearch-sql Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard All of them have Apache 2.0 license.

This is a good project to happen, and kudos to AWS for doing this! However, like some in the thread, I suspect AWS has been building this for a while for their own product and Elastic's license change may have influenced the decision to open-source. Getting a good out-of-the-box granular security has been a long overdue pain point with ElasticSearch, there have been good alternatives for the other problems it is replicating. 1.) ElastAlert - https://github.com/yelp/elastalert for alerting, 2.) ElasticSearch SQL - https://github.com/NLPchina/elasticsearch-sql for writing SQL queries. Would very much like to hear from OP why these weren't contributed to instead of creating another alternative. We've been working on a similar project in Golang and now well may be as good a day to open-source and put it out there: https://github.com/appbasei

Never used it but seems promising: https://github.com/NLPchina/elasticsearch-sql

I'm a fan of writing out queries using SQL. Here's a handy plugin to do that with Elasticsearch: https://github.com/NLPchina/elasticsearch-sql

Interesting — looks like the join isn't pipelined. The entire right-hand side evaluates synchronously. So it has to wait for the entire right-hand result set before it can evaluate the join operator, instead of streaming it concurrently. I'm surprised anyone would do it this way in Java, which has good support for concurrency. Edit: Actually the file you linked to was a test file. Hash join code is here [1], and it uses ES' scrolling feature to incrementally join, though it's not pipelined. Not sure scrolling is entirely appropriate for this; it will potentially hold an unpredictable amount of memory on the server end. [1] https://github.com/NLPchina/elasticsearch-sql/blob/5cd6ab639...

Joins are absent from the documented list of features and examples on the initial README but there's "limited support" for joins as described here: https://github.com/NLPchina/elasticsearch-sql/wiki/Join

It does have joins. https://github.com/NLPchina/elasticsearch-sql/blob/5cd6ab639...

https://github.com/NLPchina/elasticsearch-sql/ does exactly that with a reasonably good subset of SQL and ES query language mixed in. It operates in 2 modes; in one it runs the query, in another it spits back out what the equivalent ES JSON query is. We use this as a quick prototyping tool and modify the ES query as needed, as most of us here still "think" in SQL for a lot of things.

There is a great plugin for Elasticsearch that translates SQL into Elastic queries also [1]. I am unsure if it supports this new major version though. 1 - https://github.com/NLPchina/elasticsearch-sql/