VIvistarchris·Dec 27, 2016
I've never tried it myself, but there is this standlone JAR for dumping columns from a parquet file:
https://github.com/Parquet/parquet-mr/tree/master/parquet-to...
In Indexing JSON logs with Parquetparquet
parquet-mr
HISTORICAL RECORD2013–2016
parquet / parquet-mr · Not publicly available
parquet-mr
Discover this repository through its Hacker News conversations.
2Distinct people
3Mentions
0Explicit recommenders
What to know before exploring
What it is
A public description has not been verified.
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year3 dated mentions
Observed mentions: 2013-08-07 02:26:55 UTC to 2016-12-27 20:37:46 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2013 | 2013-01-01 00:00:00 UTC | 2014-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2014 | 2014-01-01 00:00:00 UTC | 2015-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2015 | 2015-01-01 00:00:00 UTC | 2016-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2016 | 2016-01-01 00:00:00 UTC | 2017-01-01 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
3 sourcesImpala isn't technically Cloudera only, it's open source (https://github.com/cloudera/impala), and other people have gotten it to run on their Hadoop distribution, but since it's developed by Cloudera, it was developed to run on the CDH platform (Hadoop).
Parquet was a joint effort between Cloudera and Twitter, and now it's being developed by many other companies. You can use it with Hive, Pig, MapReduce, Cascading, Crunch and I think Apache Drill's first milestone has adopted it as a columnar format as well. Parquet also allows you to use your Avro or Thrift schema (soon Protobuffs) to write Parquet data, too.
It's a separate project in the ecosystem and has its own roadmap (https://github.com/Parquet/parquet-mr).
In Presto: Interacting with petabytes of data at FacebookNo problem, glad I could help. Your use case sounds pretty interesting, HDFS should fit the bill for sure.
You should take a look at Parquet (http://parquet.io/) for storing your data. It's an open source columnar format that was designed for Hadoop, it even supports nesting (https://github.com/Parquet/parquet-mr/wiki/The-striping-and-... <-- really interesting). Also, it already works with a lot of the Hadoop ecosystem components (MR, Hive, Pig, Cascading, Impala, ..), so your data doesn't have to move once it's in HDFS.
Good luck!
In Big Data Debate: HBase