HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
BurntSushi
xsv
Rust2015–2026
BurntSushi / xsv · Archived

xsv

A fast CSV command line toolkit written in Rust.

76Distinct people
103Mentions
4Explicit recommenders
10.8KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

A fast CSV command line toolkit written in Rust.

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

Yesss. Glad to see this getting some love. I use this tool very frequently in my day-to-day and my work would be so much harder without it. A lot of folks mentioning great…

nkcmr used a recommendation phrase. 4 indexed comments match. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 11:39:46 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year103 dated mentions

Observed mentions: 2015-02-22 09:10:33 UTC to 2026-01-19 06:08:33 UTC.

051015202015-01-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 6 mentions; 1 submissions, 5 comments20152016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 5 mentions; 1 submissions, 4 comments2017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 6 mentions; 1 submissions, 5 comments2018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 14 mentions; 1 submissions, 13 comments2019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 18 mentions; 0 submissions, 18 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 10 mentions; 0 submissions, 10 comments20202021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 9 mentions; 2 submissions, 7 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 12 mentions; 1 submissions, 11 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 13 mentions; 0 submissions, 13 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 7 mentions; 0 submissions, 7 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20152015-01-01 00:00:00 UTC2016-01-01 00:00:00 UTC615
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC514
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC615
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC14113
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC18018
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC10010
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC927
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC12111
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC13013
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC707
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC202
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC101

From the project README

An excerpt from the repository, fetched Nov 13, 2026.

Read the project’s introduction
xsv is now unmaintained In lieu of xsv, I'd recommend either qsv or xan. xsv is a command line program for indexing, slicing, analyzing, splitting and joining CSV files. Commands should be simple, fast and composable: Simple tasks should be easy. Performance trade offs should be exposed in the CLI interface. Composition should not come at the expense of performance. This README contains information on how to install xsv, in addition to a quick tour of several commands. Dual-licensed under MIT or the UNLICENSE. Available commands cat - Concatenate CSV files by row or by column. count - Count the rows in a CSV file. (Instantaneous with an index.) fixlengths - Force a CSV file to have same-length records by either padding or truncating them. flatten - A flattened view of CSV records. Useful for viewing one record at a time. e.g., xsv slice -i 5 data.csv | xsv flatten. fmt - Reformat CSV data with different delimiters, record terminators or quoting rules. (Supports ASCII delimited data.) frequency - Build frequency tables of each column in CSV data. (Uses parallelism to go faster if an index is present.) headers - Show the headers of CSV data. Or show the intersection of all headers between many CSV files. index - Create an index for a CSV file. This is very quick and provides constant time indexing into the CSV file. input - Read CSV data with exotic quoting/escaping rules. join - Inner, outer and cross joins. Uses a simple hash index to make it fast. partition - Partition CSV data based on a column value. sample - Randomly draw rows from CSV data using reservoir sampling (i.e., use memory proportional to the size of the sample). reverse - Reverse order of rows in CSV data. search - Run a regex over CSV data. Applies the regex to each field individually and shows only matching rows. select - Select or re-order columns from CSV data. slice - Slice rows from any part of a CSV file. When an index is present, this only has to parse the rows in the slice (instead of all rows leading up to the start of the slice). sort - Sort CSV data. split - Split one CSV file into many CSV files of N chunks. stats - Show basic types and statistics of each column in the CSV file. (i.e., mean, standard deviation, median, range, etc.) table - Show aligned output of any CSV data using elastic tabstops. A whirlwind tour Let's say you're playing with some of the data from the Data Science Toolkit, which contains several CSV files. Maybe you're interested in the population counts of each city in the world. So grab the data and start examining it: $ curl -LO https://burntsushi.net/stuff/worldcitiespop.csv $ xsv headers worldcitiespop.csv 1 Country 2 City 3 AccentCity 4 Region 5 Population 6 Latitude 7 Longitude The next thing you might want to do is get an overview of the kind of data that appears in each column. The stats command will do this for you: $ xsv stats worldcitiespop.csv --everything | xsv table field type min max min_length max_length mean stddev median mode cardinality Country Unicode ad zw 2 2 cn 234 City Unicode bab el ahmar Þykkvibaer 1 91 san jose 2351892 AccentCity Unicode Bâb el Ahmar ïn Bou Chella 1 91 San Antonio 2375760 Region Unicode 00 Z9 0 2 13 04 397 Population Integer 7 31480498 0 8 47719.570634 302885.559204 10779 28754 Latitude Float -54.933333 82.483333 1 12 27.188166 21.952614 32.497222 51.15 1038349 Longitude Float -179.983333 180 1 14 37.08886 63.22301 35.28 23.8 1167162 The xsv table command takes any CSV data and formats it into aligned columns using elastic tabstops. You'll notice that it even gets alignment right with respect to Unicode characters. So, this command takes about 12 seconds to run on my machine, but we can speed it up by creating an index and re-running the command: $ xsv index worldcitiespop.csv $ xsv stats worldcitiespop.csv --everything | xsv table ... Which cuts it down to about 8 seconds on my machine. (And creating the index takes less than 2 seconds.) Notably, the same type of "statistics" command in another CSV command line toolkit takes about 2 minutes to produce similar statistics on the same data set. Creating an index gives us more than just faster statistics gathering. It also makes slice operations extremely fast because only the sliced portion has to be parsed. For example, let's say you wanted to grab the last 10 records: $ xsv count worldcitiespop.csv 3173958 $ xsv slice worldcitiespop.csv -s 3173948 | xsv table Country City AccentCity Region Population Latitude Longitude zw zibalonkwe Zibalonkwe 06 -19.8333333 27.4666667 zw zibunkululu Zibunkululu 06 -19.6666667 27.6166667 zw ziga Ziga 06 -19.2166667 27.4833333 zw zikamanas village Zikamanas Village 00 -18.2166667 27.95 zw zimbabwe Zimbabwe 07 -20.2666667 30.9166667 zw zimre park Zimre Park 04 -17.8661111 31.2136111 zw ziyakamanas Ziyakamanas 00 -18.2166667 27.95 zw zizalisari Zizalisari 04 -17.7588889 31.0105556 zw zuzumba Zuzumba 06 -20.0333333 27.9333333 zw zvishavane Zvishavane 07 79876 -20.3
Read the full README on GitHub

What Hacker News says

12 sources
zJayv·Dec 20, 2022Submission

Ask HN: Programs that saved you 100 hours? (2022 edition)

I've asked this previously: https://news.ycombinator.com/item?id=22849208 But a few years have past and want to see what's emerged since. Although can be new to you vs. released in 2021 or 2022 necessarily. Things I've come across in the meantime: * mock AWS services https://github.com/spulec/moto * query cloud services https://github.com/turbot/steampipe * munge CSV https://github.com/johnkerl/miller https://github.com/BurntSushi/xsv * page json https://github.com/PaulJuliusMartinez/jless * text to tree structure [I use this as a hack to version control my music library] https://github.com/birchb1024/frangipanni
470 points · Open discussion
Nice article. I'm left wondering: - Are there any benchmarks available? - Is any full code example available (as opposed to a snippet)? - Does it handle real-world edge cases (unix vs dos line ends, non-"normal" quoting etc) - Does it normalize (e.g. `"hi"` and `hi` are both returned as `hi`)? - How does it compare, performance-wise (or in other ways) with https://github.com/BurntSushi/xsv, https://github.com/eBay/tsv-utils or https://github.com/liquidaty/zsv? (disclaimer: I wrote zsv)
In Splitting CSV files at 3GB/s
xsv does similar stuff for CSV, and very rapidly: https://github.com/BurntSushi/xsv https://miller.readthedocs.io/en/latest/why/ has a nice section on "why miller": > First: there are tools like xsv which handles CSV marvelously and jq which handles JSON marvelously, and so on -- but I over the years of my career in the software industry I've found myself, and others, doing a lot of ad-hoc things which really were fundamentally the same except for format. So the number one thing about Miller is doing common things while supporting multiple formats: (a) ingest a list of records where a record is a list of key-value pairs (however represented in the input files); (b) transform that stream of records; (c) emit the transformed stream -- either in the same format as input, or in a different format.
In Modernizing AWK, a 45-year old language, by adding CSV support
FYI I ran on the worldcities data at https://github.com/petewarden/dstkdata (credit to xsv for choosing that dataset) against https://github.com/BurntSushi/xsv and https://github.com/liquidaty/zsv (full disclosure: I am one of the zsv authors). Here's what I got. fastest to slowest: zsv (0.07), xsv (0.16), goawk (0.42), python (~1.6) Obviously, does not tell the whole story as this test was limited to "count" and an interpreted language is expected to always be slower compared to a precompiled command, but, it might be relevant to a user deciding what tool to use. Also, might be instructive as to room for improvement in the go code (or possibly the go code could use the c lib)-- I note that even if the goawk command is '{}' the runtime is still about the same. full results: --- goawk: 1000001 7000007 real 0m0.435s user 0m0.435s sys 0m0.03
Read the rest
1s 1000001 7000007 real 0m0.413s user 0m0.419s sys 0m0.024s 1000001 7000007 real 0m0.425s user 0m0.430s sys 0m0.024s xsv: 1000000 real 0m0.157s user 0m0.141s sys 0m0.013s 1000000 real 0m0.156s user 0m0.141s sys 0m0.012s 1000000 real 0m0.158s user 0m0.142s sys 0m0.013s zsv: 1000000 real 0m0.066s user 0m0.053s sys 0m0.010s 1000000 real 0m0.077s user 0m0.060s sys 0m0.012s 1000000 real 0m0.069s user 0m0.056s sys 0m0.010s python: 1000001 7000007 real 0m1.589s user 0m1.553s sys 0m0.026s 1000001 7000007 real 0m1.583s user 0m1.550s sys 0m0.025s 1000001 7000007 real 0m2.122s user 0m1.675s sys 0m0.037s --- The script for this was: --- echo 'goawk:' (time goawk -i csv '{ w+=NF } END { print NR, w }' < worldcitiespop_mil.csv) 2>&1 | xargs (time goawk -i csv '{ w+=NF } END { print NR, w }' < worldcitiespop_mil.csv) 2>&1 | xargs (time goawk -i csv '{ w+=NF } END { print NR, w }' < worldcitiespop_mil.csv) 2>&1 | xargs echo 'xsv:' (time xsv count < worldcitiespop_mil.csv) 2>&1 | xargs (time xsv count < worldcitiespop_mil.csv) 2>&1 | xargs (time xsv count < worldcitiespop_mil.csv) 2>&1 | xargs echo 'zsv:' (time zsv count < worldcitiespop_mil.csv) 2>&1 | xargs (time zsv count < worldcitiespop_mil.csv) 2>&1 | xargs (time zsv count < worldcitiespop_mil.csv) 2>&1 | xargs echo 'python:' (time python3 count.py < worldcitiespop_mil.csv) 2>&1 | xargs (time python3 count.py < worldcitiespop_mil.csv) 2>&1 | xargs (time python3 count.py < worldcitiespop_mil.csv) 2>&1 | xargs
In Modernizing AWK, a 45-year old language, by adding CSV support
Assuming you want to do this on a single machine with finite memory, a feasible approach would be: 1. Split into multiple shard using e.g. `split`, where each piece is small enough to fit comfortably into memory 2. Sort each shard. You could use something like `zsv` (sql sub-command) (https://github.com/liquidaty/zsv) or `xsv` (sort subcommand) (https://github.com/BurntSushi/xsv) for this 3. Use mergesort to sort all shards, at the same time, into a single output table, and in the process, de-dupe and/or merge (if you had too many shards for your operating system to allow so many files to be open at once, you might need to do an intermediate merge to consolidate shards) For #3, I don't know of a tool to do this-- probably one exists for simple de-duping, but it may be harder if you need support for merge logic. If a tool for this does not
Read the rest
exist, I would be imagine `zsv` could quite be easily extended to handle it.
In Ask HN: How to Deal with CSVs?

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.