HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
oils-for-unix
oils
Python2016–2025
oils-for-unix / oils

oils

Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!

35Distinct people
225Mentions
1Explicit recommenders
3.4KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

There are Usenix papers on bash and ksh -- I archived the ksh one here: http://www.oilshell.org/archive/ksh-usenix.pdf The surprising thing to me is that ksh was Lua-like in…

chubot used a recommendation phrase. 4 indexed comments match. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 4:59:52 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year225 dated mentions

Observed mentions: 2016-11-26 10:24:57 UTC to 2025-11-06 22:34:08 UTC.

02040602016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments20162017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 13 mentions; 0 submissions, 13 comments2018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 9 mentions; 0 submissions, 9 comments2019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 27 mentions; 2 submissions, 25 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 52 mentions; 3 submissions, 49 comments20202021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 41 mentions; 2 submissions, 39 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 27 mentions; 3 submissions, 24 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 16 mentions; 1 submissions, 15 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 17 mentions; 2 submissions, 15 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 20 mentions; 2 submissions, 18 comments2025

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC303
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC13013
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC909
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC27225
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC52349
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC41239
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC27324
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC16115
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC17215
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC20218

What Hacker News says

52 sources
I'd say that JSON is "structured", and probably HCL as far as I know, but the others have problems. I haven't worked with an API that uses XML in a long time, but does it express the difference between "0123" and 123 well? I think that it either doesn't, or you add some really awkward schema language with XML namespaces ... I just translated my .travis.yml to JSON, and I learned that deploy: on: branch: master gets translated to {"deploy": {"true": {"branch": "master"}}} because "on" is a synonym for true. I'd call that unstructured. It also has problems with keys that start with -, etc. Also, all of the languages are often used with say Go templates, which makes them more unstructured. They're no longer statically parsable, which is "table stakes" for doing anything else (validation or static analysis). I took some
Read the rest
pains to make shell statically parsable [2], and I think it's now a better foundation than these config languages, per my other comment here: https://news.ycombinator.com/item?id=25483270 I think HCL is probably a lot better, but it also seems to be the least used! It's structured, and has its own mechanism for expressing duplication that's not the equivalent of a 70's-style #include. [1] https://github.com/oilshell/oil/blob/master/.travis.yml [2] http://www.oilshell.org/blog/2019/02/07.html
In Back to the '70s with Serverless
Yes, the lobste.rs thread also has that feedback: Github Actions is a tremendously useful platform, but the syntax and tooling sucks! Since JSON is valid YAML, it's possible to generate JSON from an Oil configuration with embedded and parsed shell, rather than using YAML. (Actually there's no problem generating YAML either, but I suppose it's easier to think about.) Example: I changed my own configs to use .yml.in and .yml and it worked on both Travis CI and sourcehut. https://github.com/oilshell/oil/tree/master/.builds Some brainstorming here on this feature: https://github.com/oilshell/oil/wiki/Config-Dialect The benefit would be that it would validate your syntax and possibly your schema locally, not remotely. Some services have command line tools, but I'm sure they don't validate your shell, only the configuration. Another benefit i
Read the rest
s the ability to reuse code and configuration. For example, my .builds/ dir has a fair bit of redundancy in its configuration, and that could be "factored" with a real language. YAML has a limited << reference mechanism that does some of this. I'm looking for feedback on Oil, as well as the YAML + shell replacement idea. I hope that Oil can be a flexible language that will let people write these sorts of front ends for many cloud services, i.e. "distributed shell scripts". And eventually enable portability across clouds too. https://github.com/oilshell/oil/wiki/Where-To-Send-Feedback
In Become Shell Literate
Yeah the way I think of this is that some people write a text or Markdown file with shell commands for documentation. I INVERT this, and write a shell script with comments :) That way someone else can reuse your knowledge more easily (and your future self as well). Examples: Constructing a big curl command to use the Zulip API, and also using jq for the first time. I used this to easily make a blog post out of a long Zulip thread [1] https://github.com/oilshell/oil/blob/master/services/zulip.s... (oops some tabs snuck in here) Figuring out how to use uftrace (and successfully optimizing Oil with it [2]): https://github.com/oilshell/oil/blob/master/benchmarks/uftra... Though one issue is that shell scripts don't really specify their environment, but there is a large number of tools growing around containers that can solve this problem.
Read the rest
(Basically Docker is being refactored into something more sane; thank you to OCI and others.) So I hope to integrate the Oil shell and containers more so shell scripts are more reproducible. I mean most of the container tools are already command line tools so in some sense it's already done, but you can imagine closer integration (e.g. not just passing code as text from outside the container to inside the container). ---- I wrote some notes about the documentation issue here: http://www.oilshell.org/blog/2020/02/good-parts-sketch.html#... And one thing I've wondered is if Oil should literally run shell out of markdown, so you can create executable docs. I can see it being useful, but it might be something you should do with a separate tool that converts markdown code blocks to a shell script... [1] http://www.oilshell.org/blog/2020/11/more-syntax.html [2] http://www.oilshell.org/blog/2020/01/parser-benchmarks.html
In Become Shell Literate
I just ported the continuous build for https://www.oilshell.org/ to sr.ht for this reason: http://www.oilshell.org/blog/2020/11/fixes-and-updates.html#... A contributor added .travis.yml about 3 years ago, before I had ever used it. But I've been around the block enough to know that getting stuff for free is temporary. (And to be fair, I did like Travis CI free service a lot better than I thought I would.) So when I needed to enhance the continuous build back in January, I did it with a PORTABLE SHELL SCRIPT, NOT with yaml. Both Travis CI and sr.ht provide Linux VMs, which are easily operated with a shell script. The script called "Toil" does the following: 1. Configures which steps are run in which build tasks (both Travis CI and sr.ht can run multiple tasks in parallel for each commit) 2. Logs each step, times it, summarizes failure/
Read the rest
success 3. Archives/compresses the logs 3. Publishes the result to my own domain, which is A LOT FASTER than the Travis CI dashboard. (sr.ht is very fast too; it has a great web design.) This ended up working great, so I have multiple CI services running the same jobs, and publishing to the same place: http://travis-ci.oilshell.org/ (I plan to create srht.oilshell.org for security reasons; it isn't ideal that both services have a key to publish to the same domain.) ---- I think this is the future of the Oil project: shell scripts to enable portability across clouds. If you want to be fancy, it's a distributed or decentralized shell. This is natural because shell already coordinates processes on a single machine. - A distributed shell coordinates processes across multiple machines (under the same domain of trust) - A decentralized one does so across domains of trust (across clouds) ----- Really great work in this direction is gg: https://buttondown.email/nelhage/archive/papers-i-love-gg/ comments: https://lobste.rs/s/virbxa/papers_i_love_gg which is a tool that runs distributed programs across multiple FaaS providers like Amazon Lambda, Google Cloud Functions, etc. https://github.com/StanfordSNR/gg My "toil" script is a lot more basic, but an analogous idea. I would like to create a slightly tighter but improved abstraction that runs on multiple cloud services. Notes on gg here: https://github.com/oilshell/oil/wiki/Distributed-Shell If anyone wants to help, get in touch! If you are pissed off about Travis then you might want this :) I think these kinds of multi-cloud setups are inevitable given the incentives and resources of each party, and they already exist (probably in a pretty ugly/fragile form).
In Travis CI is no longer providing CI minutes for open source projects
I collected some experiments in that area here (and anyone should feel free to add more, including their own): https://github.com/oilshell/oil/wiki/Interactive-Shell Many of the articles have nice screenshots / animations to give you a feel for what it is. I plan to turn https://www.oilshell.org/ into a library and I hope people will build GUIs around it. There are some UI possibilities that are difficult or impossible unless you have hooks to the shell other than stdin/stdout. https://github.com/oilshell/oil/issues/738
In A terminal-based workflow for research, writing, and programming
Ultimately there’s no format that corrects all of CSVs flaws while retaining its benefits. I'm planning to fix this with https://www.oilshell.org/ ! Eliminating the need for ad hoc parsing and splitting is one of the Four Features That Justify a New Unix Shell: http://www.oilshell.org/blog/2020/10/osh-features.html I've designed a format called QTSV (quoted TSV) which lets you embed newlines and tabs in fields. It's built on QSN (quoted string notation), which are essentially just Rust string literals: http://www.oilshell.org/release/latest/doc/qsn.html QSN is fully implemented, but QTSV isn't done yet. Proposal here: https://github.com/oilshell/oil/wiki/TSV2-Proposal However the latest Oil release actually does emit QTSV! The "pp" builtin (for pretty-print) will print a QTSV file of shell function names (procs) and their doc comments
Read the rest
(like docstrings). ---- If you'd like this to be a reality, contact me and help out! Useful things to do: - write a messy CSV to strict QTSV converter (it will need some flags for the heuristics, like Python's CSV library) - write a QTSV library in your language [1] https://www.oilshell.org/release/0.8.4/
In CSVs: The Good, the Bad, and the Ugly
A bunch of points: (1) Except when your tokens can contain newlines, which we're stuck with for the foreseeable future. -- not sure what you mean here, because QSN strings are defined not to contain literal newlines. They're escaped like '\n'. The invariant of QSN is: EVERY BYTE STRING, including those with newlines and nuls, can be represented on a single line. I guess I should put this in the documentation. (2) A QSN decoder is very easy to write. For example, here's a ~6 line regex that validates all of QSN: https://github.com/oilshell/oil/blob/master/qsn_/qsn.py#L498 True, you need like ~20 lines of code to decode it, but that's very easy too. You can also make a QSN decoder from a JSON string decoder. It's basically changing your tests and moving a few statements around. (And yes, QSN is a regular language [1]) (3) Oil should g
Read the rest
row [2] an awk-like dialect [3] that understands QSN and QTSV. (4) Although you also don't have to decode it for it to be useful. 1. I can use wc -l on a stream of QSN strings 2. If I know the strings are QSN-encoded, I can search for NUL bytes with fgrep '\0' It's basically like the UTF-8 philosophy. ASCII is valid utf-8. QSN lines are lines of text. (5) base64 is bad for humans at the terminal, because it makes everything unreadable. QSN preserves all printable ASCII and unicode. ----- I don't think it's going to solve all problems, but it will solve some. It's there if you need it. In typical Unix style, the solutions will be heterogeneous. You can absolutely use the NUL format in Oil, and it now has support for it. [1] http://www.oilshell.org/blog/2020/07/eggex-theory.html [2] That is, if I get help :) [3] http://www.oilshell.org/blog/tags.html?tag=awk#awk
In Four features that justify a new Unix shell
chubot·Oct 24, 2020Recommendation phrase
(author here) I do use Python. In fact I wrote something like 30K lines of Python for Oil. The question isn't shell OR Python. My shell scripts call Python scripts, many of white I wrote myself. That is working as intended. They also call C programs, C++ programs and R programs. And put JavaScript programs in various places. ---- I guess this is a huge misconception about shell that I have to write a blog post about. https://news.ycombinator.com/item?id=24083764 I'm often asked this about Oil [1]: Why do you want to write programs in shell? That's not the idea of shell. The idea is that I write programs in Python, JavaScript, R, and C++ regularly, and about 10 different DSLs (SQL, HTML, etc.) And I work on systems written by others, consisting of even more languages. I need a language to glue them together. A language to express build
Read the rest
automation and describe deployed systems. Most big systems consist of more than one language. Shell is the best language for that, but it's also old and crufty, with poor implementations. When you program in shell, gcc, git, pip, npm, markdown, rsync, diff, perf, strace, etc. are part of your "standard library". ----- If you want some concrete examples, look in the Oil repo. There are dozens of shell scripts that invoke custom tools in Python, R, and C++. https://github.com/oilshell/oil/tree/master/benchmarks For example, to generate this report on how much we speed up Python by translating it to C++: https://www.oilshell.org/release/0.8.3/benchmarks.wwz/mycpp-... The tables are manipulated by R code, and shell/Python/CommonMark generates the HTML. ----- Another example is that the release page is essentially a huge shell script: https://www.oilshell.org/release/0.8.3/ This page and all the linked pages are generated by: https://github.com/oilshell/oil/blob/master/devtools/release... ------ If you don't understand why shell should call Python and C, I recommend reading The Art of Unix Programming http://www.catb.org/~esr/writings/taoup/html/ https://www.amazon.com/UNIX-Programming-Addison-Wesley-Profe...
In Four features that justify a new Unix shell
Yes absolutely! That's been on the table for along time, and we even have a nascent xargs from a contributor (that has languished -- I need help!) Let's discuss on the issue if interested: https://github.com/oilshell/oil/issues/843 Oil is easy to prototype (it's just Python code), so anyone who wants to make this feature their own should feel free to dive in :) A lot of people have ideas, which may or may not be implementable with fork(), wait(), pipe() and dup(). Anything that can be done safely with those syscalls is basically viable. The syntax is generally the easy part compared to the runtime execution. Oil has first-class blocks already which will help a lot. ---- Related: I also want to fix redirect and here doc syntax (in a compatible way): https://github.com/oilshell/oil/issues/841 https://github.com/oilshell/oil/issues/832
In Four features that justify a new Unix shell
Right now, I use osh interactively in a very bash compatible mode. Then at the top of my scripts I put this if I still want to run with bash: shopt -s strict:all 2>/dev/null || true Or this if I don't need to run with bash anymore: shopt --set oil:basic Example: https://github.com/oilshell/oil/blob/master/test/spec.sh#L9 docs in progress: https://www.oilshell.org/release/0.8.3/doc/oil-options.html Having the OSH runtime exist everywhere is something we should work on: https://github.com/oilshell/oil/issues/463
In Four features that justify a new Unix shell
Yes, I would like for that to happen: Shell as an engine for TUI or GUI https://github.com/oilshell/oil/issues/738 C++ code should be embeddable in another program https://github.com/oilshell/oil/issues/822 I think you can already use bash in Jupyter, but the integration is necessarily kind of coarse. I don't know exactly what a good integration is, but I'm looking for people interested in UIs to help me figure it out. At the very least, you can use Oil's parser to provide completions: http://www.oilshell.org/blog/tags.html?tag=interactive-shell... (which is a separate integration point that running commands) Also, I mentioned in the blog post that there's some interest from maintainers in combining the fish shell and Oil. However someone has to do that work, and there's a lot of it! https://github.com/oilshell/oil/wiki/Where-To-Send-Fe
Read the rest
edback ---- Also, I use shell scripts as a productivity tool like a Jupyter notebook: http://www.oilshell.org/blog/2020/02/good-parts-sketch.html#... I think we need to come up with a clever name for it -- some people call it "Taskfile", "go script", and I call it "run.sh". The basic idea is that I don't remember how to do things with computers, because there are too many things to do. I just remember WHERE I WROTE IT DOWN in a shell script. Some people write their notes in text files; I write them in executable shell scripts.
In Four features that justify a new Unix shell
(author here) Yes, I actually proposed that in the comments [1]. Some should add command_sub_errexit to bash: echo $(date %x) # can you tell what's wrong here? echo 'script should fail before this' local d=$(date %x) echo 'script should fail before this' And process_sub_fail: diff <(sort left.txt) <(sort /oops/error) echo 'script should fail before this' As mentioned in the comment, bash doesn't even wait() on process subs! The 'run' builtin to fix the "if myfunc" problem should also be adopted by other shells. POSIX basically calcified broken language semantics and needs to be fixed. I also document stuff like shopt -s simple_word_eval for other shell implementers: https://www.oilshell.org/release/0.8.3/doc/simple-word-eval.... ----- BTW Oil's code is very short (see comment), and I'm looking for help :) L
Read the rest
et me know if you can't get a bin/osh working in 1 to 5 minutes. (If you have a Debian/Ubuntu-ish machine it should take about 1 minute.) This is a pure Python program that you can quickly modify/prototype, and then for the release it's translated into C++ for a 30-50x speedup [2]. https://github.com/oilshell/oil/wiki/Contributing [1] https://lobste.rs/s/qfiki1/four_features_justify_new_unix_sh... [2] http://www.oilshell.org/blog/2020/01/parser-benchmarks.html
In Four features that justify a new Unix shell

Showing the 15 most recent of 52 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.