GEgeophile·Oct 24, 2023
Sorry, osh as in object shell: https://github.com/geophile/osh.
In Marcel the Shellgeophile
osh
Python2013–2023
geophile / osh
osh
Osh (Object SHell) is a command-line and API toolkit combining cluster access, database access, and data slicing and dicing. Sort of like awk and cssh morsels wrapped up in a Python crust.
2Distinct people
15Mentions
0Explicit recommenders
59GitHub stars
What to know before exploring
What it is
Osh (Object SHell) is a command-line and API toolkit combining cluster access, database access, and data slicing and dicing. Sort of like awk and cssh morsels wrapped up in a Python crust.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year15 dated mentions
Observed mentions: 2013-02-05 21:32:12 UTC to 2023-10-24 15:24:51 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2013 | 2013-01-01 00:00:00 UTC | 2014-01-01 00:00:00 UTC | 1 | 1 | 0 |
| 2014 | 2014-01-01 00:00:00 UTC | 2015-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2015 | 2015-01-01 00:00:00 UTC | 2016-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2016 | 2016-01-01 00:00:00 UTC | 2017-01-01 00:00:00 UTC | 3 | 0 | 3 |
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 3 | 0 | 3 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 3 | 0 | 3 |
What Hacker News says
15 sourcesNo. Marcel is the successor to osh (http://geophile.com/osh, https://github.com/geophile/osh) which actually precedes powershell. The fact is that many many people have implemented the idea of a shell piping objects instead of strings.
In Marcel the ShellGitHub repo says it's obsolete: https://github.com/geophile/osh
In Object SHellTake a look at Object Shell: https://github.com/geophile/osh
In Computational survivalistI think they took one step in the right direction, where two more were needed.
The UNIX idea of piping text output from one command to the next was fine at the time, but it means that the downstream command has to parse text. With nushell, you are parsing slightly more structured text.
The two steps they could have taken in addition:
1. Instead of generating rows of a table, why not lists of strings? E.g. ("abc", "def", 123) instead of "abc | def | 123". Much easier to deal with in the command receiving this input.
2. And instead of just lists, why not objects? E.g. a process object, a file object, or, if you'd like a list as above.
I have implemented this idea: https://github.com/geophile/osh. For example, if you want to find processes in state 'S' with more than 3 descendents, and print the pid and command line of each, sorted by pid:
Read the rest
osh ps ^ select 'p: p.state == "S" and len(p.descendents) > 3' ^ f 'p: (p.pid, p.commandline)' ^ sort $
^ is used to denote piping. ps yields Process objects. Select examines those processes and keeps those with the right characteristics. Then f applies a function, mapping to the the pid and commandline in a list, and then sort sorts the lists, by pid (the first field of each list). $ does output.
Take a look at osh (Object SHell): https://github.com/geophile/osh
It is a Python implementation of this idea: OS objects like files and processes are represented in Python. You construct pipelines as in UNIX, but passing Python objects instead of strings. E.g. to find the pids of /bin/bash commands:
osh ps ^ select 'p: p.commandline.startswith("/bin/bash")' ^ f 'p: p.pid' $
- osh: Runs the tool, interpreting the rest of the line as an osh command.
- ^: Piping syntax.
- ps: Generate a stream of process objects, (the currently running processes).
- select: Select those processes, p, whose commandline starts with /bin/bash.
- f: Apply the function to the input stream and write function output to the output stream, (so, basically "map"). The function computes the pid of an input process.
- $ Print each item received in the input stream.
Read the rest
Osh also does database access (query results -> python tuples) and remote access. E.g., to get a process listing of (pid, commandline) on every node in a cluster:
osh @clustername [ ps ^ f 'p: (p.pid, p.commandline)' ] $
Along the same lines: https://github.com/geophile/osh
Osh (Object shell) is a command-line utility for running sequences of operations connected by a pipe-like construct. Objects are piped, not strings. Included in the sequence of operations may be remote sequences of command, which run on all nodes of a cluster. For example, if you have configured a cluster named foo, then you can get the pid and commandline of every node of your cluster like this:
osh @foo [ ps ^ f 'p: (p.pid, p.commandline)' ] $
- osh: Invokes the interpreter
- @foo [ ... ]: Run the bracketed commands on every node of cluster foo.
- ps: The osh ps command, which yields a stream of Process objects.
- ^: The osh pipe symbol
- f: Function application. The function in this case takes a Process p, and returns a tuple containing the process's pid and command line.
- $:
Read the rest
Print the contents of the input stream, one per line.
This is a python tool, so the stuff piped between operations are python objects.
APL, (which is the array language I had some exposure to), had three
interesting characteristics:
1. It implements an algebra: Operators take arrays as input and yield
arrays as output.
2. The special characters bound to those array operators.
3. It is functional.
#2 definitely gave APL programs a distinct and elegant personality,
but probably didn't help with widespread adoption.
But #1 and #3 are powerful ideas, and have showed up in other
forms. SETL was a set-oriented language from the late 70s/early 80s. I
took compiler courses from the guy behind it, R. B. K. Dewar (of
Spitbol fame). Fantastic teacher. SETL programs were concise and
powerful like APL programs, exactly because of #1 and #3.
Later in my grad school career, I got into relational algebra, which
also has these characteristics. My advisor was T. H. Merrett who
engaged on
Read the rest
a many-years, quixotic quest to explore relational algebra
as the basis of a general-purpose programming language. I didn't buy
into all of that, but as a database guy, I find relational algebra to
be powerful and useful on a day-to-day basis.
Finally, I really like functional stream approaches as a great way to
bridge set-at-a-time programming (for lack of a better term) with
ordinary, low-level programming. Java 8 streams and lambdas is a great
(if flawed) example of this approach. Even Map/Reduce is, under the
vast amounts of complexity.
I like this approach so much that I built a command-line tool, osh
(http://github.com/geophile/osh) based on it. The idea there is the
Unix idea of composing simple commands using pipes, except in osh, the
pipes carry streams of Python objects. And the commands that operate
on the objects in these streams are Python. E.g., to print the pids
and commands of processes whose commandline contains "emacs":
$ osh ps ^ select 'p: "emacs" in p.commandline' ^ f 'p: (p.pid, p.commandline)' $
(37268, '/usr/bin/emacs24 ')
(42107, 'emacs ')
(63758, '/usr/bin/python /usr/bin/osh ps ^ select p: "emacs" in p.commandline ^ f p: (p.pid, p.commandline) $ ')
(113605, '/usr/bin/emacs24 ')
In 2003 I started working on a distributed file store for archival, (an "object store"). I was responsible for the metadata management component, which was implemented by a Java distribution layer over a postgres database on each node.
I needed to operate on all nodes of the cluster. cssh opens a console on each node, and echoes commands to each. But that doesn't scale well past about 12 nodes, doesn't combine results from nodes, it really didn't work for me.
I wrote a tool I called Object Shell, or osh. The idea is to combine commands by passing data over pipe-like connectors. Osh is written in Python, and the commands are customized by Python functions. E.g., to find all the files recursively inside the current directory, and add up all the sizes:
osh ls -rf ^ f 'file: file.size' ^ red + $
osh: Invokes the tool
ls -rf: List files (
Read the rest
f) recursively (r), yielding a stream of Python file objects.
^: Pipe the stream to the next command.
f: Apply a function to each item in the input stream, writing function output to the output stream.
'file: file.size': A function that returns the size of a file.
red +: Reduction using +, combining all the numbers coming in.
$: Print result to stdout.
And then to do this for all files under /foo/bar, in all nodes of a cluster named mycluster, and to do a cluster-wide sum:
osh @mycluster [ ls -rf ^ f 'file: file.size' ^ red + ] ^ f 'host, sum: sum' ^ red + $
@mycluster [ ... ]: Distribute the bracketed code to each node of mycluster.
f 'host, sum: sum': The distributed command returns results that include the host name; drop the host name.
red +: Sum the sums from each node.
Osh is available here: https://github.com/geophile/osh
Check out osh: https://github.com/geophile/osh
In Skip grep, use awkhttps://github.com/geophile/osh
Osh (Object SHell) is a python application giving you a set of Linux like commands which can be composed similar to pipes. However, it is objects, not strings, that are passed from one command to the next.
It includes typical shell stuff, listing files and processes; database access, in which queries yield Python tuples; and distributed access, which distributes commands to the nodes of a cluster and then combines the results. For example, to submit a SQL query to each node of a cluster, getting a count on each, and combining the results:
osh @cluster [ sql "select count(*) from request where state = 'open'" ] ^ f 'node, count: count' ^ red + $
- osh: Invoke the interpreter.
- @cluster: Relay the bracketed command to each node of the cluster. The bracketed command returns (node, count) tuples.
- sql: S
Read the rest
ubmit a sql query (on a cluster node).
- ^: Denotes piping results from one command to the next.
- f: For each result from the cluster, run a function on (node, count) which returns just the count.
- red +: Reduce using +, summing up all the counts.
- $: Print result on the console.
Along the same lines, for Linux and MacOS: https://github.com/geophile/osh
In PowerShell is open sourced and is available on LinuxLooks a lot like osh (https://github.com/geophile/osh). Here is the osh version of the FizzBuzz example (run in a Linux shell):
osh gen 100 1 ^ f 'x: "FizzBuzz" if x % 15 == 0 else "Fizz" if x % 3 == 0 else "Buzz" if x % 5 == 0 else x' $
The function in single quotes is a python function, and in fact osh streams python tuples between commands (which are separated by ^).
While this example runs on the command line, there is also a python API.
In Matz's programming language based on prototype of streamAlong the same lines: https://github.com/geophile/osh
* Python expressions on the command line
* That are connected by piping
* But in a single python process
* And adding distribution
* And database access
In Pythonpy – Command-line Kung Fu with Python