(1) everything is text
(2) everything (ish) is a file
(3) including pipes and fds
(4) every piece of software is accessible as a file, invoked at the command line
(5) ...with local arguments
(6) ...and persistent globals in the environment
A lot of understanding comes once you know what execve does, though such knowledge is of course not necessary. It just helps.
Unix is seriously uncool with young people at the moment. I intend to turn that around and articles like this offer good material.
And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them.
There are two uses of the Unix “api”:
[A] Long lived tools for other people to use.
[B] Short lived tools one throws together oneself.
The fact that most things work most of the time is why the shell works so well for B, and why it is indeed a poor choice for the sort of stable tools designed for others to use, in A.
The ubiquity of the C APIs of course solved [A] use cases in the past, when it was unconscionable to operate a system without cc(1). It’s part of why they get first class treatment in the Unix man pages, as old fashioned as that seems nowadays.
Source: I wrote a shell and it solves a great many of the space / quoting problems with POSIX shells.
luck, or simply execute the steps you want to check by typing them on the command line. I find the pipes approach incredibly powerful and simple because you can compose and check each step, and assemble. That's really the point, and the power, of a simple and extensible approach.
echo "foo bar fnord" | cut --delimiter " " --output-delimiter ":" -f1-
# foo:bar:fnordXML? SMH.
Well, that's easy. Don't escape or quote the strings; encode them. Turn them into opaque tokens, and then do Unix things to the opaque tokens, before finally decoding them back to being strings.
There's a reason od(1) is in coreutils. It's a Unix fundamental when working with arbitrary data. Hex and base64 are your friends (and Unix tools are happy to deal with both.)
Everything is a byte stream. Usually that means text but sometimes it doesn't. Which means you can do fun stuff like:
- copy file systems over a network: https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.htm...
- stream a file into gzip
- backup or restore an SD card using `cat`
See one here: https://unix.stackexchange.com/questions/6852/best-way-to-re...
Those damn kids with their loud rockn'roll music and their Windows machines. Back in my day we had he vocal stylings of Dean Martin and the verbal stylings of Linus Torvalds let me tell ya.
Seriously though, I'm actually seeing younger engineers really taking the time to learn how to do shell magic, using vim, etc. It's like the generation of programmers who started up until the late 90s used those by default, people who like me started 15-20 years ago grew up on IDEs and GUIs, but I've seen a lot of 20-30 something devs who are really into the Unix way of doing things. The really cool kids are in fact doing it.
Giving up the idea that CLI = pain (i.e figuring out how to to navigate a file system, ssh keys, etc) for sure was a learning curve, but now I can't imagine using computers without it.
Not at all, you can pipe around all the binary you want. Until GNU tar added the 'z' option, the way to extract all files in a tarball was:
`gunzip -c < foo.tar.gz | tar x`
However, "text files" in Unix do have a very specific definition, if you want them to work with standard Unix text manipulation utilities like awk, sed, and diff:
All lines of text end are terminated by a line feed, even the last one.
I can't tell you how many "text files" I run across these days that don't have a trailing newline. It's as bad as mixing tabs and spaces, maybe worse.
While posix and streams are nice (if you squint, files look like a poor man’s version of objects imposed on top of bytestreams), it’s about time we moved on to better things, and not lose sight of the bigger picture.
Objects aren't human readable and if they get corrupted the object's format is very difficult to recover, you need an intimate knowledge of both the specific format used (of which there might be thousands of variations). Plaintext, however, if that gets corrupted it's human-readable. The data might not be more compact (Although TSV files tend to come out equal), but it's more robust against those kinds of changes.
Have you ever examined a protobuf file without knowing what the protobuf structure is? I have, as part of various reverse-engineering I did. It's a nightmare, and without documentation (That's frequently out of date or nonexistent) it's almost impossible to figure out what the data actually is, and you never know if you've got it right. Even having part of the recovered data, I can't figure out what the rest of it is.
It is allowed, you can pass around data in whatever encoding you desire. Not many do though because text is so useful for humans.
> You know what might be even better — passing around objects you could interact with by passing messages (gasp! cue, Alan Kay)
That's a daemon/service and basic unix utils make it easy to create, a shell script reading from a FIFO file fits this definition of object, the message passing is writing to the file and the object is passed around with the filename. Unix is basically a fulfillment of Alan Kay's idea of OO.
Well, it is allowed. You can of course serialize some data into a structured format and pass it over a byte stream in POSIX to this end, and I think this is appropriate in some cases in terms of user convenience. Then you can use tools like xpath or jq to select subfields or mangle.
> (if you squint, files look like a poor man’s version of objects imposed on top of bytestreams)
If all you have is a hammer...
It's wonderful only if compared to worse things, pretending that PowerShell is not a thing, and that Python doesn't exist.
UNIX pipes are a stringly-typed legacy that we've inherited from the 1970s. The technical constraints of its past have been internalised by its proponents, and lauded as benefits.
To put it most succinctly, the "byte stream" nature of UNIX pipes means that any command that does anything high-level such as processing "structured" data must have a parsing step on its input, and then a serialization step after its internal processing.
The myth of UNIX pipes is that it works like this:
process1 | process2 | process3 | ...
(process1,serialize) | (parse,process2,serialize) | ...
This is not a limitation of PowerShell, which is object oriented, and passes strongly typed, structured objects between processing steps. It makes it much more elegant, flexible, and in effect "more UNIX than UNIX".
If you want to see this in action, check out my little "challenge" that I posed in a similar thread on YC recently:
https://news.ycombinator.com/item?id=23257776
The solution provided by "JoshuaDavid" really impressed me, because I was under the impression that that simple task is actually borderline impossible with UNIX pipes and GNU tools:
https://news.ycombinator.com/item?id=23267901
Compare that to the much simpler equivalent in PowerShell:
https://news.ycombinator.com/item?id=23270291
Especially take note half of that script is sample data!
> Unix is seriously uncool with young people at the moment. I intend to turn that around and articles like this offer good material.
UNIX is entirely too cool with young people. They have conflated the legacy UNIX design of the 1970s with their preferred open source software, languages, and platforms.
There are better things out there, and UNIX isn't the be all and end all of system management.
This can also be a security concern. According to research, a great number of defects with security implications occur at the input handling layers:
The reality is that physically, it actually works like this:
(process1,serialize) | (parse,process2,serialize) | ...
Most data being line oriented (with white space field separators) has historically worked well enough, but I get your point that the wheels will eventually fall off.
It’s important to remember that the shell hasn’t always been about high concept programming tasks.
$ grep TODO xmas-gifts
grandma TODO
auntie Sian TODO
jeanie MASTODON T shirt
(7) and common signalling facility
(8) also nice if some files have magic properties like /dev/random or /proc or /dev/null
(9) every program starts with 3 streams, stdin/stdout for work and stderr for out of band errors
This is false - if a process closes stdin/out/err, its children won't have these files open.
2. Everything is a file descriptor
3. File descriptors are things that support standard I/O system calls such as read and write
I'm not all that knowledgable about Unix history, but one thing that has always puzzled me was that for whatever reason network connections (generally) aren't files. While I can do:
cat < /dev/ttyS0
bind /tmp/mysocket 12.34.56.78 80
cat < /tmp/mysocket
That's a whole rabbit hole to go down, but Plan 9 networking is much more like your hypothetical example [0]. Additionally, things like the framebuffer and devices are also exposed as pure files, whereas on Linux and most unix-like systems such devices are just endpoints for making ioctl() calls.
For networking, I don't think there's a particular reason it doesn't exist, but it's worth noting sockets are a little special and require a bit extra care (Which is why they have special syscalls like `shutdown()` and `send()` and `recv()`). If you did pass a TCP socket to `cat` or other programs (Which you can do! just a bit of fork-exec magic), you'd discover it doesn't always work quite right. And while I don't know the history on why such a feature doesn't exist, with the fact that tools like socat or curl do the job pretty well I don't think it's seen as that necessary.
exec 3<>/dev/tcp/www.google.com/80
echo -e "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n" >&3
cat <&3
Additionally, there is /dev/tcp in bash.
>(2) everything (ish) is a file
I'm having a moment. I have to get logs out of AWS, the thing is frustrating to no end. There's a stream, it goes somewhere, it can be written to S3, topics and queues, but it's not like I can jack it to a huge file or many little files and run text processing filters on it. Am I stupid, behind the times, tied to a dead model, just don't get it? There's no "landing" anywhere that I can examine things directly. I miss the accessibility of basic units which I can examine and do things to with simple tools.
Yes, it’s very annoying I have to do all that, but I commend Bezos for his foresight in demanding everything be driven by APIs.
Yes, I know. You've probably never touched either one. But the point is that this is simply chain loading. It's a concept not limited to Unix. execve() does it at the level of processes, where one program can chain to another one, both running in a single process. But it is most definitely not magic, nor something entirely alien to the world of Windows.
* https://en.wikibooks.org/wiki/QBasic/Appendix#CHAIN
If you are not used to pipes on Windows, you haven't pushed Windows nearly hard enough. The complaint about Microsoft DOS was that it didn't have pipes. But Windows NT has had proper pipes all along since the early 1990s, as OS/2 did before it since 1987. Microsoft's command interpreter is capable of using them, and all of the received wisdom that people knew about pipes and Microsoft's command interpreters on DOS rather famously (it being much discussed at the time) went away with OS/2 and Windows NT.
And there are umpteen ways of improving on that, from JP Software's Take Command to various flavours of Unix-alike tools. And you should see some of the things that people do with FOR /F .
Unix was the first with the garden hosepipe metaphor, but it has been some 50 years since then. It hasn't been limited to Unix for over 30 of them. Your operating system has them, and it is very much worthwhile investigating them.
Such programs typically don’t have any interaction so the v and the e parts refer to the two ways you can control what the program does.
v: a vector (list) of input parameters (aka arguments)
e: an environment of key-value pairs
The executed program can interpret these two in any way it wishes to, though there are many conventions that are followed (and bucked by the contrarians!) like using “-“ to prefix flags/options.
This is in addition to any data sent to the program’s standard input — hence the original discussion about using pipes to send the output of one command to the input of another.
LOL. What fantasy land are you living in?
Pipes are great. Untyped, untagged pipes are not. They’re a frigging disaster in every way imaginable.
“Unix is seriously uncool with young people at the moment.”
Unix is half a century old. It’s old enough to be your Dad. Hell, it’s old enough to be your Grandad! It’s had 50 years to better itself and it hasn’t done squat.
Linux? Please. That’s just a crazy cat lady living in Unix’s trash house.
Come back when you’ve a modern successor to Plan 9 that is easy, secure, and doesn’t blow huge UI/UX chunks everywhere.
Not all of us. I much prefer the Unix way of doing things, it just makes more sense then just trying to become another Windows.
I offer you my well wishes on that.
“Linux, ew!”
A big part of teaching computer science to children is breaking this obsession with approaching a computer from the top down — the old ICT ways, and the love of apps — and learning that it is a machine under your own control the understanding of which is entirely tractable from the bottom up.
Unlike the natural sciences, computer science (like math) is entirely man made, to its advantage. No microscopes, test tubes, rock hammers, or magnets required to investigate its phenomena. Just a keyboard.
https://trends.google.com/trends/explore?date=all&q=bash,awk...
> everything is text
I'd often like to send something structured between processes without needing both sides to have to roll their own de/serialization of the domain types; in practice I end up using sockets + some thrown-together HTTP+JSON or TCP+JSON thing instead of pipes for any data that's not CSV-friendly
> everything (ish) is a file
> including pipes and fds
To my mind, this is much less elegant when most of these things don't support seeking, and fnctls have to exist.
It'd be nicer if there were something to declare interfaces like Varlink [0, 1], and a shell that allowed composing pipelines out of them nicely.
> every piece of software is accessible as a file, invoked at the command line
> ...with local arguments
> ...and persistent global in the environment
Sure, mostly fine; serialization still a wart for arguments + env vars, but one I run into much less
> and common signalling facility
As in Unix signals? tbh those seem ugly too; sigwinch, sighup, etc ought to be connected to stdin in some way; it'd be nice if there were a more general way to send arbitrary data to processes as an event
> also nice if some files have magic properties like /dev/random or /proc or /dev/null
userspace programs can't really extend these though, unless they expose FUSE filesystems, which sounds terrible and nobody does
also, this results in things like [2]...
> every program starts with 3 streams, stdin/stdout for work and stderr for out of band errors
iow, things get trickier once I need more than one input one one output. :)
I also prefer something like syslog over stderr, but again this is an argument for more structured things.
[0]: https://varlink.org/
[1]: ideally with sum type support though, and maybe full dependent types like https://dhall-lang.org/
[2]: https://www.phoronix.com/scan.php?page=news_item&px=UEFI-rm-...
I think it a result of there being just a bit too much friction in building a pipeline. A good portion tends to be massaging text formats. The standard unix commands for doing that tend to have infamously bad readability.
Fish Shell seems to be making this better by making a string which has a syntax that makes it clear what it is doing: http://fishshell.com/docs/current/cmds/string.html I use fish shell, and I can usually read and often write text manipulations with the string command without needing to consult the docs.
Nushell seems to take a different approach: add structure to command output. By doing that, it seems that a bunch of stuff that is super finicky in the more traditional shells ends up being simple and easy commands with one clear job in nushell. I have never tried it, but it does seem to be movement in the correct direction.
It's more that people like building features and people don't like saying no to features.
The original unix guys had a rare culture that was happy to knock off unnecessary features.
I tried nushell a few times and the commands really compose better due to the structured approach. How would one sort the output of ls by size in bash without letting ls do the sorting? In nushell it is as simple as "ls | sort-by size".
And yet, at least it's possible to have more than one Macintosh application use the same data. Half the world has migrated to web apps, which are far worse. As a user, it's virtually impossible to connect two web apps at all, or access your data in any way except what the designers decided you should be able to do. Data doesn't get any more "specific to an application" than with web apps.
In a way web apps are then more alike standard unix stuff, where you parse whatever output you get, hoping that it has enough usable structure to do an acceptable job.
The most reusable web apps are those that offer an API, with JSON/XML data formats where you can easily automate your work, and connect them together.
It hasn't aged very well, either. "Even today, the X server turns fast computers into dumb terminals" hasn't been true for at least a couple of decades...
You're not wrong, but that's only because people wrote extensions for direct access to the graphics hardware... which obviously don't work remotely, and so aren't really in the spirit of X. It's great that that was possible, but OTOH it probably delayed the invention of something like Wayland for a decade+.
It's been ages since I've used X for a remote application, but I sometimes wonder how many of them actually really still work to any reasonable degree. I remember some of them working when I last tried it, albeit with abysmal performance compared to Remote desktop, for example.
If you really look at how pipelines are typically used is: lines are analogous to objects and [whatever the executable happens to use] delimiters are analogous to fields. Piping bare objects ("plain old shell objects") makes far more sense, involves far less typing and is far less fragile.
Regex should not be the first hammer you reach for, because it's a scalpel.
I recently wanted cpu cores + 1. That could be a single regex. But this is more maintainable, and readable:
echo '1 + '"$(grep 'cpu cores' /proc/cpuinfo | tail -n1 | awk -F ':' '{print $2}')" | bc
Practically speaking, as an individual, I don't have the needs of the DMV. For my personal use I'm combing through pip-squeaky data file downloads and CSV's.
So even though using Python or some other language causes a gigantic performance hit, it's just the difference between 0.002s and 0.02 seconds: a unit of time/inefficiency so small I can't ever perceive it. So I might also well use a language to do my processing, because at my level it's easier to understand and practically the same speed.
> As for me? I switched to the Mac.
It's amusing that Apple would go on to switch to a unix-based OS in '01.
Exercise for the reader to find out its name.
Pipelines also lack some concept similar to exceptions, but it would also suck to handle those interactively.
This is mere speculation, but I doubt he would have appreciated Plan 9.
$ alias fail=exit 1
$ find / | fail | wc -l; echo $?
0
0
$ set -o pipefail
$ find / | fail | wc -l; echo $?
0
1
Of course, a lot of scripts also forget to enable the similarly strict "errexit" (-e) and "nounset" options (-u), which are also important in modern scripting.
There's another error that hardly anyone bothers to handle correctly:
x=$(find / | fail | wc -l)
if ! x=$(find / | fail | wc -l); then
echo "Fail!" >&2
exit 1
fi
Of course, if you also want the error message from the command. If you want that, you have to start using name pipes or temporary files, with the attendant cleanup. Shell scripting is suddenly much more complicated, and the resulting scripts become much less fun to write.
And that's why shell scripts are so brittle.
I'm developing a shell based on these ideas: https://github.com/geophile/marcel.
Piping is great if you memorize the (often very different) syntax of every individual tool and memorize their flags, but in reality unless it's a task you're doing weekly, you'll have to go digging through MAN pages and documentation every time. It's just not intuitive. Still to date if I don't use `tar` for a few months, I need to lookup the hodge podge of letters needed to make it work.
Whenever possible, I just dump the data in Python and work from there. Yes some tasks will require a little more work, but it's work I'm very comfortable with since I write Python daily.
Your project looks like, but honestly iPython already lets me run shell commands like `ls` and pipe the results into real python. That's mostly what I do these days. I just use iPython as my shell.
I am pretty sure I've seen a Python-based interactive shell a few years ago but I can't remember the name. Have you heard of it?
Edit: x1798DE beat me to it :D
A case in point is this pipeline that I came across in the wild:
TOKEN=$(kubectl describe secret -n kube-system $(kubectl get secrets -n kube-system | grep default | cut -f1 -d ' ') | grep -E '^token' | cut -f2 -d':' | tr -d '\t' | tr -d " ")
In this case, perhaps awk would have absorbed 3 to 4 stages.
Or at the very least, use structured output with "-o json" and jq [1], like they mention in the article.
I have always found that trying to parse JSON with native shell tools has been difficult and error-prone.
[0] https://kubernetes.io/docs/reference/kubectl/jsonpath/ [1] https://stedolan.github.io/jq/
The first involves a lot of single commands and temporary files.
The second uses pipes, but only tacks on commands with no refactoring.
The third would recognize that all the grep and cut should just be awk, that you can redirect the cumulative output of a control statement, that subprocesses and coroutines are your friend. We should all aspire to this.
The fourth stage is rare. Some people start using weird file descriptors to chuck data around pipes, extensive process substitution, etc. This is the Perl of shell and should be avoided. The enlightened return back to terse readable scripts, but with greater wisdom.
Also, I always hear of people hating awk and sed and preferring, say, python. Understanding awk and especially sed will make your python better. There really is a significant niche where the classic tools are a better fit than a full programming language.
If so, what resource do you recommend?
Why do you think so?
kubectl get something -o jsonpath='{.items[*].metadata.name}'First I'll use the command line to, say, grab a file from a URL, parse, sort and format it. If I find myself doing the same commands a lot, I'll make a .sh file and pop the commands in there.
But then there's that next step, which is where Bash in particular falls down: Branching and loops or any real logic. I've tried it enough times to know it's not worth it. So at this point, I load up a text editor and write a NodeJS script which does the same thing (Used to be Perl, or Python). If I need more functionality than what's in the standard library, I'll make a folder and do an npm init -y and npm install a few packages for what I need.
This is not as elegant as pipes, but I have more fine grained control over the data, and the end result is a folder I can zip and send to someone else in case they want to use the same script.
There is a way to make a NodeJS script listen to STDIO and act like another Unix utility, but I never do that. Once I'm in a scripting environment, I might as well just put it all in there so it's in one place.
$ node query-cdc.js 7
It's nothing special, but I wouldn't want to try to do this with command line utilities. (And yes, it was a bit uglier, but I cleaned it up of random crap I had thrown in before posting it.)
1. https://gist.github.com/russellbeattie/f9edf91115b43d6d7ca3c...
This is one clear area where Powershell with its object model has got it right.
You can even redirect some half processed data to a file so you don't have to re-run the first half of the pipe over and over while you work out what's going wrong in the tail end.
https://docs.microsoft.com/en-us/powershell/module/microsoft...
Compare the following 2 equivalent snippets. Which one seems more understandable?
iris_data %>%
names() %>%
tolower() %>%
gsub(".", "_", ., fixed=TRUE) %>%
paste0("(", ., ")")
paste0("(", gsub(".", "_", tolower(names(iris_data)), fixed=TRUE), ")")What Is a Data Frame? (In Python, R, and SQL)
http://www.oilshell.org/blog/2018/11/30.html
The idea is essentially to combine tidyverse and shell -- i.e. put structured data in pipes. I use both shell and R for data cleaning.
My approach is different than say PowerShell because data still gets serialized; it's just more strictly defined and easily parseable. It's more like JSON than in-memory objects.
The left-to-right syntax is nicer and more composable IMO, and many functional languages are growing this feature (Scala, maybe Haskell?). Although I think Unix pipes serve a distinct use case.
Unix pipelines actually helped me make sense of Haskell's monad.
main = getArgs >>= processData >>= displayData
The (imaginary) function getArgs takes no arguments and returns a list of command-line arguments wrapped in an IO monad. The first >>= takes that list and feeds it into the function on its right, processData, which accepts it and returns some other value in an IO monad, which the third >>= accepts and feeds into displayData, which accepts it and returns nothing (or, technically, IO (), the unit (empty) value wrapped in an IO monad) which is main's return value.
See? Everything is still function application, but the mental model becomes feeding data through a pipeline of functions, each of which operates on the data and passes it along.
a; b # Execute command b after a. The result of a is not used by b.
a >> b # Run function b after a. The result of a is not used by b.
a | b # Execute command b after a. The result of a is used by b.
a >>= b # Run function b after a. The result of a is used by b.For pipelines to be monadic, their structure must be able to change depending on the result on the part of pipeline. The type of binding operator hints at it: (>>=) :: m a -> (a -> m b) -> m b.
The second argument receives result of first computation and computes the way overall result (m b) will be computed.
As for program composition, I would like to add gstreamer to the mix: it allows for DAG communication between programs.
https://github.com/prithugoswami/personal-website/blob/maste...
Passing objects through the pipeline and being able to access this data without awk/sed incantations is a blessing for me.
I think anyone who appreciates shell pipelines and python can grok the advantages of the approach taken by Powershell, in a large way it is directly built upon existing an Unix heritage.
I'm not so good at explaining why, but for anyone curious please have a look at the Monad manifesto by Jeffrey Snover
https://devblogs.microsoft.com/powershell/monad-manifesto-th...
You may not agree with the implementation, but the ideas being it, I think, are worth considering.
I've also recently seen it being described in shorter words as a "sticky REPL for shell". Hope you like it, and it makes your life easier!
Pipes that loop, outputting a declarative gui text format, and listen for events from the gui, would be marvellous.
I can’t think how to do that without sockets and a bash loop. And that seems to create the kind of complexity that pipes manage to avoid.
Seriously though, I use both, and IMO they serve different purposes. gron is incredibly useful for exploring unknown data formats, especially with any form of
something | gron | grep something
So in practice, gron for prompts and jq for scripts.
Python scripts that call a few third party programs are notoriously unreadable, full of subprocess call getouptut decode(utf8) bullshit. Python is alright as a language, but it is a very bad team player: it only really works when everything is written in Python, if you want to use things written in other languages it becomes icky really fast. Python projects that use parts written in other languages inevitably gravitate to being 100% Python. Another way to put it, is that Python is a cancer.
file -b `echo $PATH:|sed 's/:/\/* /g'`|cut -d\ -f-2|sort|uniq -c|sort -n
- When PATH contains entries that no longer exist / are not mounted, substitution with "sed" gives a wildcard that is not expanded, and eventually makes "file" report an error, which is not filtered out
- If PATH contains entries that have spaces, the expansion is incorrect
Anyways, if your PATH is malicious then you have worse problems than this silly script :)
It's just a few lines in python, probably takes just as long to write because you don't have to play with what needs to be escaped and what doesn't. You can actually tell what's the intent of each line and it doesn't fail on paths starting with minuses or including spaces. Outside of one time use or a code golf challenge, it's not cute.
file -b /bin/* /usr/bin/* |cut -d' ' -f-2|sort|uniq -c|sort -n
Looking at this code, I am tempted to reimplement this using pipes but saner mind took over and said "don't fix something that is not broken"
I probably would be still do it and get some benchmark numbers to compare both.
At that point I usually fall back to "awk '{ xs[$0]++ } END { for (x in xs) print xs[x], x }'", but it's quite long and awkward to type, so I don't do it by default every time.
At some point I'll add an alias or a script to do this. One day.
edit: OK I finally did it; it's literally been on my to-do list for about 10 years now, so thanks :)
$ cat ~/bin/count
#!/usr/bin/awk -f
{ xs[$0]++ } END { for (x in xs) print xs[x], x }
Text filtering approach narrowly rescued by website feature.
Phew, that was close!
"Chad is a man who automatically and naturally turns girls on due to his appearance and demeanor" https://www.reddit.com/r/ForeverAlone/comments/6bgbc2/whats_...
"chad 1. Residue of faecal matter situated between arse cheeks after incomplete wiping and can spread to balls."
I have no idea why the author decided to use that term.
There is a related term "Chad" (capital C) which invokes the image of a man who is attractive to women. Again, I have no idea why the author decided to use that term.
There are some commands that need literal arguments which is different than standard input. For instance the echo command. `echo < list.txt` will not work assuming you want to print the items inside the list. `echo itemA itemB itemC` will work. This is where xargs comes into play -- it converts the standard input stream to literal arguments among other things.
If you like pipes and multimedia, checkout gstreamer, it has taken the custom pipeline example to real-time.
But is this really HN top page worthy? I have seen this horse beaten to death for decades now. These kind of articles have been around since the very beginning of the internet.
Am I missing something newsworthy which makes this article different from the hundreds of thousands of similar articles?
As for being top most article, if I had to guess it's because people feel related to the tool's power and enjoy sharing anecdotes of it.