I feel like the art of UNIX is slowly fading into oblivion, especially with the new generation of programmers/developers.
Eventually, they'll become the ones that decide the fate of software engineers (by being hiring managers, etc.) and we'll see more and more monstrosity like the article portraits, instead of cleverly using UNIX tools where applicable.
There's so many things that the software world is doing wrong that I am surprised that even at this inefficacy, it's such a viable and well-paid profession. It's almost as if we are creating insanely complex solutions that in turn require a large amount of developers to support them, whereas we could have chosen a much more practical solution which is self-sustaining.
As a young programmer I certainly found some love for all these unix tools and learn new things every day.
I think the problem is scale. Back in the day (before I was born), very few people were programmers and the resources they could use were limited. This means they didn’t need insanely complex solutions because they already needed complex solutions just to make it work on the limited hardware. People were trying to solve problems with computers. Nowaday you take a problem that could be solved by an microcontroller with three buttons and make it a cloud app with web server, web interface and all kind of other things like containers.
We donlt really tend to ask the question what a good solution would look like. Often it is the case that you just use the technology the developer wants to learn
I guess you should sell them the choice of either the big complex thing written by a large team for $2.3 MM, or the lean highly-optimized solution written by a handful of elite "10X" developers for $1.9 MM. (Of course the 10X developers are in high demand and very expensive.)
I think it's mostly a matter of chance more than anything else.
If you've jumped straight into programming, you'll probably consider any of those problems as a nail to your C/JS/Java/Python Hammer.
I was lucky to be initiated to the GNU / UNIX toolset by operation folks when doing tech support in a SAAS biz.
We were dealing with a lot of text files and it didn't feel right to offload whatever my
problem was to them, so I started learning from them and the scripts they wrote.
Mostly trivial stuff, such as in a directory, only select the relevant files, iterate through them to sum up something or find exactly the bit of information you need.
This allowed me to decrease time spent looking for the answer to my current case or colleague's case.
Of course, I then built those functionalities into a .bashrc (or was it bash_profile?) script to help my fellow support folks finding the answer on their own..
Turns out, most people don't want to have anything to do with a command prompt, even if the hard part has been done for you. That's been a pretty good lesson BTW.
Fast forward a few years later to now and I'm realising a good amount of stuff I've been writing lately could potentially be GNU'd instead of writing a Python script.
I have also found this to be the case too. Most people would rather have a GUI before even touching the command line. Most notably is Git; every single one of my developers use Sourcetree and if I have to help them with something, I always have to pop open the terminal. It's gotten to the point where I'm considered "odd" because I use the command line. It's become a running joke among everyone.
I don't see how this can be a target of their joke: they have problems with their GUI (I am assuming that's what Sourcetree is), you solve them with your CLI. If they want to laugh, they 'd better fix their own problems themselves, I guess?
They probably respect his knowledge and know he is the only competent one, but they simply don't care to learn the CLI method as they are lazy, or can't justify all the time it takes to learn it if he is there to fix their problems (more efficient). I've been on both sides of that situation before. There is always an expert in something you want to know, but can't justify the time. So don't take it too hard on them. I support a Linux based app at work and thus got pretty comfortable with the command line (vim, grep, awk, head, tail, cut, sort, ls, cp, top, cat...etc). To my knowledge I'm the only one in engineering with this knowledge (not in IT support). I also notice my fellow engineers will frequently use more complicated techniques for something that is a single piped command for me. I don't fault them for it though as they have no need to use Linux.
>It's gotten to the point where I'm considered "odd" because I use the command line. It's become a running joke among everyone.
That's... very bizarre. Amongst the engineers I work with it's always an "oh cool, you know how to use bash/terminal" rather than "weird, why are you using terminal".
> Turns out, most people don't want to have anything to do with a command prompt, even if the hard part has been done for you. That's been a pretty good lesson BTW.
My experience has been that some folks are resistant to the command line (but I wouldn't say most). This is too bad because I feel like it's a crucial part of development. I even wrote a post about it: https://letterstoanewdeveloper.com/2019/02/04/learn-the-comm...
In my specific case, it wasn't dev profiles but people on a spectrum going from I grok tech stuff to only browsing and the office suite.
So obviously, whenever I started doing trainings on how to use the cli / the toolkit, I could see what is best described as mild panic.
Which I get, CLI isn't inviting at all, it's pretty daunting, there's no real emphasis on safety (as in not breaking anything), it's far from being easy to reason with when you're used to the Windows / GUI world.
The lesson was not so much about the prompt and more about understanding your target audience and catering to them really.
Ah, that makes sense. Yes, understanding your audience and meeting them where they are (or maybe just a bit closer to where you want them to get to) is always very important.
This is something I've noticed in the last 8-10 years. The rise of the python/js/java paradigm everywhere. Some of the associated LDIF (json) I enjoy much more than XML and flat files but the misapplication of tools is becoming an epidemic. When I can write: awk -F "," '{for (x = 1 ; x <= NF ; x++) {if ($x ~ /[0-9]+/) {a[x] = a[x] + $x}}} END { for (p in a) {printf "%d = %d\n",p,a[p]}}' to sum columns in 5 seconds and people are scrambling with libraries to do matrix operations I tend to scratch my head and walk away. The aversion to the command line is also something that bothers me but I don't run into it as much in my field.
import pandas as pd
data = pd.read_csv(filename)
print(data.sum())
and have the same result, I'm going to do the one that is faster to write, fewer characters, and lets me understand what's going on.
And don't get me wrong, I've written some gnarly pipelined bash before, although I'm by no means an expert, but that doesn't mean its always the right tool for the job.
I was going to be the long haired *nix geek here but I have no hair and the world is moving on. I can't pick bones with python for data science/analysis and personal convenience. _However_ as a principal engineer if someone was to say, for a trivial dataset, that we need python and pandas for an operation like this where python + pandas was not already provisioned the answer would be no.
>When I can write: awk -F "," '{for (x = 1 ; x <= NF ; x++) {if ($x ~ /[0-9]+/) {a[x] = a[x] + $x}}} END { for (p in a) {printf "%d = %d\n",p,a[p]}}'
To be fair, what the matrix libraries do is provide readability and clarity.
Show a programmer unfamiliar with awk your statement and they're going to be spending quite a while parsing it.
Show a programmer unfamiliar with numpy/pandas some matrix multiplication and they may understand it intuitively for the most part without even having to look up references.
edit: I'm actually an awk noob myself but after rereading your code for a second, it makes quite a bit of sense. "For all rows in the column, take all digits 0-9 and sum them". So perhaps not much is gained by the library
>The aversion to the command line is also something that bothers me but I don't run into it as much in my field.
The company I recently started with is really big on Splunk.
The fact that they're proud enough of coming up with the tagline "Taking the sh out of IT" to print it on branded t-shirts featured in their training material was a hint that I wouldn't be a huge fan of the product, personally.
Abstracting things away is great, but something about IT pros being proud of avoiding the command line rubs me entirely the wrong way.
I don't think so. What is old is new again; even this blog post is an example of a developer relearning that core lesson.
Also, I'd like to point out that in both this blog post and the Taco Bell one, the "UNIX way" is being compared against a straw man example, not a real example of over-engineering. Neither post actually provides any evidence of any real inefficiency. Both authors are just trying to explain and improve their own thinking about programming, not trying to cast judgement on a generation.
> I feel like the art of UNIX is slowly fading into oblivion
I think that by itself isn't a problem, but fading right along with it is the capacity to decompose and structure the problem domain.
Even if one ends up writing a solution in a different language for whatever reasons, starting out by mapping the problem with UNIX command line tools will result in a better understanding of the problem; an understanding that is language agnostic and can be transferred to any preferred method of implementation.
These complex solutions allow one operations engineer to manage thousands and thousands of servers/containers. Guys that just knew how to bang together bash and Perl scripts got laid off all over the place in favor of people that know cloud stuff.
I believe the author probably understands that fully and is instead referring to the situation where the optimal solution is a little bash script and somehow the developer designs this horribly complex solution that isn't any more performant, but seems more fancy. Sadly, I think developers sometime do this just to learn new things and keep their resume up to date. It's rough out there from what I read (glad I'm not a developer).
I dunno -- there's definitely a community of developers who prefer not to use the command-line / Unix toolset, but at least half of those I know [especially the macOS users and those with some ops experience] routinely reach for the core Unix utilities , especially when doing ad hoc quick fixes / file processing. For my part, I can't imagine doing my job without them [or I guess more accurately, my job wouldn't exist in the first place with them]
I think git has a pretty significant amount to do with the persistence of command-line tool use among the younger set -- it's just so much more efficient / easier to use from the terminal.
> I feel like the art of UNIX is slowly fading into oblivion, especially with the new generation of programmers/developers. ...
People said the exact same thing in the 1990s, too. "Enterprise"-managed projects, using tech stacks like Win32 or Java, have generally tended to produce large, unwieldly monoliths.
Things fade and shine in succession. Good bits will always come back. It's a bit like the saying about mathematics truthiness nature: doesn't matter who or when you look, they will re-emerge as is. composing tiny bits is always good, whether it's unix commands, lisp functions, or forth words..
Societies are large and full of random fluxes and waves.. right now it might be the time for Wirth 17 pages long solutions .. but McIlroy one liner will come back.
> composing tiny bits is always good, whether it's unix commands, lisp functions, or forth words..
It's not so much about "tiny" but about no fuss, low complexity, no-nonsense, etc. For example, Forth may be tiny, but it's unusable for most applications. Assembler languages are tiny, too.
> composing tiny bits is always good, whether it's unix commands, lisp functions, or forth words
The problem with this type of thinking is that "composing tiny bits" is obviously good to anyone. BUT there are vast differences between composing unix commands and composing lisp functions (and composing tiny JS libraries in NPM etc.) and some of these are good, while some are not.
It's actually very difficult to properly create a system out of composable, re-usable components. I would actually say that Unix pipes are a particularly bad model of how to do that (as are NPM micro-libraries). No well-defined interfaces, arcane naming etc. - they are all reasons why most modern versions of these tools have grown and grown.
Composing unix commands means endlessly adjusting the textual output of one tool to the input requirements of another. Most such coding only cares about the happy cases; it will march on with an incorrect datum because of some unexpected twist in a textual format. For instance, a field delimiting character suddenly occurs in a field. Or some code is scraping data by hard columns, but a field width overflows and the columns shift.
> I feel like the art of UNIX is slowly fading into oblivion, especially with the new generation of programmers/developers.
For what it's worth, I'm a first-year computer science major and the class I'm currently taking is very much focused on the "art of Unix." We've been doing shell scripting and regular expressions and the like. I quite enjoy it.
Leads are having the responsibility. Most of them are good and know there shit.
So I wouldn't worry to much. All other small companies who have people who don't know better, don't have too much technology inside anyway. Or at least often enough.
Eventually, they'll become the ones that decide the fate of software engineers (by being hiring managers, etc.) and we'll see more and more monstrosity like the article portraits, instead of cleverly using UNIX tools where applicable.
There's so many things that the software world is doing wrong that I am surprised that even at this inefficacy, it's such a viable and well-paid profession. It's almost as if we are creating insanely complex solutions that in turn require a large amount of developers to support them, whereas we could have chosen a much more practical solution which is self-sustaining.