Count matching lines in a file.

Say you want to know how many lines in a CSV file match a given regexp. One way you could do it is by combining the sed and wc commands:

sed -n -e '/regexp/p' filename.txt | wc -l

This should tell you how many lines there are in the file that match regexp.

The Boring Stack

I think  someone finally put in writing something I’ve been thinking for  some time: https://justmarkup.com/notes/2020-01-14-boring-by-default/

To me it makes a lot of sense to start any new project with a blank static html page, some old fashion CSS and the vanilla js framework. Over complicating things seems to be the norm nowadays, and it’s leading us to an increasingly fatter web.

I remember back when I first started learning about web development there was a strong push for fast websites in terms of page load, and interaction. File size and external resources was something you were trying to keep as low as possible. But things quickly got out of control with the rise of javascript frameworks. Suddenly there were many different javascript frameworks and developers started loading them in their pages, sometimes more than one at the same time. The fact that libraries had to implement a no-conflict function is a testimony of this issue.

Many new web developers compensated their lack of experience with a ton of plugins. Inexperienced programmer quickly began creating fat websites where simple static HTML would have sufficed; no plugins required. WordPress development, for example, is based completely around the use of plugins, many of which load their own version of X library. It became common to see websites that loaded 2 or more version of jQuery on the same page because different plugins required different versions. At some point it seemed to me that a WordPress developer was just someone who could login to the admin panel, and install a bunch of plugins. And the plugin market didn’t make it any better.

Plugin makers, in an effort to capture as many sells as possible, ship plugins with a gazillion features out of which you will only use 1 or 2. The thing is, developer Y will require a different 1 or 2 features of the same plugin, so having them available increases the chance of a purchase.

Then came heavier frameworks like Vue and React. The problem is not the frameworks themselves, but developers who just use anything new and fancy for the sake of appearing cool, and in with the current web trends. Add to all of this the ton of tracking scripts that websites add plus the ads and you have a mess. No wonder people have been talking about a bloated web for years.

I hope one day we will realize the error in our ways and get back to the right path. But for now, do me a favor, be boring…

I Quit Platzi. Here is why.

So, I finally decided to cancel, or rather pause my platzi subscription for the longest period of time they allow –3 months. In case you don’t know what platzi is, it is a mostly-Spanish learning platform that tries to be the leading learning platform for the Hispanic world in the tech industry. And they really try to do it. The key word there being “try”.

So, what is wrong with it? In my particular case I am what is wrong. I’m too picky; it’s easy for small things to bother me more than they should. I’m a really hard judge. But mostly because I have years of experience programming. I’m not their target audience.

I don’t pretend to know EVERYTHING they teach. That would be absurd given how they cover a large range of topics. But to be honest, the things they teach that I don’t know are the things I’m not interested on at the moment, otherwise I would have learned them by now as well.

I find their classes to be basic introductions that never really dive deep in any topic. Their professional courses teach how to do things, not the theory behind that how. That is to say, they don’t really tell you why, only how. In my experience, and given my somewhat obsessive personality, the why is just as important if not more than the how. Theory matters.

I don’t meant to say platzi is a bad option. It may really be the best option for the Spanish-speaking world. This reflects either that I don’t know how to recognize a good thing when I see it, or that there is a lack of higher quality competitors that keep pushing the boundary of what is considered a good option. I think it is the latter.

When I first started my subscription I was excited. I took a couple of great courses that kept the hype high, but as time progressed and I took more courses I started to realize that newer courses seem to be made in a hurry just for the sake of expanding their library of content; it was quite disappointing to see courses that could easily be replaced by reading a man page.

So, is platzi really that bad? NO. Introductory courses are not a bad thing, they are a necessity, and there is only so much you can cover in a single course. Platzi’s job is not to please my peculiar taste for advanced higher education in very specific topics. Their job is to introduce the tech world to people so they can aspire to a better life through well paid jobs in the industry. And they are doing it. There is a plethora of testimonies from people who have better their lives thanks to what platzi has to offer. And that is their mission. They are nailing it.

I can recognize the value in being able to access a large array of courses covering a variety of topics in design, engineering, economics, and personal growth. I would definitely recommend platzi to some people, but not to all people. I myself will continue to be a sporadic student there. But for now, I have better learning resources that I want to explore.

Control Characters in Sed

After that little fiasco with my sed practice file I decided to do a search and replace for the BS characters in the file to clean it up and be able to use it as I originally intended. Immediately the question arouse: How do I match control characters?

The sed man page in Mac OS tells you that “The re_format(7) manual page fully describes [the regexps] formats“, so my first stop was that man page:

man re_format

There is a section in there that says you can use \xx.. to specify a literal sequence using a hexadecimal number. But that didn’t seem to work. My guess is that it doesn’t work because sed doesn’t use enhanced regexps.

A quick search lead me to an article where it says you can use command substitution to enter the control character, which worked nicely:

sed -i'' -e 's/.'$(echo "\x08")'//g' input_file

The dot (.) before the command substitution is because we need to remove the character immediately before the BS character otherwise we will end up with garbage:

N^HNA^HAM^HME^HE

would result into

NNAAMMEE

if we didn’t remove the character immediately before the BS character represented as ^H here.

When in Sed Land, Read the Input File

A couple of days ago I was playing around with sed in Mac just for fun, and came up to a quite confusing issue: I could not match NAME using /NAME/ as a context address. In fact, trying to match anything using a longer-than-2-chars regexp would not work:

/NA/ would match NAME
/AM/ would match NAME
/ME/ would match NAME
/NAM/ would not match NAME
/AME/ would not match NAME

This was quite puzzling. After trying different things and looking around the internet I was still where I had started. Then I decided to try using small letters:

/th/ would match many different lines in the input file
/the/ would match many different lines in the input file

Woah! I could use longer-than-2-chars with small case letters, but not with capital ones. WTF? I continued trying with different combinations using capital letters, and anything with 1 or 2 chars would work, but more than 3 chars wold fail to match anything even though there seemed to be matches in the input file.

After a long time with this I decided to check the input file as a last act of hope. I found the problem.

At this point I should tell you how I got the input file. Remember how I said I was just practicing some sed. Well, I thought that an easy way to get a practice input file would be to put the contents of the man page for sed in a regular text file:

man sed > input_file

Well, that was not the best idea. My file ended up something like:

N^HNA^HAM^HME^HE

I didn’t notice that when I fist made the file because ^H is a BS control character, and doing

cat input_file

produces the expected text NAME because cat interprets those control characters, but opening the input file in vim revealed the control characters.

This is one of those life lessons that you never really forget, but if you do, you will feel stupid the next time you have to re-learn it. Read the input file in a text editor.

How to Whitelist an IP with IP Tables

I’ve had a server for a while now that I usually don’t use for much. From time to time I tell myself that I should start using it for some personal projects, but I never really do. Until now…

About a week ago I decided to finally do something with it besides hosting buzu.me. From some time now I’ve had a small app to manage a side business of mine. I had it running on a local host on a laptop that I could take anywhere to manage my business, but at times that became a little problematic for numerous reasons, including pure laziness to turn the computer on, or not being able to register a payment because I didn’t have the laptop close when visiting friends who had happen to purchase something “on credit”.

Last Saturday while at the store, which by the way opens only on Saturdays in a farmers market of sorts, I decided to take the time to start setting up the server for my app, and in the process I locked my self out of it after failing to login a couple of times. I contacted support to ask them to remove my IP from the blacklist, and was able to log in again just to lock myself out again trying to ssh into the server instead of using the control panel. I asked support to again remove my IP address from the blacklist, and to resend my SSH login information. It turns out it was different from my control panel information.

Since, in the past, this has happened to me a couple of times, I decided to just learn how to remove my IP from the blacklist so that if I ever lock myself out again on a certain IP because of a mistype or something I could just connect to another network and log in correctly to remove my original IP from the blacklist. Given how my server blocks any IP after only 3 failed attempts, it is not really unthinkable that at some point I will need to clear my IP from the blacklist.

It turns out that using iptables this is a really simple process:

  1. Find out the IP you want to clear out.
  2. Search the iptables filter table for that IP to get the rule number.
  3. Delete that chain/rulenumber

You can find the IP address that you want to remove by connecting to that network and using any of the services out there to check your IP.

To search iptables for that IP you can do this in the console:

iptables -L --line-numbers | grep "111.111.111.111"

using your ip instead of 111.111.111.111. That will return the lines where that IP is found. Those are the lines you need to remove, but before doing that you will need to find out the chain where those lines are. For this you can just list all the rules for the filter table, and look for the lines you got before:

iptables -L --line-numbers

You do not have to look at all the rules one by one, but rather at the line number you got before in each of the chains. Once you find a line that matches, just look a few lines before. Right before the first line in the current sequence of lines you should see a line with no number that starts with “Chain ” and then it says the chain name. In my specific case searching for my IP in iptables got me 2 results in  DENYIN, and DENYOUT respectively.

Once you have this information all you have to do is delete those lines:

iptables -D DENYIN NNN

Where NNN is the line number you want to remove, and DENYIN is your chain name which may be different.

You should do this with care as it relates to the security of your server.

If you would like to learn more about iptables​ you can read the manual:

man iptables

The Asterisk Next to the Filename in Vim’s Netrw

Yesterday I was opening up a file, but since I didn’t quite remember the name of it, I decided to instead open the containing directory in Vim:

:vsp ../dir-name

This worked as expected, opening the file browser, which I learned is called Netrw. However, I noticed that one of the files had an asterisk next to its name:

my-file.php*

I immediately started to wonder what that was about, but a quick search on duckduckgo.com produced no satisfactory results, so I hit F1 in vim, and the help page came up. Then I searched for *

/*

which, of course, started to find a lot of asterisks because it seems vim uses them a lot in the help pages, but at some point I finally came to a section that explains that

“The |getftype()| function is used to append a bit of filigree to indicate filetype to locally listed files:”

Those indicators are as follows:

directory : /
executable : *
fifo : |
links : @
sockets : =

So there you have it; an asterisk next to a filename in Netrw means the file is executable.

The Netrw man page seems to be full of really nice information, so I now owe it to myself to give it a good read. You should too if you are into Vim.