Fixing MySQL Corrupt Table “ERROR 2013 (HY000): Lost connection to MySQL server during query”

Today I tried to drop a database in mysql


Which resulted in a error: “ERROR 2013 (HY000): Lost connection to MySQL server during query”. At that point everything started to go wrong, to the point where the MySQL server went completely down and I was unable to run it again. In short this is how things happened:

  1. Try to drop the database, which resulted in an error
  2. Keep trying to drop it, which resulted in the same error.
  3. Try increasing the net_read_timeout and net_write_timeout on the server, as well as –connect-timeout in the mysql client. No luck.
  4. Search the web, find no useful advice, but found a possible reason “data corruption”
  5. Try to recover data with a backup of the database from the production server. Shit went dow!
  6. The server stopped and refuse to start again. Following the instructions in the stop error lead me to new search terms.
  7. We have a new problem now, do more web searching.
  8. Find a useful page which recommended disabling innodb, and setting myisam as the default database engine.
  9. After doing that, the server worked, but trying to dump the database resulted in an error: ERROR 1010 (HY000): Error dropping database (can’t rmdir ‘./db_name’, errno: 39 “Directory not empty”)
  10. I decided to manually remove the directory, which removed the database.
  11. Create the database again and use the backup to restore it.
  12. Got an error saying that the table ‘table’ already existed. ‘table’ is the first table that the backup file tries to create.
  13. Try to dump the database, but got the same error as in #9.
  14. Manually delete the database again, Create it and the do a proper DROP.
  15. Create the database again and restore from backup. It worked!

I read a few pages during this problem, and found these useful:

SOLVED: InnoDB Error: space header page consists of zero bytes: xampp

MySQL: Error dropping database (errno 13; errno 17; errno 39)


Fixing PHP’s ‘Warning: simplexml_load_file(): I/O warning : failed to load external entity’

Today I had to re-visit a Drupal module I wrote in 2015. The module parses an external XML file which it loads using simplexml_load_file. When I opened the version of the site that lives in my local server, and visited the page that uses the module, I got the error in the title of this entry. I checked php.ini to make sure allow_url_fopen was On. It was.

Searching the web I came to a bug report in the php website. It was suggested to restart the apache server since the contents of resolv.conf may have changed after the server, and php got their DNS servers entry. This made sense to me, since I have switched networks since turning the computer on. I gave it a try, and it worked:

service httpd restart

Other info:

  • The location of php.ini in my case, running Fedora with php installed using dnf, is /var/php.ini
  • The location of resolv.conf is /etc/resolv.conf

The sources I visited:

Opening port 80 with iptables

A few days ago I was contacted by a client whose website had been offline for a month. The site has a history of going offline because of server overloads. The client is OK with the occasional down time as the site is just a pet project. Every time the server goes down we usually restart the MySQL server and everything goes back to normal, but this time it was different. Instead of restarting the MySQL server, the whole server was restarted by power cycling it, and that is when everything went wrong.

When the site came back up, the HTTP and MySQL server had to be manually started, and the program that communicates with the server control panel for status reporting was down as well. It had to also be started manually. However, after all of this, the site was still off line. Trying to access the website resulted in an error like the one you see when you are not connected to the internet.

I decided to curl the site, and in return I got an error saying “No route to host”. Pinging the server worked, but form some reason I could not connect to it, because of this I knew it was not a DNS problem, since the url resolved to the server IP correctly, but the connection was refued. Could it be a server issue?

I decided to take a look at the server config files, only to find out everything was configured correctly. At this point I was absolutely intrigued.

I decided to search the interwebs to find out what people said about the “no route to host” problem, and, as I thought, it was a connection problem. The machine was not refusing the connection, the network was. I confirmed that by fetching localhost from the server. I got the expected result, so the server was not refusing the connection.

It was at this point that I decided to concentrate on the network side of things. I started by determining the port the server was listening on. I did this by checking the server configuration files. When I confirmed the server was listening on port 80, I decided to check if port 80 was open. Using iptables I determined that it was not. From there, it was just a mater of opening the port. I did that by running

iptables -I INPUT 1 -p tcp --dport 80 -j ACCEP

Here are some documents I consulted while trying to solve this problem. Some proved useful, others did not. It is worth mentioning the server runs Fedora.

Some linux FS management tips

A few days back, I decided to upgrade from Fedora 19, to 24, and decided to do so using a usb stick. I downloaded the installer, and copied into the usb stick using dd.

A couple of days later, when I wanted to use that usb stick, I found out I could do nothing with it since its file system was write-protected. Trying to format the device did not work for the same reason. Using the disks utility in Fedora 24 showed 3 partitions in the usb stick, but trying to delete them would result in an error related to block size. The driver reported one size, but linux reported another. After a bit of searching, I came to an ask ubuntu answer that recommended using mkfs.vfat to solve the problem.

  • You can use mkfs.vfat, or one of the other mkfs.* programs to format a stubborn drive, or any other drive for that matter.

Being able to fix this problem, I decided to try with an old usb stick I got at a meet up back when Barnes & Noble were preparing to launch their android-based reader NOOK. The usb stick has never worked. It doesn’t mount, and reviewing it, there is no partition in it, let along a file system. I tried making a partition using the disk utility, but failed. I decided to use parted in the command line, but no luck, the device was write-protected. After a bit of searching, I found out you can use hdparm on write-protected devices to make them writeable. This did not work on my usb stick, which leaves me thinking that the device is just damaged.

  • You can use parted to make partitions in a device
  • You can use hdparm to set various option on drives, like write-protection.

Speaking about partitions, having a usb stick with more than one partition in it can be quite useful, and cool. Just remember that dumb windows doesn’t mount all of them, only the first one.

Other tips:

  • Use mount to find out how a device is mounted.
  • Use dosfsck to check and repair DOS file systems

I hope these tips prove useful at some point in your life as they have done in mine. Thanks to all that share wisdom around the net, what would we do without them…

Fixing Fedora 24 dnf problems.

The short story:

If you are getting the following error when trying to install software using dnf in fedora 24:

python3: /builddir/build/BUILD/hawkey-0.6.3/src/python/exception-py.c:101: ret2e: Assertion `0′ failed.

You may try doing:

dnf --disablerepo=updates <package>

Where <package> is the thing you are trying to install.

The long story:

Today, I was working on a site on my newly installed Fedora 24, but the drupal-based site failed to show anything after I updated a block. All non-admin pages would just show blank. So I started tracking code, and at some point it occurred to me that maybe I had to run the update script, which fixed the problem. I still don’t know why updating the content of a block would cause the site to break, but my guess is that it has to do with drupal caches. This because the function that failed is called “block_page_build”. Anyway, before I could run the update script, drupal was complaining about some missing php components, so I wanted to install them.

I went ahead and did

dnf install php-xml

but that returned an error (see the short story above). I didn’t know what to do, so I started searching the web, but found nothing. Then I turned to the dnf manual. In there I saw you can add -v to get a more verbose output. Running dnf with -v got me this:

cachedir: /var/cache/dnf
 Loaded plugins: config-manager, protected_packages, generate_completion_cache, Query, builddep, langpacks, debuginfo-install, download, needs-restarting, playground, reposync, noroot, copr
 langpacks: No languages are enabled
 Error reading file : /var/lib/dnf/plugins/langpacks/installed_langpacks as it does not exist
 initialized Langpacks plugin
 DNF version: 1.1.10
 repo: using cache for: fedora
 not found deltainfo for: Fedora 24 - x86_64
 not found updateinfo for: Fedora 24 - x86_64
 repo: using cache for: google-chrome
 not found deltainfo for: google-chrome
 not found updateinfo for: google-chrome
 repo: using cache for: updates
 python3: /builddir/build/BUILD/hawkey-0.6.3/src/python/exception-py.c:101: ret2e: Assertion `0' failed.

Look at that! the problem happens right after trying to use the updates repo.

I had read in the dnf man page that there is a location where the repo files are kept, so I went and moved all update repos out of the directory where they are kept, and run dnf install again. It worked.

Well, all that was left to do was find a proper way to ignore repos when doing installs, and the man page had the answer: –disablerepo

So, that is the story. Keep coding kids.

The unset($args[0]) PHP Pattern

Suppose you are working on a system, and want to allow users to use so-called hooks to hook into the system and change it’s behavior. Think of it as wordpress’ filter/action kind-of-thing. This is in a way, something similar to events in javascript, and the purpose is basically the same: you want to do, or let others do something when something else happens, and potentially change the way the system works, or extend its functionality.

Implementing this kind of feature is surprisingly simple. All you need is a way to let people subscribe to your hooks, and a way to call all subscribed members for a certain hook at some point. The implementation details are not really our concern right now, but what happens when you call each subscribed member.

In order for hooks to be useful, you need to be able to pass relevant information to the subscribed members at call time. For example, in Javascript, native events get passed an event object that contains all sorts of useful information. The way you pass this information to the subscribed members will depend on your implementation of the subscribe method you use.

One pattern I’ve noticed working with drupal is unset($args[0]).

Drupal uses a hook system that allows developers to hook into the system simply by following some function naming conventions. The drupal specifics are not important right now. What is important is the way drupal calls its hooks: it uses a function called module_invoke_all.

The module_invoke_all function, in its definition, takes a single argument: the name of the hook. We already mentioned the importance that being able to pass information to the subscribed member has, so why is it that the drupal function to invoke hooks takes only the name of the hook as argument? How do you pass the relevant information?

PHP, just like other programming languages allows you to pass more arguments to a function that its formal parameters. The way you access the extra arguments passed to the function varies depending on the language. In php all arguments passed to a function are kept in the $args array.

Lets say you call module_invoke_all as follows:

module_invoke_all(‘my_hook’, ‘something cool’, ‘something interesting’);

Inside the function, the $args array will have 3 members, one for each argument passed to it. So, if you want to pass those values, minus the hook name to the subscribed members, all you need to do is get rid of that, and pass the resulting array to the subscribed members. That is why unset($args[0]) does.

I’m not sure this is the best way to do it, but it is certainly interesting. If you want to read the full code for the module_invoke_all function, just check out the drupal documentation on it (!

Learning Programming

Over the years, I’ve seen many people struggle with learning how to program. I’ve never really understood what the big deal is. When I first learned programming, I did it on my own, with a bunch of internet tutorials, the language documentation, and some time. I kept being surprised by how easy it was to do the things I had seen on the computer. I understand we all are different, but I believe the real reason why many people struggle to learn programming is because of the way it is taught.

My initial take on programming was Javascript. I didn’t think I was learning programming. All thought was that I was learning how to make websites. I kept thinking that programming must be something different, because people always say programming is hard, and what I was doing was really easy. I learned Javscript along with HTML, and CSS, and honestly, I used to think real programmers did things differently. It was surprising that calling alert would pop up an alert window, just like that. It could not be that easy at the professional level, right?

Learning programming, and learning a programming language a two different things -quite different, actually- that are usually thought of as being the same thing. Because of this, we tend to want to teach people both things at the same time, and that is confusing. Learning a programming language means learning all the details of it, the syntax, the grammar, the definitions, the different components, data types, and operators, among many other things. Learning programming, means learning how to do things. Doing things is more fun, and most people learn by doing. Back when I was calling alerts, or prompts, I had no idea that those things were functions, and that I was passing strings to them as parameters. I had no idea what the signature of a function was, or what the difference between arguments and parameters was. All I knew was that a hello world program in Javascript was easy to write.

Many years have passed since my first programming adventures. I’ve learned so much in those years, but I feel like I didn’t learn enough. I am a lazy programmer; I waste time on this and that, I read, and then forget what I read. I’ve learned a few more programming languages, but I didn’t really learn how to program in those languages. For example, I learned python, I read the entire python documentation, and I built some python programs a couple of years ago. Today, I’m learning python again. This time I’m not worried about the documentation, I just want to learn to program in python, because one can easily forget all the details that are in the docs, but it is harder to forget how to do things once you learn them.

I could easily criticize the way programming is taught in school, but I won’t. I’m more interested on talking about how programming is learned outside of the classroom. One thing you must learn, if you want to become a programmer is that you need to read. Reading a book or two is fine to get your feet wet, but after that, most books become monotonous, more of the same. What you really need to do is read code.

If you wanted to learn how to write novels, it would be OK to read one or two books on grammar, and such, but that will teach you the rules, and rules are boring. If you want to become a great author, you need to read novels. One of my favorite quotes from the film industry is “I didn’t go to film school, I went to films” There is so much one can learn from just looking at how others do things.

I’m learning python once again. This is the last time I learn python. My new strategy is simple. Read through the book Learn Python the Hard Way, and that’s it. The book, plus the knowledge I keep from that Sunday morning I read the entire documentation should be all I need to go out to github and start ready code. Reading code will make me a better programmer, not just in python. Once we learn something, we tend to think of ways to apply that knowledge elsewhere.

Wish me luck.

My Take on Passwords

I’ve been wanting to write about this for quite some time now. There is a lot of effort that is put into making systems secure, and it all goes to waste when you choose a weak password. “123456789” is a weak password, “password” is a weak password, “god” is a weak password, your pet’s name is a weak password. A weak password is anything that can be easy to guess by someone that knows you well, or by a computer. A password that is only a few characters long is a weak password no matter how many symbols or strange characters you use because it can be easily guessed by a computer. Nonetheless, websites all over the web want you to choose a password that is at least N number of character or at most M number of characters; a password that contains symbols, but does not contain you name, or part of your email, or your user name, and the list goes on. This is damn stupid.

Over the years, we’ve been trained to chose really bad passwords. We’ve been lead to believe that “m00Npi3” is a strong password because it is over 4 characters long, and has weird characters. Sure, your friends may not be able to guess it, but a computer could do it rather easily. However, we go on about our lives believing that is a good password, and then we use it for everything. Websites all over the web warn us not to use the same password everywhere, but who wants to remember a hundred passwords? Sure, you can use a plugin on your browser that remembers the passwords for you, but what about when you need to access your email from the public library, or from your friend’s house? Good luck!

Yet, we’ve been lead to believe this is all for our own good.

I mentioned I’ve been wanting to write about this for a long time, but today paypal was the last load of crap that I was willing to take before hitting the keyboard. I logged into my account, and paypal kindly suggests that I should change my password. I started changing my passwords last week, so I thought “hey, perfect timing!”. I clicked the link they provided, and I was taken to a page that asked me to confirm that I was who I said I was by providing either my bank account number, my credit card number, or my debit card number. WTF? Why? I’m already in my account! But OK, lets just pretend that this actually makes any sense, because after all I could be an attacker trying to hijack someone else’s account. So I filled out my information, and I’m taken to another page that asks me for my current password, my new password, and a confirmation of my new password. I go on and enter my new password. Paypal tells me that my password is too weak because so far I’ve entered only letters, but I don’t mind, I know the juicy stuff comes in a little bit, but then, all the sudden paypal says that I’ve entered all the allowed characters, which are not many (20). WTF? Why? Why can’t I have a long password, Why?!? Paypal just made me less secure by limiting the amount of characters I can use for my password. Are they going to start charging for extra characters now? I would pay 1 cent a piece, no kidding, as long as I could get a longer password, but then that would be something really bad wouldn’t it? Imagine a company that charges you to let you choose the password you want. Wouldn’t that be something?

Anyway, I decided to leave my current password as it is. Thanks PayPal!

What is the Big Deal

Twenty characters are enough for a password, aren’t they? After all, people want to get 4-letter passwords so they don’t forget them, but that is just stupid. I can see a valid reason to set a minimum amount of characters, but why limit the maximum amount?

You may be wondering why it is such a big deal for me. Let me explain how I set my passwords.

Chosing Long-A** Passwords that You Can Remember

I start buy choosing something memorable to me, for example, I really like the movie V, so I may want to use a base for my password like:

Remember remember the fifth of november

The first problem I see is that there are spaces, and for some stupid reason a lot of websites don’t want you to use spaces in your password, so let’s fix that:


There you have it, 34 freaking characters, and this is just the base of my password. I should note that by removing the spaces I just made it harder to type it, which is a bad thing, I’ll explain why later.

Now that we have a strong base, lets add a little bit of other characters. I will use a memorable date, for example. Note that I’m just choosing a random date here in this article, but in real life I would choose a really memorable date, but that few people know, such as the date of your first kiss, if you remember that.


We are now at 44 characters, our password has uppercase letters, numbers, and non-alphanumeric characters. Now this base is easy to remember because I’m using a memorable phrase, and a memorable date, and the stragne characters are just separator. In fact, you could use them in the phrase as well:


49 characters so far. Now, lets make it unique for each site:









Good luck trying to guess those passwords, even with a computer. However, Paypal won’t let me use any of that, what a stingy website. They will only give me 20 characters. What am I supposed to do with that?

Following this recipe makes it easy to create long passwords that are easy to remember, and extremely hard to guess. Not only that, but it makes it easy to change them too. For example, if I wanted to update my twitter password, I could just add something to it, which makes it even stronger:


That is 76 characters long, and I can guarantee you that I will remember it tomorrow without having to memorize it, because it is made up of stuff that I already know. But I sure won’t remember this:


Which is a password generated by a random password generator, which by the way, says that it is “easy” to remember as:

“SIERRA INDIA golf zulu @ OSCAR HOTEL india sierra 4 ! , ECHO romeo whiskey”


But a lot of websites recommend that you use one of those random password generators.

Hard to Write, Hard to Remember; Bad Combo

I mentioned before that by not letting me use spaces, websites make passwords harder to write, and that is not good. The reason is that if my password is hard to write I will have to either write it slowly, or attempt to write it a few times. This is bad because it gives people time to see what you type. You should be able to type your long-a** password at lighting speed. I don’t care if your system doesn’t take spaces for some stupid reason, fix that on your end. Get rid of my spaces before sending my password, or even better, fix your stupid system! I should be able to use as many characters as I want, and any of them. The password needs to be easy for me to remember, but harder for people and computers to guess. However, a lot of websites force me to create passwords that are easy for computers to guess and hard for me to remember, and type.

I don’t think there is any need to mention this, but if my password is hard to remember, then I’m already in a bad situation because I will have to write it down somewhere.

Why are We Using Passwords Anyway?

Seriously, why haven’t we come up with a better way? Oh, right we have. There is Open ID, and Mozilla Persona, to mention a couple, but event those are not the perfect solution. There has to be a better way, and if we look hard enough we will find it. But we’ve settled for less. We have accepted password as the one way to do authentication, and to make it worst, we have made it hard for people to use passwords, and we have misguided them to believe that a good password should be hard for them to remember, type, and guess. That is why people think that a random number is a good password, even if it consists of only 5 digits.

There is a lot more I can write about passwords, but the ultimate thought would be that we need to get rid of them. However, as bad as it is, we have to stick to passwords for now, but I wish websites would at lease make that easy, and safe.

Finally, you should check out this comic by XKCD:

Writing HTML Right

One of the things that really bothers me in today’s web environment, is that it seems like everything has been made generic. I’ve been noticing an increasing number of websites that use class names like col-md-4 or some other meaningless class name like that one. There is nothing wrong with this, until you try to automate a task such as scrapping a website.

I’ve talked about web scrapping in the past, and I’ve also talked about how the semantic web is just a dream that we should be forgetting about. However, we shouldn’t go around giving only meaningless class names to our tags just because we don’t care much about a semantic web.

When I work on a website, I write the markup first not really caring how I’m going to display it. That is the beauty of CSS, and that is why a few years ago the CSS Zen Garden was such a big deal. When we write markup we shouldn’t care about what the element that we are describing will look like, but rather about what it is. This sounds a lot like I’m going back to web semantics, and maybe I am, but just a little bit, and not for the sake of semantics, but for the sake of separation of concerns.

The whole point of having CSS live in a different space from HTML was to keep them separate, but now we are tying them up together very tightly. This is not good.

The next time you create a website, try this:

  1. Get out your text editor, and write markup.
  2. For every tag you write, think about what it describes, and assign a class name to it that describes that. For example if the element you are writing markup for is a sidebar widget, then give it a class of widget.
  3. Do number 2 for each tag you write. If you cannot think about what a tag is doing there, it may not be needed.
  4. Use IDs. I know a lot of people nowadays say you should not use IDs in your CSS. Not using IDs in your CSS is probably a good advise, but you are writing HTML here.

You should also make sure you are using the correct tag. There are a lot of tags in HTML, and you should always make sure you are using the correct one that describes what is going on in your markup. A few years ago, a group of people decided that the web needed badly an article tag, so they added it. Personally, I think it is stupid, but I use it because it conveys more meaning that a generic div tag. The reasons why I think it is stupid that HTML 5 implements a bunch of new tags are outside the scope of this article, so I won’t go into that, but the point is that if there is something available for you to use, you should be using it.

Here is an exercise you can try:

Write a simple page using nothing but div tags. This page should be simple, but still have a variety of things like some title, a sidebar, maybe some quote from a famous person, or some contact information; just add anything you think of, but use only div tags. Once you are happy with your page, open it up in a browser and try to make sense of it. You can even take a screenshot of it.

After looking at your page for a little bit, go back, and start replacing the div tags for something more meaningful. For example, if you have a title, use and h1 tag, or if you have a a paragraph, use a p tag. Look at each div tag you wrote, and think about what it is doing, then, find the tag that better fits that description. For a list of html5 tags, you can go here: and for a list of tags organized semantically, you can go here:

Once you are done, look at your page in the browser again, and you will notice the huge difference that using the right tag does.

There is a lot more to writing HTML than this, but one thing is certain. When you write HTML you should think about content, not presentation. HTML describes what something is. After you are done, you may go back and add some classes that aid you in your CSS such as classes for columns, or font awesome classes, but do that only once you’ve made sure your HTML is marked up, properly.

Fixing Error Code 32 when Using the Twitter API

Yesterday, I started working with the twitter API. It’s been a long time since I did anything with it, and now that it uses Oauth, I found that it is a bit less intuitive, but after a couple of times you get used to validating requests, and to the tedious signing process. However, I noticed that some of my requests were failing with an error code of 32. This is the actual message I got as response:

{“errors”:[{“message”:”Could not authenticate you”,”code”:32}]}

This message is really useless considering there are many reasons why your authentication could be failing. By looking at that error message all you know is that you could not be authenticated, but you don’t know why. The reasons, as it turns out, are not as intuitive as you may think.

When someone tells you that you could not be authenticated, your first thought is to consider whether you have the right credentials. But once you’ve verified that you do, what other options do you have? Well, it could be that the signature is incorrect because you did not order the parameters in the right order. It could also be that your server is using a different timezone. Who would ever think of these as reasons for failing authentication? That is where the message becomes useless.

However, these two instances are pretty much the main reason for failed authentications, so a search will most likely give you a variation of any of these two possible reasons. However, what do you do after you are certain that none of them are the reason? You are pretty much out of luck.

One of the first things I do if something is not going according to plan, is to check that the signature being generated is the same one that twitter generates when using their Oauth Tool. For this you need to know how to modify the timestamp, and the nonce in whatever framework you are using. In mi case, I’m just doing everything by hand, so editing those values is a matter of editing a couple variables. If the signature, the Authorization string, and the signature string coincide with the ones generated by twitter, there should be no reason why your request should fail. However, despite of all 3 matching, my requests were being failed over and over.

The first thing that I noticed that made little sense was that some request were successful, while other were not. This made no sense. Why would twitter fail to authenticate one request, but not another one when both use the same credentials. This is yet another instance of that message being totally useless.

I decided to go over my requests and find out what was different. I noticed that the request that failed were the ones where data was being posted. I was using php’s curl to do the requests, so I decided to go and take a look at the documentation just to make sure I was doing everything right.

I discovered that PHP can take POST parameter in two ways when using curl. The first one is a string of key value pairs separated by ampersands:


The second one is using an array:

array (
  'key1' => 'val1',
  'key2' => 'val2'

Well, it turns out that if you use an array, php will set the Content-Type header to multipart/form-data, but then twitter will fail to authenticate you. Although the reason for failing the request may be an acceptable one, the message you get back is not, because it gives you no indication whatsoever as to why the request failed.

Next time you encounter that error code 32, remember to check you headers as well as your data, along with the server timezone, the order of your parameters when building the signature string, and you may as well check that you are not wearing anti-twitter outfit, just to be sure twitter will not deny your request because of that too.