The Wordpress Auto Draft

Disclaimer: I’ve been away from the wordpress world for over a couple of years now, so I don’t know if this is common knowledge in the wordpress circles or not. Heck! I don’t even know if this is something that has always happened, or if it is something that was implemented while I was away from WP, but it is definitely something that threw me off a bit recently.

After some time away from WP, I decided it was time to go back and try all the new features that have arrived to WP since I left it a while back. I started developing a small webapp. I know I could have used something better suited for the task, like Laravel, but the goal here is to get WP to do something that clearly stretches its limits. One of the things I wanted to do was to assign an ID to custom-type posts. I didn’t want to rely on the post’s id, but rather create an md5 hash of the post’s title. The key part was to create it when the post was first created and then never change it again, even if the title changes. The post’s id would have suffice, but remember that this is an exercise to get back into wordpress development.

My first instinct was to use the `save_post_{$post->type}` action hook. This action passes a boolean parameter to the callback function that specifies if the post being saved is an update of an existing one. If it isn’t, we can assume the post being created is brand new. Relying on this boolean seemed like the obvious choice since it would allow me to save the md5 hash only once when the post is first created, and never touch it again. So, I did it this way. However, later, when I decided to display a custom row in the all-posts table that displayed the post’s custom id, I realized that all the posts had the same custom id. How was this possible? Digging into wordpress I found out that every time you open the new post page, a new post is saved in the database with title “Auto Draft” so all my md5 hashes were being generated from that “Auto Draft” title. No wonder they were all the same!

So, how does this happen?

When you first open the new post page (post-new.php) a function named `get_default_post_to_edit` is called, which, among other things, creates a new post in the DB, creates a new post object for it, and passes it along to other functions and hooks. This is actually a good thing because it makes sure you are always working with a real post object that has a DB record, and an ID. However, if you aren’t aware of it, it may get you scratching your head for a little while when something doesn’t happen as you’d expect it to happen. Knowing these kind of subtleties is what makes the difference between a good, and excellent developers.

Read, the docs, but more importantly, read the codes!

Advertisements

Nested Form Input in Drupal

HTML forms are made up of input controls that, when submitted, are converted into an array of values. For example, consider the following form:

<form method="post" action="/">
<input type="text" name="input1" />
<input type="text" name="input2" />
<input type="submit" name="send" value="Send" />
</form>

When a user fills out this form, and submits it, the data is sent by the browser using the POST HTTP method in this fashion:

input1: value
input2: value
send: send

If you were to read that with PHP, you would have to access the `$_POST` array, which would contain those key-value pairs. However, some forms are complex enough that flat arrays like that make it hard to work with the data. In those cases it is useful to get multidimensional arrays. HTML allows you to do just that:

<form method="post" action="/">
<input type="text" name="persons[0][name]" />
<input type="text" name="persons[0][address]" />
<input type="text" name="persons[1][name]" />
<input type="text" name="persons[1][address]" />
<input type="text" name="persons[2][name]" />
<input type="text" name="persons[2][address]" />
<input type="submit" name="send" value="Send" />
</form>

In PHP that results in the following `​$_POST` array

Array
(
 [persons] => Array(
   [0] => Array(
     [name] => john
     [address] => smith
   )
   [1] => Array(
     [name] => jane
     [address] => doe
   )
   [2] => Array(
     [name] => jason
     [address] => foo
   )
 )
 [send] => send
)

As you can see, the information for `persons` is now nicely arranged in an array of arrays, each of which contains the information associated with a person.

If you want to replicate that same structure in Drupal, it is quite easy. Using the Drupal Form Api you can easily create form arrays that Drupal knows how to render. But those forms usually submit data in the flat array form. If you want to get multidimensional array like we did above, you need to use the `#tree` attribute on the element that you want to be the root array member. For example, to get the same input we have above, you would do something like:

$form = [
  'persons' => [
    '#type' => 'markup',
    '#tree' => TRUE,
    '0' => [
      'name' => [
        '#type' => 'textfield'
      ],
      'address' => [
        '#type' => 'textfield'
      ],
    ],
    '1' => [
      'name' => [
        '#type' => 'textfield'
      ],
      'address' => [
        '#type' => 'textfield'
      ],
    ],
    '2' => [
      'name' => [
        '#type' => 'textfield'
      ],
      'address' => [
        '#type' => 'textfield'
      ],
    ],
  ],
];

If you were to omit the `#tree` attribute on the `persons` element in the form array, you would get a flat array of values with name, and address values for the last input fields only. Also note that the nested arrays begin at the point where the `#tree` attribute is specified. For example, if `persons` was itself a child of another element called `personal_data`, the resulting POST data wold remain the same, unless the `#tree` attribute was specified in the `personal_data` element, in which case, the `persons` array would also be child to a `personal_data` member in the `values` array where Drupal saves the submitted data when dealing with form submissions.

404 on Private Drupal Files

A few days ago, it was brought to my attention that some links pointing to private images in a drupal site were not working. The images are submitted by users of the site to request an estimate on repairs for their luxurious cars, and they go to the `private` directory in the drupal site.

For those who don’t know, drupal uses different types of files, such as public, and private. Private files, with the protocol `private://` live in a different place than public ones in the server. When Drupal builds a link to the file, it uses a path that is not really the actual path on disk. The path usually starts with  `system/files/` , but in the drupal directory structure there is no directory called `system`. This means a redirection is made from that url to the actual path of the file in the system.

For some reason, however, when trying to access a private file in the site, all that was returned was a 404 error. This started to happen after the site was moved to a new server. Tracking the problem I found that the issue was that the drupal function drupal_fast_404 was being called in the settings.php file. This function takes care of returning a 404 page whenever the requested url doesn’t match certain criteria. In this case, the fix was to simply edit the 404_fast_paths_exclude to make sure that any path starting with system/files/ and ending with a recognized image extension would not fast 404.

Fixing MySQL Corrupt Table “ERROR 2013 (HY000): Lost connection to MySQL server during query”

Today I tried to drop a database in mysql

DROP DATABASE db_name;

Which resulted in a error: “ERROR 2013 (HY000): Lost connection to MySQL server during query”. At that point everything started to go wrong, to the point where the MySQL server went completely down and I was unable to run it again. In short this is how things happened:

  1. Try to drop the database, which resulted in an error
  2. Keep trying to drop it, which resulted in the same error.
  3. Try increasing the net_read_timeout and net_write_timeout on the server, as well as –connect-timeout in the mysql client. No luck.
  4. Search the web, find no useful advice, but found a possible reason “data corruption”
  5. Try to recover data with a backup of the database from the production server. Shit went dow!
  6. The server stopped and refuse to start again. Following the instructions in the stop error lead me to new search terms.
  7. We have a new problem now, do more web searching.
  8. Find a useful page which recommended disabling innodb, and setting myisam as the default database engine.
  9. After doing that, the server worked, but trying to dump the database resulted in an error: ERROR 1010 (HY000): Error dropping database (can’t rmdir ‘./db_name’, errno: 39 “Directory not empty”)
  10. I decided to manually remove the directory, which removed the database.
  11. Create the database again and use the backup to restore it.
  12. Got an error saying that the table ‘table’ already existed. ‘table’ is the first table that the backup file tries to create.
  13. Try to dump the database, but got the same error as in #9.
  14. Manually delete the database again, Create it and the do a proper DROP.
  15. Create the database again and restore from backup. It worked!

I read a few pages during this problem, and found these useful:

SOLVED: InnoDB Error: space header page consists of zero bytes: xampp

MySQL: Error dropping database (errno 13; errno 17; errno 39)

Fixing PHP’s ‘Warning: simplexml_load_file(): I/O warning : failed to load external entity’

Today I had to re-visit a Drupal module I wrote in 2015. The module parses an external XML file which it loads using simplexml_load_file. When I opened the version of the site that lives in my local server, and visited the page that uses the module, I got the error in the title of this entry. I checked php.ini to make sure allow_url_fopen was On. It was.

Searching the web I came to a bug report in the php website. It was suggested to restart the apache server since the contents of resolv.conf may have changed after the server, and php got their DNS servers entry. This made sense to me, since I have switched networks since turning the computer on. I gave it a try, and it worked:

service httpd restart

Other info:

  • The location of php.ini in my case, running Fedora with php installed using dnf, is /var/php.ini
  • The location of resolv.conf is /etc/resolv.conf

The sources I visited:

Opening port 80 with iptables

A few days ago I was contacted by a client whose website had been offline for a month. The site has a history of going offline because of server overloads. The client is OK with the occasional down time as the site is just a pet project. Every time the server goes down we usually restart the MySQL server and everything goes back to normal, but this time it was different. Instead of restarting the MySQL server, the whole server was restarted by power cycling it, and that is when everything went wrong.

When the site came back up, the HTTP and MySQL server had to be manually started, and the program that communicates with the server control panel for status reporting was down as well. It had to also be started manually. However, after all of this, the site was still off line. Trying to access the website resulted in an error like the one you see when you are not connected to the internet.

I decided to curl the site, and in return I got an error saying “No route to host”. Pinging the server worked, but form some reason I could not connect to it, because of this I knew it was not a DNS problem, since the url resolved to the server IP correctly, but the connection was refued. Could it be a server issue?

I decided to take a look at the server config files, only to find out everything was configured correctly. At this point I was absolutely intrigued.

I decided to search the interwebs to find out what people said about the “no route to host” problem, and, as I thought, it was a connection problem. The machine was not refusing the connection, the network was. I confirmed that by fetching localhost from the server. I got the expected result, so the server was not refusing the connection.

It was at this point that I decided to concentrate on the network side of things. I started by determining the port the server was listening on. I did this by checking the server configuration files. When I confirmed the server was listening on port 80, I decided to check if port 80 was open. Using iptables I determined that it was not. From there, it was just a mater of opening the port. I did that by running

iptables -I INPUT 1 -p tcp --dport 80 -j ACCEP

Here are some documents I consulted while trying to solve this problem. Some proved useful, others did not. It is worth mentioning the server runs Fedora.

http://unix.stackexchange.com/questions/29319/fedora-16-iptables-installed-but-no-iptables-service-available
https://www.cyberciti.biz/faq/linux-iptables-firewall-open-port-80/
https://fedoraproject.org/wiki/How_to_edit_iptables_rules
http://superuser.com/questions/720851/connection-refused-vs-no-route-to-host

Some linux FS management tips

A few days back, I decided to upgrade from Fedora 19, to 24, and decided to do so using a usb stick. I downloaded the installer, and copied into the usb stick using dd.

A couple of days later, when I wanted to use that usb stick, I found out I could do nothing with it since its file system was write-protected. Trying to format the device did not work for the same reason. Using the disks utility in Fedora 24 showed 3 partitions in the usb stick, but trying to delete them would result in an error related to block size. The driver reported one size, but linux reported another. After a bit of searching, I came to an ask ubuntu answer that recommended using mkfs.vfat to solve the problem.

  • You can use mkfs.vfat, or one of the other mkfs.* programs to format a stubborn drive, or any other drive for that matter.

Being able to fix this problem, I decided to try with an old usb stick I got at a meet up back when Barnes & Noble were preparing to launch their android-based reader NOOK. The usb stick has never worked. It doesn’t mount, and reviewing it, there is no partition in it, let along a file system. I tried making a partition using the disk utility, but failed. I decided to use parted in the command line, but no luck, the device was write-protected. After a bit of searching, I found out you can use hdparm on write-protected devices to make them writeable. This did not work on my usb stick, which leaves me thinking that the device is just damaged.

  • You can use parted to make partitions in a device
  • You can use hdparm to set various option on drives, like write-protection.

Speaking about partitions, having a usb stick with more than one partition in it can be quite useful, and cool. Just remember that dumb windows doesn’t mount all of them, only the first one.

Other tips:

  • Use mount to find out how a device is mounted.
  • Use dosfsck to check and repair DOS file systems

I hope these tips prove useful at some point in your life as they have done in mine. Thanks to all that share wisdom around the net, what would we do without them…

Fixing Fedora 24 dnf problems.

The short story:

If you are getting the following error when trying to install software using dnf in fedora 24:

python3: /builddir/build/BUILD/hawkey-0.6.3/src/python/exception-py.c:101: ret2e: Assertion `0′ failed.

You may try doing:

dnf --disablerepo=updates <package>

Where <package> is the thing you are trying to install.

The long story:

Today, I was working on a site on my newly installed Fedora 24, but the drupal-based site failed to show anything after I updated a block. All non-admin pages would just show blank. So I started tracking code, and at some point it occurred to me that maybe I had to run the update script, which fixed the problem. I still don’t know why updating the content of a block would cause the site to break, but my guess is that it has to do with drupal caches. This because the function that failed is called “block_page_build”. Anyway, before I could run the update script, drupal was complaining about some missing php components, so I wanted to install them.

I went ahead and did

dnf install php-xml

but that returned an error (see the short story above). I didn’t know what to do, so I started searching the web, but found nothing. Then I turned to the dnf manual. In there I saw you can add -v to get a more verbose output. Running dnf with -v got me this:

cachedir: /var/cache/dnf
 Loaded plugins: config-manager, protected_packages, generate_completion_cache, Query, builddep, langpacks, debuginfo-install, download, needs-restarting, playground, reposync, noroot, copr
 langpacks: No languages are enabled
 Error reading file : /var/lib/dnf/plugins/langpacks/installed_langpacks as it does not exist
 initialized Langpacks plugin
 DNF version: 1.1.10
 repo: using cache for: fedora
 not found deltainfo for: Fedora 24 - x86_64
 not found updateinfo for: Fedora 24 - x86_64
 repo: using cache for: google-chrome
 not found deltainfo for: google-chrome
 not found updateinfo for: google-chrome
 repo: using cache for: updates
 python3: /builddir/build/BUILD/hawkey-0.6.3/src/python/exception-py.c:101: ret2e: Assertion `0' failed.
 Aborted

Look at that! the problem happens right after trying to use the updates repo.

I had read in the dnf man page that there is a location where the repo files are kept, so I went and moved all update repos out of the directory where they are kept, and run dnf install again. It worked.

Well, all that was left to do was find a proper way to ignore repos when doing installs, and the man page had the answer: –disablerepo

So, that is the story. Keep coding kids.

The unset($args[0]) PHP Pattern

Suppose you are working on a system, and want to allow users to use so-called hooks to hook into the system and change it’s behavior. Think of it as wordpress’ filter/action kind-of-thing. This is in a way, something similar to events in javascript, and the purpose is basically the same: you want to do, or let others do something when something else happens, and potentially change the way the system works, or extend its functionality.

Implementing this kind of feature is surprisingly simple. All you need is a way to let people subscribe to your hooks, and a way to call all subscribed members for a certain hook at some point. The implementation details are not really our concern right now, but what happens when you call each subscribed member.

In order for hooks to be useful, you need to be able to pass relevant information to the subscribed members at call time. For example, in Javascript, native events get passed an event object that contains all sorts of useful information. The way you pass this information to the subscribed members will depend on your implementation of the subscribe method you use.

One pattern I’ve noticed working with drupal is unset($args[0]).

Drupal uses a hook system that allows developers to hook into the system simply by following some function naming conventions. The drupal specifics are not important right now. What is important is the way drupal calls its hooks: it uses a function called module_invoke_all.

The module_invoke_all function, in its definition, takes a single argument: the name of the hook. We already mentioned the importance that being able to pass information to the subscribed member has, so why is it that the drupal function to invoke hooks takes only the name of the hook as argument? How do you pass the relevant information?

PHP, just like other programming languages allows you to pass more arguments to a function that its formal parameters. The way you access the extra arguments passed to the function varies depending on the language. In php all arguments passed to a function are kept in the $args array.

Lets say you call module_invoke_all as follows:

module_invoke_all(‘my_hook’, ‘something cool’, ‘something interesting’);

Inside the function, the $args array will have 3 members, one for each argument passed to it. So, if you want to pass those values, minus the hook name to the subscribed members, all you need to do is get rid of that, and pass the resulting array to the subscribed members. That is why unset($args[0]) does.

I’m not sure this is the best way to do it, but it is certainly interesting. If you want to read the full code for the module_invoke_all function, just check out the drupal documentation on it (https://api.drupal.org/api/drupal/includes!module.inc/function/module_invoke_all/7)

Learning Programming

Over the years, I’ve seen many people struggle with learning how to program. I’ve never really understood what the big deal is. When I first learned programming, I did it on my own, with a bunch of internet tutorials, the language documentation, and some time. I kept being surprised by how easy it was to do the things I had seen on the computer. I understand we all are different, but I believe the real reason why many people struggle to learn programming is because of the way it is taught.

My initial take on programming was Javascript. I didn’t think I was learning programming. All thought was that I was learning how to make websites. I kept thinking that programming must be something different, because people always say programming is hard, and what I was doing was really easy. I learned Javscript along with HTML, and CSS, and honestly, I used to think real programmers did things differently. It was surprising that calling alert would pop up an alert window, just like that. It could not be that easy at the professional level, right?

Learning programming, and learning a programming language a two different things -quite different, actually- that are usually thought of as being the same thing. Because of this, we tend to want to teach people both things at the same time, and that is confusing. Learning a programming language means learning all the details of it, the syntax, the grammar, the definitions, the different components, data types, and operators, among many other things. Learning programming, means learning how to do things. Doing things is more fun, and most people learn by doing. Back when I was calling alerts, or prompts, I had no idea that those things were functions, and that I was passing strings to them as parameters. I had no idea what the signature of a function was, or what the difference between arguments and parameters was. All I knew was that a hello world program in Javascript was easy to write.

Many years have passed since my first programming adventures. I’ve learned so much in those years, but I feel like I didn’t learn enough. I am a lazy programmer; I waste time on this and that, I read, and then forget what I read. I’ve learned a few more programming languages, but I didn’t really learn how to program in those languages. For example, I learned python, I read the entire python documentation, and I built some python programs a couple of years ago. Today, I’m learning python again. This time I’m not worried about the documentation, I just want to learn to program in python, because one can easily forget all the details that are in the docs, but it is harder to forget how to do things once you learn them.

I could easily criticize the way programming is taught in school, but I won’t. I’m more interested on talking about how programming is learned outside of the classroom. One thing you must learn, if you want to become a programmer is that you need to read. Reading a book or two is fine to get your feet wet, but after that, most books become monotonous, more of the same. What you really need to do is read code.

If you wanted to learn how to write novels, it would be OK to read one or two books on grammar, and such, but that will teach you the rules, and rules are boring. If you want to become a great author, you need to read novels. One of my favorite quotes from the film industry is “I didn’t go to film school, I went to films” There is so much one can learn from just looking at how others do things.

I’m learning python once again. This is the last time I learn python. My new strategy is simple. Read through the book Learn Python the Hard Way, and that’s it. The book, plus the knowledge I keep from that Sunday morning I read the entire documentation should be all I need to go out to github and start ready code. Reading code will make me a better programmer, not just in python. Once we learn something, we tend to think of ways to apply that knowledge elsewhere.

Wish me luck.