Some linux FS management tips

A few days back, I decided to upgrade from Fedora 19, to 24, and decided to do so using a usb stick. I downloaded the installer, and copied into the usb stick using dd.

A couple of days later, when I wanted to use that usb stick, I found out I could do nothing with it since its file system was write-protected. Trying to format the device did not work for the same reason. Using the disks utility in Fedora 24 showed 3 partitions in the usb stick, but trying to delete them would result in an error related to block size. The driver reported one size, but linux reported another. After a bit of searching, I came to an ask ubuntu answer that recommended using mkfs.vfat to solve the problem.

  • You can use mkfs.vfat, or one of the other mkfs.* programs to format a stubborn drive, or any other drive for that matter.

Being able to fix this problem, I decided to try with an old usb stick I got at a meet up back when Barnes & Noble were preparing to launch their android-based reader NOOK. The usb stick has never worked. It doesn’t mount, and reviewing it, there is no partition in it, let along a file system. I tried making a partition using the disk utility, but failed. I decided to use parted in the command line, but no luck, the device was write-protected. After a bit of searching, I found out you can use hdparm on write-protected devices to make them writeable. This did not work on my usb stick, which leaves me thinking that the device is just damaged.

  • You can use parted to make partitions in a device
  • You can use hdparm to set various option on drives, like write-protection.

Speaking about partitions, having a usb stick with more than one partition in it can be quite useful, and cool. Just remember that dumb windows doesn’t mount all of them, only the first one.

Other tips:

  • Use mount to find out how a device is mounted.
  • Use dosfsck to check and repair DOS file systems

I hope these tips prove useful at some point in your life as they have done in mine. Thanks to all that share wisdom around the net, what would we do without them…

Advertisements

Write Less Code, by Writting Intellingently

I am currently working on a project, and when I say currently, I mean it literally, as in right now. But I think it is worth making a parenthesis to write this entry. I just wanted to point out the huge difference that can be made in the code you write, if you do it more intelligently.

Lets look for example at how we might write some piece of code:

var addSomething, addSomethingElse
function start(){
  addSomething = B('a[href="something"]')
  addSomethingElse = B('a[href="something-else"]')

  if(addSomething.length > 0){
    prepareAddSomething();
  }
  if(addSomethingElse.length > 0){
    prepareAddSomethingElse();
  }
}

function prepareSometing(){
  addSomething.click(handleClick('something'))
}

function prepareSometingElse(){
  addSomethingElse.click(handleClick('something-else'))
}

function handleClick(type){
  switch(type){
    case 'something':
      return someFunc;
    break;
    case 'something-else':
      return someOtherFunc;
    break;
  }
}

This piece of code has some nice “features”. For example, it divides all the work in functions that do one task only. It has a nice “dispatcher” that handles adding the click handlers to each element.In my case, instead of returning someFunc and someOtherFunc in the dispatcher, I execute another function, which then returns the function that will be listening for the events. So, as you can see there are some good things about this piece of code. However, it is too long. Many of the code lines are repetitive, and it is too verbose to my taste. We can simplify this code by identifying the pieces that repeat too much.

The firs thing we see, is that instead of having prepareAddSomething, and prepareAddSomethingElse, we can have a single prepare function which can handle the task that this function handle:

var addSomething, addSomethingElse
function start(){
  addSomething = B('a[href="#something"]')
  addSomethingElse = B('a[href="#something-else"]')

  if(addSomething.length > 0){
    prepare(addSomething)
  }
  if(addSomethingElse.length > 0){
    prepare(addSomethingElse)
  }
}

function prepare(reference){
  reference.click(handleClick(reference.attr('href').replace('#','')))
}

function handleClick(type){
  switch(type){
    case 'something':
      return someFunc;
    break;
    case 'something-else':
      return someOtherFunc;
    break;
  }
}

We have successfully replaced two functions with one, and the code still does the same thing. Notice how we realized that the argument that we pass to handleClick is the value of the href attribute of the anchor elements without the hash (#), and used that to our advantage.

Next, we notice that we only want to run prepare if the collections of elements returned by the function B (which is just an alias of jQuery’s $) is greater than zero. We can also get rid of the double if statements, and replace them with a single if:

var addSomething, addSomethingElse
function start(){
  addSomething = B('a[href="#something"]')
  addSomethingElse = B('a[href="#something-else"]')

  prepare(addSomething)
  prepare(addSomethingElse)
}

function prepare(reference){
  if(reference.length > 0)
    reference.click(handleClick(reference.attr('href').replace('#','')))
}

function handleClick(type){
  switch(type){
    case 'something':
      return someFunc;
    break;
    case 'something-else':
      return someOtherFunc;
    break;
  }
}

Since the idea here is to write less code, we are not using the curly braces “{ and }” since they are optional in this situation.

Now we have a shorter version of our code, that is still very readable, and that does the same thing as the original code. All we had to do was look at it, find the things that get repeated over and over again, and write more general functions.

This is just a quick note I wanted to make, maybe it inspires you to look at your code and try to find out where you can make it shorter.

One last thing you should notice is how it would be really hard to make one single prepare function if we didn’t have that little dispatcher function (handleClick). That is one of the reasons I like using those kind of functions.

Problems Updating Database Tables (MySQL)

I run into a little problem a few days ago when I was trying to update a database table’s structure. The table had been created with the wrong settings, and I had to go and add an AUTO_INCREMENT to the primary key. Before you get out of here thinking I’m such a stupid person, you should know that the table was created, not by me, but by a person who is bravely learning about this stuff.

The table is using InnoDB engine, which enforces constraints between tables, and this was causing me not to be able to change the table. Everytime I tried, I jut got back an error similar to this:

ERROR 1025 (HY000): Error on rename of './e3/#sql-17f3_f894' to './e3/cls_ev_items' (errno: 150)

To solve this problem, one just needs to remove the contraint, change the database, and add the constraint again.

To remove the contraint you would do something like this:

ALTER TABLE table_name DROP FOREIGN KEY relationship_name;

Then you can alter your table, in my case I did something like this:

ALTER TABLE table_name CHANGE id id INT UNSIGNED NOT NULL AUTO_INCREMENT

Then add the constraint again:

ALTER TABLE table_name ADD CONSTRAINT `relationship_name` FOREIGN KEY(`field_in_table`) REFERENCES `other_table` (`foreing_key`);

And that is basically all you have to do. Im my case, all the tables were missing the AUTO_INCREMENT, so I just DROPed them all and creted them anew with the right settings, but sometimes that is not possible.

Some links that were useful, or maybe not, are recorder here for future reference:
http://www.simplicidade.org/notes/archives/2008/03/mysql_errno_150.html
http://stackoverflow.com/questions/160233/what-does-mysql-error-1025-hy000-error-on-rename-of-foo-errorno-150-me

Editing Remote Files on lftp

Well, after that happy day when I finally set fireFTP to open remote files on vim for editing, I’ve decided that I won’t use fireFTP any more. The reason for this is basically that I want to migrate to a command-line-based life. There are things I won’t be able to do through the cl, but that is OK. All I want is to be more efficient, and the command line is one way to really be more efficient. Those tasks that I cannot do via the command line I will do them in the most efficient way I know. For example, I installed pentadactyl on firefox, and it really speeds up any kind of activity you do on firefox.

Anyway, I stated using lftp because it is a really nice ftp client. It come pre-installed on Ubuntu, or at least I don’t remember installing it, and it is really powerful. However, it does have some disadvantages, and one of them is that you cannot edit remote files.

The way you would do it is downloading the file, editing it locally, and then uploading it to the server, but that is just a really ugly way to do things. I decided I would look for a solutions. A quick search led me to this page:
http://readme.maven.pl/2007/06/15/lftp-and-editing-remote-files/
but that is not what I wanted. This script allows you to open any remote file on vim, edit it, and when you close vim, the file gets automatically updated on the remote server. This is not really a good way to do it, because many times you want to make changes, reload the page, make some more changes, and reload again. If every time you make a change you have to close vim for the script to upload the file, you will end up wasting a lot of time.

The ideal way to do it would be to upload the file as soon as it has been edited and saved even if vim is still open. Well, I just spend more than 8 hours writing a script to do just that.

As a side note, I didn’t even feel the time passing. Rarely do I spend so much time working on something straight.

I found many challenges writing this script, especially because I don’t really know shell scripting. The first problem I found is documented on http://superuser.com/questions/374406/why-do-i-get-not-found-when-running-a-script
The other challenges I faced are related to shell commands and the fact that shell commands run one after the other. That is, if you type 2 commands, the first one has to finish execution in order for the second to start executing. You can run commands asynchronously, but that is something I did not know when I started this, or at least I didn’t exactly know how to do it.

All I knew about running command asynchronously was using the & at the end of a command. I use it sometimes when I want to launch a program, but I don’t want it to block the current terminal window.

I started by modifying the script presented on maven.pl. After so many modifications I got to this:

#!/bin/bash
tempid=$$
save() {
#echo shell echo $COPROC_PID
        tempid=$1
        if [[ /tmp/$tempid$2 -nt /tmp/$tempid$2.check ]]; then 
                echo  shell echo "Uploading file " $2
                cp /tmp/$tempid$2 /tmp/$tempid$2.check;
                echo put /tmp/$tempid$2 -o $2;
        fi;
        sleep 2
        if [[ $COPROC_PID != '' ]]; then
                save $tempid $2
        fi;
}
echo get $2 -o /tmp/$tempid$2
cp /tmp/$tempid$2 /tmp/$tempid$2.check;
file2=/tmp/$tempid$2.check;
coproc terminator -x $1 /tmp/$tempid$2
save $tempid $2 $COPROC
echo put -E /tmp/$tempid$2 -o $2
rm $file2

just create a file under ~/.lftp/ named edit.sh, and write that code in it.
Then create another file on the same location and name it rc and add the following:

alias edit source -e ~/.lftp/edit.sh

Now you can open your files and edit them. They will be uploaded when the script detects that they have changed.

There are still many things that can be done, like adding the possibility of editing more than one file at the same time, but for now I’m happy with it the way it is.

The usage is simple:

start lftp:

$ lftp user@host.com

enter your password and navigate the folder where the file you want to modify is located. The type:

edit vim file.php

vim is the program you want to use to edit the file. You could use gimp for images, vim for text files, or gedit for text files, or any other text editor, or in fact any program that is suitable to edit the kind of file you want to edit.
file.php is the name of the file you want to edit.

The most useful resource while developing this was the man page for bash.

Some links that were useful, or maybe not, are recorded here just for reference:
http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/x6632.html
http://steve-parker.org/sh/functions.shtml
http://www.thegeekstuff.com/2010/05/watch-command-examples/
http://serverfault.com/questions/207067/cd-in-linux-shell-script-returns-command-not-found
http://michael.stapelberg.de/lftp_vi/
http://unix.ittoolbox.com/groups/technical-functional/unixadmin-l/how-to-check-file-exists-in-one-command-line-925140
http://vim.wikia.com/wiki/Edit_remote_files_locally_via_SCP/RCP/FTP

Not all this links are completely related to the script, and not all of them were useful. Some of them I didn’t read in their entirety or I just red over the surface without really paying much attention to them. They are there just for future reference in case I want to go back to them later. Bookmarking them seemed too

QuickTip: Opening Files from Remote Server over FTP with Vim Using FireFTP

I use FireFTP for my daily FTP needs. It’s really nice, works great, has lots of flexibility and it integrates right into firefox; What else could I ask for?

I also use vim for my editing needs. Vim is powerful, flexible, and in my experience, the best text editor out there. However, until today,I opened files from remote servers using gEdit. When I first set up FireFTP I was still a gEdit user. Even today, I occasionally use gEdit, especially when I need to scan through large files.

Today I finally decided it was time to start opening files from remote servers on vim after finding myself using a lot of the vim commands on gEdit while editing remote files. It turns out setting files to open on vim in FireFTP is quite simple:

Simply open FireFTP.
On one of the panels, right-click on a file and go to Open With, and click on Add Programs…
On the window that appears, fill in the Name, Path, and Arguments text fields like this:

Name -> Vim
Path -> /usr/bin/terminator
Arguments -> -x vim %file%

I use terminator instead of the gnome terminal, but if you use terminal, then your path will be /usr/bin/gnome-terminal

That’s it.