EnvironmentError: “mysql_config not found” While Installing MySQL-Python

While running “pip install mysql-python” on a fresh installation of Linux Mint 17, the following error occured:

Traceback (most recent call last):
  File "", line 17, in 
  File "/tmp/pip_build_root/MySQL-python/setup.py", line 17, in 
    metadata, options = get_config()
  File "setup_posix.py", line 43, in get_config
    libs = mysql_config("libs_r")
  File "setup_posix.py", line 25, in mysql_config
    raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found

Cause:

This problem is caused by the ‘mysql_config’ file not being in your PATH, likely because it’s not there at all.

Solution:

Ensure that the libmysqlclient-dev package is installed:

sudo apt-get install libmysqlclient-dev -y

If you are still getting the error after this, ensure that your MySQL library is in your path:

echo $PATH

If that’s still not working, you can edit the “setup_posix.py” file and change the path attribute to match your local installation:

mysql_config.path = "/path/to/mysql_config"

(Note that the python-MySQL can also be installed with apt-get install python-mysqldb)

EnvironmentError: “mysql_config not found” While Installing MySQL-Python

Error loading docker apparmor profile –

I recently installed Docker and came across an error while starting the daemon:

INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] +job init_networkdriver()
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
INFO[0000] -job init_networkdriver() = OK (0)
INFO[0000] WARNING: Your kernel does not support cgroup swap limit.
FATA[0000] Error loading docker apparmor profile: fork/exec /sbin/apparmor_parser: no such file or directory ()

The error indicates that /sbin/apparmor_parser couldn’t be found. The easiest route is probably to just apt-get install apparmor, but I didn’t want to add apparmor to this machine for a number of reasons. Without apparmor, I couldn’t care less if the profiles are parsed, so I decided to substitute the binary with a shell script.

In this instance, the fork call probably just needs to find a file to execute and receive an exit code of 0.

sudo emacs /sbin/apparmor_parser
#!/bin/bash
# Dummy program
exit 0;

After closing the file, be sure to chmod +x /s/bin/apparmor_parser to make it executable. This technique works because the program is looking for a binary to execute and will most likely check the return code (or output of stderr) of the callee. Note that this won’t always work, as some scripts and programs rely on program output, or a lack of program output (if not stderr).

If modifying programs in /bin/ or /sbin/ makes you uneasy, you can always add them to ~/bin/apparmor_parser. Recent versions of Ubuntu and Mint include a statement in .bashrc to include ~/bin in the PATH if it exists. (Of course, you can always export any arbitrary folder to your PATH too.)

 

Error loading docker apparmor profile –

Default parameter values in bash

Since it’s often easier to understand with an example rather than a detailed explaination, here are a couple of examples illustrating how to handle default variable values in Bash. In addition, it’s often useful to be able to use environment variables (e.g., to specify the path to a binary in a build script), so I’ve included that as well. All of the code is available on GitHub Gists.

#1 – Specifying a default value for a Bash variable

Here’s a quick and easy method to provide default values for command-line arguments in Bash. It relies on Bash’s syntax for accepting default variable values, which is ${VARNAME:-“default”}. The double quotes allow anything that normal variable expansion allows as far as I can tell.

#2 – Specifying a default value in a Bash function

This is really no different than above, but illustrates how you can rely on the. In this example, the interface name ($iface) can be specified as the first parameter. Each of the functions then uses the same method to gather its arguments, resorting to the “global” defaults (CLI args) if not specified. (Note that in Bash, variables are global in scope by default. To override this behavior, use the local keyword)

#3 – Command output as default variable values

It’s also simple to use the output of an evaluated expression as the default value. This is great for getting system information (username, current working directory, etc.) or information that is easily generated on the command line — date constructs, random passwords, etc.

#4 – Override default values with environment variables

The following script uses the ‘htpasswd’ and ‘openssl’ binaries, which are usually specified by the full path (output of ‘which htpasswd’). By prefixing the standard definition with ${ENV_VAR-$(which htpasswd)}, you can now ‘override’ the default value with the use of an export  statement.

The script also takes an optional first and second parameter, which default to the current user and a random password respectively. If a password wasn’t specified, show the generated password to the user (otherwise, don’t display raw password info).

Example #5 – Just Because

Just a shorter, harder-to-read version.

Example #6 – Exit with an error if parameter is empty

Sometimes the input must come from the user, and the script needs to terminate if the user hasn’t specified the correct arguments. This can be done by using a question mark instead of a default value:
This results in output like:

./foo.sh: line 2: 1: You must specify a username

Example #7 – Exit with an error if binary not found

This could probably be made shorter, but it works. This statement tries to fill the value of $ifconfig with either $IFCONFIG or the output of which ifconfig. If both are empty, the boolean OR || is triggered, which echos an error and returns 1. Still unsatisfied, the final OR is triggered, causing the script to exit with status 1. Structuring your exit codes like this allows this script to be used in a similar fashion inside of other scripts or crontabs.

Default parameter values in bash

Extract one table from a mysqldump file

I recently had to restore a MySQL table from a nightly database backup. Given the size of the dumpfile and the fact that only one table needed modified, I ended up using sed to extract the table:

sed -n '/CREATE TABLE.*table/,/UNLOCK TABLES/p' full_database_backup.sql > table.sql

The -n flag is an alias for –quiet, which suppresses output other than what sed is told to print.  The p at the end of the expression tells sed to print the matches to the screen.
I’ve created a bash script to handle this, and placed it in /bin/dbextract. It’s intended to be used the same way  as the actual command, in that output is directed to stdout. (You’ll want to redirect it with “> outfile”)

Extract one table from a mysqldump file

MySQL datadir on different partition

This writeup will walk you through installing MySQL with the data directory on a separate partition. Although a new install is pretty straightforward, we ran into some quirks when trying to move the data directory on an existing installation. For this tutorial, I’ll be using an otherwise-fresh Ubuntu 14.04 install with MySQL already installed.

The default MySQL data directory (where the database files are stored) is in /var/lib/mysql. I’ll be moving this to a disk mounted at /mnt/SAN for the purpose of freeing up disk space on the VM. (I’m not going to discuss the benefits and drawbacks of doing so, as that’s beyond the scope of this article. I assume that if you’re here, you’ve already determined a need to mount the data directory on another filesystem.)

There are a couple of steps involved in this:

  1. Create the new directory
  2. Stopping the MySQL service
  3. Copying the files to the new location
  4. Editing /etc/mysql/my.cnf
  5. Editing the AppArmor profile
  6. Reloading the AppArmor profile and restarting MySQL

The new data directory will be located at /mnt/SAN/mysql, which will have to be created. When creating this directory, ensure it’s owned by the mysql group and user, and set permissions to 700.

sudo mkdir -p /mnt/SAN/mysql
sudo chown mysql:mysql /mnt/SAN/mysql
sudo chmod 700 /mnt/SAN/mysql

Next, stop the MySQL service:

sudo service mysql stop

or

sudo /etc/init.d/mysql stop

Once you’ve set up the new data directory on your mounted partition, copy the files over:

cp -dpR /var/lib/mysql/* /mnt/SAN/mysql/

The -dpR flags do the following:

-d prevents symlinks from being followed
-p preserves ownership, timestamps and permissions
-R copies recursively

Once the files have copied, ensure the permissions match those of the original data directory (/var/lib/mysql/). Make sure the new mysql directory has the correct ownership and permissions as well!

At this point, a directory listing of /mnt/SAN/mysql should match /var/lib/mysql exactly.

Now, we’ll edit the MySQL config file, located at /etc/mysql/my.cnf. I recommend backing this file up first!

sudo cp /etc/mysql/my.cnf /etc/mysql/my.cnf.bak
sudo emacs /etc/mysql/my.cnf

Look for the “datadir” param, which should be set to the default value of “/var/lib/mysql”

[mysqld]
#
# * Basic Settings
#
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking

Change this value to your new mysql data directory (/mnt/SAN/mysql) and save the file.

If you try to start the MySQL service now, it’ll likely fail because AppArmor sees it accessing a directory it’s not supposed to. Dmesg will show errors like this:

init: mysql main process ended, respawning
 init: mysql post-start process (14005) terminated with status 1
 apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=14020 comm="apparmor_parser"
 init: mysql main process (14032) terminated with status 1
 init: mysql respawning too fast, stopped

In order to correct this, we’ll have to tell AppArmor to allow mysql to read/write to the new data directory. Open up the MySQL AppArmor profile:

sudo emacs /etc/apparmor.d/usr.sbin.mysql

Comment out the lines pertaining to the old data directory, and add the new data directory to the AppArmor profile:

...
#/var/lib/mysql/ r,
#/var/lib/mysql/** rwk,
/mnt/SAN/mysql/ r,
/mnt/SAN/mysql/** rwk,
...

Once this is done, reload the AppArmor profile:

sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.mysql

If all the permissions are correct, the mysql service should now start:
sudo service mysql start

or

sudo /etc/init.d/mysql start

If you’re still running into issues, make sure that:

  • The new data directory has the correct permissions
  • The AppArmor profile is correct
  • You’ve started the mysql service (mysqld)
MySQL datadir on different partition

MySQL Database Backup With mysqldump + netcat

I ran into a situation recently where I had to copy a database, but didn’t have the disk space for a full dump. Instead of rsync or scp (which can be done over netcat as well), I opted to pipe the output of mysqldump to netcat and transfer the data directly to the other server.

My setup was Ubuntu server 12.04 and Linux Mint 16 (client). First, start netcat on the client machine on an available port (e.g., 1234) and redirect the output to the desired .sql file:

nc -l 1234 > backup.sql.gz

On the server, we’ll route the mysqldump output through a gzip wrapper and into netcat. In this example, the destination machine (above) is 172.21.1.2, and should hopefully be listening on port 1234 already. (It is worthwhile to note that you should supply the MySQL password in the command itself, rather than just using the “-p” option. The password prompt will be transmitted to the listening machine,  which will end the netcat session. Security-conscious users can enter a space before the command to keep it from being stored in bash history.)

mysqldump -u root -pP@$$w0rd db_name | gzip | nc -w1 172.21.1.2 1234

MySQL Database Backup With mysqldump + netcat

Space Before Command Not Working on Linux Mint

Bash has a useful feature where it won’t store command history if the command begins with a space.  This was Ubuntu’s default functionality, and it’s useful for commands like:

mysqldump -u root -prootpassword db_name > db_name.export.sql

Unfortunately, this wasn’t enabled by default on Mint, and I was able to do so by adding  the following line to my ~/.bashrc:

HISTCONTROL=ignorespaces:ignoredups

The ignorespace flag won’t store commands whose first character is a space, and ignoredups intuitively causes bash to not store duplicates (when entered sequentially). If using the two together, you can also specify ignoreboth, rather than spelling it out.

ignoredups vs. erasedups

There’s another setting for duplicates – erasedups. While ignoredups prevents repeated entries, erasedups will erase any older instances of the command. I’ll use this series of commands to illustrate the difference.
emacs test.py
emacs test.py
python test.py
emacs test.py

With ignoredups only, your history would look like this:
emacs test.py
python test.py
emacs test.py

With erasedups, your history would look like so:
python test.py
emacs test.py

Of course, you’ll need to restart your bash session (log out/log in, type ‘bash’, etc) for these changes to take effect.

Space Before Command Not Working on Linux Mint