where is mysql persisted to ?

quick answer:  The files are persisted to the directory specified by mysql's 'datadir' variable.

To get the current value do:

- start up mysql
mysql -u -p

- use the SHOW VARIABLES command
mysql> show variables;

or like this:

mysql> show variables like 'datadir';
+---------------+-----------------+
| Variable_name | Value           |
+---------------+-----------------+
| datadir       | /var/lib/mysql/ |
+---------------+-----------------+

If you cd into the datadir, here you can find there is on directory per database and inside of each database directory, each table has it's own .frm file

MySql5 uses innodb
http://en.wikipedia.org/wiki/InnoDB

as the storage engine which is creating these files.

erlang how to fix "Can't set long node name"

For this post, my platform is Ubuntu 10.10

# Start up an erlang node with a long name as follows:
erl -name mynode


# This results in a long stack trace from erlang and the first line says:
{error_logger,{{2011,9,16},{18,1,5}},"Can't set long node name!\nPlease check your configuration\n",[]}

# The problem is the way your hostname is set. e.g.

philip@myserver:$ hostname
myserver

# now you can do
sudo hostname myserver.mydomainname.com

#next time you enter hostname you should get
myserver.mydomainname.com


# In order to make this permanent, you need to edit your /etc/hostname file and change the hostname from
myserver

# to be something like
myserver.mydomainname.com



# and now you can start the erlang node
philip@myserver:$ erl -name mynode
Erlang R13B03 (erts-5.7.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.4  (abort with ^G)
(mynode@myserver.mydomainname.com)1>

Django South: Changing a field from null = True to null = False

# We have this table
mysql> show columns from myapp_mymodel where Field = "my_field_id";

+-------------+---------+------+-----+---------+-------+
| Field       | Type    | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+-------+
| my_field_id | int(11) | YES  | MUL | NULL    |       |
+-------------+---------+------+-----+---------+-------+
1 row in set (0.00 sec)


# We want to change this table so that column Null is NO for Field my_field_id


# First create an empty migration
bin/django schemamigration myapp mymodel_my_field_cannot_be_null --empty



#Then add the forward migration:
    def forwards(self, orm):
        #db.alter_column(table_name, column_name, field, explicit_name=True)
        db.alter_column('myapp_mymodel', 'my_field_id', models.ForeignKey(orm['myapp.MyModel'], null = False), explicit_name=True)


When this is run you get the following code executed:
DEBUG:django.db.backends:(0.000) SET FOREIGN_KEY_CHECKS=0;; args=()
DEBUG:south:south execute "
            SELECT kc.constraint_name, kc.column_name
            FROM information_schema.key_column_usage AS kc
            JOIN information_schema.table_constraints AS c ON
                kc.table_schema = c.table_schema AND
                kc.table_name = c.table_name AND
                kc.constraint_name = c.constraint_name
            WHERE
                kc.table_schema = %s AND
                kc.table_catalog IS NULL AND
                kc.table_name = %s AND
                c.constraint_type = %s
        " with params "['django', 'myapp_mymodel', 'FOREIGN KEY']"
DEBUG:django.db.backends:(0.111)
            SELECT kc.constraint_name, kc.column_name
            FROM information_schema.key_column_usage AS kc
            JOIN information_schema.table_constraints AS c ON
                kc.table_schema = c.table_schema AND
                kc.table_name = c.table_name AND
                kc.constraint_name = c.constraint_name
            WHERE
                kc.table_schema = django AND
                kc.table_catalog IS NULL AND
                kc.table_name = myapp_mymodel AND
                c.constraint_type = FOREIGN KEY
        ; args=['django', 'myapp_mymodel', 'FOREIGN KEY']
DEBUG:south:south execute "ALTER TABLE `myapp_mymodel` ;" with params "[]"
DEBUG:django.db.backends:(0.000) ALTER TABLE `myapp_mymodel` ;; args=[]
DEBUG:south:south execute "ALTER TABLE `myapp_mymodel` MODIFY `my_field_id` integer NOT NULL;;" with params "[]"
DEBUG:django.db.backends:(0.075) ALTER TABLE `myapp_mymodel` MODIFY `my_field_id` integer NOT NULL;;; args=[]
DEBUG:south:south execute "ALTER TABLE `myapp_mymodel` ALTER COLUMN `my_field_id` DROP DEFAULT;" with params "[]"
DEBUG:django.db.backends:(0.067) ALTER TABLE `myapp_mymodel` ALTER COLUMN `my_field_id` DROP DEFAULT;; args=[]
DEBUG:south:south execute "ALTER TABLE `myapp_mymodel` ADD CONSTRAINT `my_field_id_refs_id_15e652d5` FOREIGN KEY (`my_field_id`) REFERENCES `myapp_my_field` (`id`);" with params "[]"
DEBUG:django.db.backends:(0.283) ALTER TABLE `myapp_mymodel` ADD CONSTRAINT `my_field_id_refs_id_15e652d5` FOREIGN KEY (`my_field_id`) REFERENCES `myapp_my_field` (`id`);; args=[]
DEBUG:south:south execute "SET FOREIGN_KEY_CHECKS=1;" with params "[]"
DEBUG:django.db.backends:(0.000) SET FOREIGN_KEY_CHECKS=1;; args=[]
DEBUG:django.db.backends:(0.000) SELECT `south_migrationhistory`.`id`, `south_migrationhistory`.`app_name`, `south_migrationhistory`.`migration`, `south_migrationhistory`.`applied` FROM `south_migrationhistory` WHERE (`south_migrationhistory`.`app_name` = myapp  AND `south_migrationhistory`.`migration` = 0006_mymodel_my_field_cannot_be_null ); args=('myapp', '0006_mymodel_my_field_cannot_be_null')
DEBUG:django.db.backends:(0.000) INSERT INTO `south_migrationhistory` (`app_name`, `migration`, `applied`) VALUES (myapp, 0006_mymodel_my_field_cannot_be_null, 2011-09-16 14:37:50); args=('myapp', '0006_mymodel_my_field_cannot_be_null', u'2011-09-16 14:37:50')



# and the backward migraion is
    def backwards(self, orm):
        db.alter_column('myapp_mymodel', 'my_field_id', models.ForeignKey(orm['myapp.MyModel'], null = True), explicit_name=True)

Running the Thrift Tutorial with Python

See the previous post about installing thrift.


We will assume that you are currently in the thrift directory.
The tutorial.thrift file is written very well with lots of useful comments, but here are the few lines of code in this file which we really need:

namespace cpp tutorial
namespace java tutorial
namespace php tutorial
namespace perl tutorial

enum Operation {
  ADD = 1,
  SUBTRACT = 2,
  MULTIPLY = 3,
  DIVIDE = 4
}
struct Work {
  1: i32 num1 = 0,
  2: i32 num2,
  3: Operation op,
  4: optional string comment,
}


exception InvalidOperation {
  1: i32 what,
  2: string why
}

service Calculator extends shared.SharedService {

   void ping(),
   i32 add(1:i32 num1, 2:i32 num2),
   i32 calculate(1:i32 logid, 2:Work w) throws (1:InvalidOperation ouch),
   oneway void zip()

}

Copy the files tutorial.thrift and shared.thrift into a new directory 

Generate the python files

thrift -r --gen py:new_style tutorial.thrift

where

-r makes thrift generate included files

--gen py:new_style makes thrift use the py generator with the optional argument of new_style to generate new style classes

Looking at gen-py/tutorial/Calculator.py you can see that the Calculator Service has been turned into an interface (class Iface).

The Iface class is inherited by class Client and class Processor

The Client implements the methods defined by the interface.

e.g. the add method does 2 things, calls send_add and returns recv_add.

The send_add method will write the name of the method 'add' and its arguments to the Thrift transport.

The recv_add method will receive the result of the add method.

In addition to the Client, there is a class called Processor who also implements Iface.

In the case of the add method, the Processor will read the arguments, call the handler who does the add operation and writes the result.

The handler is the important part as this is that class which you will write which implements the methods.

Installing Thrift

I'm doing this on ubuntu 10.10

First the following dependencies need installed:

sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++

While that is going on you can download the latest thrift:
svn co http://svn.apache.org/repos/asf/thrift/trunk thrift

This creates the directory called thrift which you should cd into.

Next run
./bootstrap.sh
./configure
make
sudo make install

In the next post, I will look at running the tutorial which is found in the tutorial directory.

Python: Different types of member methods

In Python classes we can have the following methods:

1. Instance Methods
2. Static Methods
3. Class Methods

- Instance Methods
=====================
These are methods which are invoked upon an instance (object) of the class.
For example:

class MyClass:
    def myInstanceMethod(*args):
 print "inside myInstanceMethod, the arguments are:"
        for v in args:
            print v

Using this code we get:

>>>m = MyClass()
>>>m.myInstanceMethod()
    inside myInstanceMethod, the arguments are:
    <__main__.MyClass instance at 0xb7395a0c>

Which shows us that the object instance (commonly referred to as 'self') was passed into the instance method

If we try to call an instance method without an instance, we get:

>>>MyClass.myInstanceMethod ()
    TypeError: unbound method myInstanceMethod() must be called with MyClass instance as first argument (got nothing instead)

- Static Methods
=====================
In C++, static methods are usually used to fetch private static data from a class. The most common example is a static member which is incremented each time the constructor is called. This static member will then record the total number of instantations of the class.
In Python, the static method looks like this:

class MyClass:
    @staticmethod
    def myStaticMethod(*args):
 print "inside myStaticMethod, the number of arguments is ", len(args)
        for v in args:
            print v

Using this code we get:

>>>m = MyClass()
>>>m.myStaticMethod ()
     inside myStaticMethod, the number of arguments is  0
>>>MyClass.myStaticMethod ()
     inside myStaticMethod, the number of arguments is  0

The main difference in Python between the instance and static methods is that the static methods can be called by both instances and the class itself

- Class Methods
=====================

class MyClass:
    @classmethod
    def myClassMethod(*args):
        print "inside myClassMethod, the number of arguments is ", len(args)
        for v in args:
            print v

Using this code we get:

>>>m = MyClass ()
>>>m.myClassMethod ()
inside myClassMethod, the number of arguments is  1
__main__.MyClass

>>>MyClass.myClassMethod ()
inside myClassMethod, the number of arguments is  1
__main__.MyClass

Here we see (like the instance methods) we receive 1 argument, unlike the instance methods, the argument is the type of class, in the instance methods we received an object which was bound to the class.

Python: How useful are lambda functions ?

Lets say you have this function defined:

def square1(x) : return x*x

We can check what square is:

>>> type(square1)
 

And use it like this:

>>>square1 (8)
64

It can be rewritten as an anonymous function or lamda as:

square2 = lamda x: x*x

and again when we check the type we get

>>> type(square2)
 

And it works the same way as the previous function

>>>square2 (8)
64

Now say we define a list:

list = [1,2,3,4,5,6,7,8]

and then we want to get the square of each number. Either of these will work:

>>> map (square1, list)
[1, 4, 9, 16, 25, 36, 49, 64]

and same result with

>>> map (square2, list)
[1, 4, 9, 16, 25, 36, 49, 64]

So far the benefit of leaning lambda functions is small. We have only used them to write a small function on the fly and the function has to be small as lambdas do not support multiple statements.

Perhaps the main motivation for lambdas can be illustrated with the following:

strategy = {
    'square' : lambda x: x*x,
    'double' : lambda x: x*2,
    'half'   : lambda x: x/2,
}
list = [1,2,3,4,5,6,7,8]
while(True):
    try:
        x =  raw_input("What would you like to do with the list ? ")
        map(strategy[x], list)
    except KeyError, e:
        print "The value entered is not valid: " + str(e)

The dict called strategy contains 3 lambda functions and if we had not used lambdas the we would have had to add the following lines:

def square(x) : return x*x
def double(x) : return x*2
def half(x)   : return x/x

If our strategy dict contained a lot more functions, it would start getting tedious to define all of these and then since the definition of the function would be in another part of the code, it would not be as quick and easy to see what 'square' does for example.