Friday, January 29, 2016

How to Compile Fuego 1.1 on Ubuntu 14.04 LTS x64

My dad loves the game of GO, so I decided to let him play it vs AI. I first installed GNU GO on his computer and let him play against it; after a couple of times he played, he complained that GNU GO cannot even count accurately. So I figured I should provide him a better GO AI. That is how I decided to go for Fuego.

To install Fuego, download its source files
$ wget http://downloads.sourceforge.net/project/fuego/fuego/1.1/fuego-1.1.tar.gz

Unzip the source files
$ tar xfc fuego-1.1.tar.gz && cd fuego-1.1

Before configure, we need to install Boost library and build-essential
$ sudo apt-get install libboost-all-dev build-essential -y

Now, let's configure
$ ./configure

You will probably see some error saying 
configure: error: Could not find a version of the library!

The reason is that it cannot find the boost library, so we try to configure with the option
$ ./configure --with-boost-libdir=/usr/lib/x86_64-linux-gnu

The configure should have completed successfully. Let's compile.
$ make

You will probably encounter compilation error saying
GtpEngine.cpp:352:33: error: expected unqualified-id before numeric constant
         xtime_get(&time, boost::TIME_UTC);


We can fix this by replacing all instances of TIME_UTC with TIME_UTC_. Let's fix one by one first. Open the file
$ vim gtpengine/GtpEngine.cpp

Go to line 352 by entering :352 in vim and replace TIME_UTC with TIME_UTC_. Save and exit.

Let's compile again
$ make

This time you will see some error saying
SgPointSetUtil.h:67:25: error: reference 'm_pointSet' cannot be declared 'mutable' [-fpermissive]
     mutable SgPointSet& m_pointSet; // allow temp objects to modify


Let's solve this by removing 'mutable' in the file.
$ vim smartgame/SgPointSetUtil.h

After removing the keyword in line 67, save, and exit and give it another try.
$ make

You will encounter another error
GoGtpEngine.cpp:381:46: error: 'class boost::filesystem::path' has no member named 'native_file_string'
                            << m_sentinelFile.native_file_string() << "'";


This one is easy. If you search for native_file_string() method for Boost in Google, you will see that this method has been deprecated and replaced by string(). So, let's do that.
$ vim go/GoGtpEngine.cpp

In fact, you will have probably encountered another TIME_UTC error, which you can fix now, right? It is in the same file, so it should be easy.

Let's continue.
$ make

You will encounter another error saying
FuegoMain.cpp:65:55: error: invalid conversion from 'bool (*)(const string&) {aka bool (*)(const std::basic_string<char>&)}' to 'boost::enable_if_c<true, void>::type* {aka void*}' [-fpermissive]
     return path(programPath, boost::filesystem::native).branch_path();


This is again due to boost deprecation. One of the easiest ways to fix it for the moment is to replace line 65 with
return boost::filesystem::current_path();

This is not a remedy because this will only work if the executable is called from the same directory. However, let's just focus on compiling at the moment.

By the way, we will also need to include the boost::filesystem header file for the above quick fix to work. Insert the following at, say, line 11 of the FuegoMain.cpp file.
#include <boost/filesystem.hpp>

Let's continue.
$ make

We will encounter another complaint for native_file_string error, so let's fix it and compile again.
$ make

Lastly, we see another error saying
/usr/bin/ld: ../smartgame/libfuego_smartgame.a(libfuego_smartgame_a-SgUctSearch.o): undefined reference to symbol 'pthread_mutexattr_settype@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line


As the error suggests, it is complaining because pthread library is missing. The easiest way to fix this is to run the g++ command with -lpthread option
$ cd fuegomain
$ g++ -g -O2 -DNDEBUG -Wall -Wextra -L/usr/lib/x86_64-linux-gnu  -o fuego fuego-FuegoMain.o fuego-FuegoMainEngine.o fuego-FuegoMainUtil.o ../gouct/libfuego_gouct.a ../go/libfuego_go.a ../smartgame/libfuego_smartgame.a ../gtpengine/libfuego_gtpengine.a -lboost_program_options -lboost_filesystem -lboost_system -lboost_thread -lpthread
$ cd ..

Let's continue!
$ make

You will most likely successfully compiled it finally! By the way, in order to save some compilation time, you may want to add in -j 2 option for make command, telling it to use 2 cores.


*** EDIT ***
After I created the post, I realized that Fuego website actually has a nice tutorial on building from the source files for Mac OS X and Linux. Furthermore, the latest Fuego source files can only be obtained via svn; these source files fix all of the problems mentioned above, except for the last one. In any case, I have created a script file that will build Fuego from the latest source files and install gogui and openjdk as well. Copy the script below and paste into a new file, say build_fuego_on_Ubuntu_14.04_x64.sh

#!/bin/bash
# this script attempts to compile and install fuego on Ubuntu 14.04 LTS x64


apt-get install -y subversion build-essential autoconf openjdk-7-jdk libboost-all-dev


sudo -u $SUDO_USER bash -c "svn checkout svn://svn.code.sf.net/p/fuego/code/trunk fuego"

cd fuego

#sudo -u $SUDO_USER bash -c "export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/"

sudo -u $SUDO_USER bash -c "autoreconf --force --install"
sudo -u $SUDO_USER bash -c "./configure --with-boost=/usr/lib/x86_64-linux-gnu/ --with-boost-libdir=/usr/lib/x86_64-linux-gnu/"
sudo -u $SUDO_USER bash -c "make -j 2"

cd fuegomain
sudo -u $SUDO_USER bash -c "g++  -g -O2 -DNDEBUG -Wall -Wextra -L/usr/lib/x86_64-linux-gnu/  -o fuego fuego-FuegoMain.o fuego-FuegoMainEngine.o fuego-FuegoMainUtil.o ../gouct/libfuego_gouct.a ../go/libfuego_go.a ../features/libfuego_features.a ../smartgame/libfuego_smartgame.a ../gtpengine/libfuego_gtpengine.a -lboost_program_options -lboost_filesystem -lboost_system -lboost_thread -lpthread"

cd ..
sudo -u $SUDO_USER bash -c "make -j 2"
cd ..
sudo -u $SUDO_USER bash -c "wget http://downloads.sourceforge.net/project/gogui/gogui/1.4.9/gogui-1.4.9.zip"
sudo -u $SUDO_USER bash -c "unzip gogui-1.4.9.zip"
cd gogui-1.4.9/lib

sudo -u $SUDO_USER bash -c "java -jar gogui.jar"


To run this file, simply set the execution flag and execute with sudo
$ chmod u+x build_fuego_on_Ubuntu_14.04_x64.sh
$ sudo ./build_fuego_on_Ubuntu_14.04_x64.sh
 
That's it!

Thursday, January 28, 2016

Function and Alias in Bash

People are lazy in general. Sometimes, it is simply annoying to type long commands in bash with lots of options. Let us see how we can use bash functions and alias to relieve stress of typing too much.

alias helps us to shorten a long command. On Linux, I use $ ls --color=auto -CF commands quite a lot. However, it becomes annoying typing the command repeatedly. What one can do is create an alias to shorten this as
$ alias ls='ls --color=auto -CF'

Now, when you type the command $ ls, it will essentially execute $ ls --color=auto -CF command. In many cases, you may want to write the alias command in ~/.bashrc for Linux and ~/.bash_profile for Mac OS X to be executed every time you open up the terminal. In fact, on my system (Ubuntu 14.04 LTS), the default ~./bashrc file already contains some alias commands as below:
alias ls='ls --color=auto'
...
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'


Hence, the command $ l will actually execute $ ls -CF, which in turn will execute $ ls --color=auto -CF, i.e., $ l is a nested alias for $ ls --color=auto -CF.

By the way, the utility ls is a bit different on a Mac OS X system, since it is a BSD version of ls whereas on Linux it is GNU version of ls. The two are very similar but some of the options are different. In any case, the use of alias is identical, so I will not go into the difference of BSD ls and GNU ls here.

Next up is bash function. Although alias can shorten the commands quite significantly, it does not support arguments. That is when the bash function comes very handy. Consider the example below.

 To search for files and directories, I often use the command $ find / -name name_to_search 2> /dev/null. Again, typing this command repeatedly makes my fingers very tired. I'd like to create an alias for this, but I can't because I need to supply the argument name_to_search and alias does not support it. So, I will need to make a function here.
$ f() { find / -name $1 2> /dev/null; }

Now if I run $ f bash, it will execute the command $ find / -name bash 2> /dev/null. Note that $1 is the first argument, $2 is the second argument, and so forth. For example, the function below will take two arguments.
$ ff() { find / -name $1 -type $2 2> /dev/null; }

Now, if I execute $ ff bash f, this will execute $ find / -name bash -type f 2> /dev/null. Again, you can save these functions in the ~/.bashrc for Linux for ~/.bash_profile for Mac OS X to load the function automatically at the startup of the terminal.

Tuesday, January 26, 2016

How to Build and Install Numpy, Scipy, and Matplotlib Packages on Ubuntu

This is a continuation from the last post on How to build and install the latest version of Python on Ubuntu. If you are simply looking to install Python packages, such as numpy, scipy, and matplotlib, on the default system Python (which is 2.7.6 for Ubuntu 14.04 LTS), the answer is quite simple:
$ sudo apt-get install python-numpy python-scipy python-matplotlib -y

The command will install the packages in the /usr/lib/python2.7/dist-packages directory. With this method, however, you can import those packages from the system default Python (version 2.7.6) but not from newly installed Python (version 2.7.11). Thus, in this post we will go over the steps to build and install the Python packages for the newly installed version.

First, we need to install setuptools package. We will need to download the script and run it.
$ wget https://bootstrap.pypa.io/ez_setup.py && sudo python ez_setup.py

Next, we need to install pip package. We will download the archive, extract, and run the script
$ wget https://pypi.python.org/packages/source/p/pip/pip-8.0.2.tar.gz#md5=3a73c4188f8dbad6a1e6f6d44d117eeb
$ tar -xf pip-8.0.2.tar.gz
$ cd pip-8.0.2
$ sudo python setup.py
$ cd ..

Finally, use pip to install the packages
$ sudo pip install numpy scipy matplotlib

It will take quite some time to build, so please be patient. The packages will be installed in /usr/local/lib/python2.7/site-packages directory.

How to Build and Install the Latest Version Python on Ubuntu

Ubuntu 14.04 LTS comes with Python 2.7.6 and 3.4.0 pre-installed. However, sometimes you may wish to install the most recent version, say 2.7.11 as of now along with the default system python. Unfortunately, Ubuntu's repository does not have the most recent version, so you will need to manually build Python from source files. This post will show you how to do so. Note that in this example, I will be installing 2.7.11, but you may want to use a different version as desired.

Before we do anything, let us examine where the system default Python is located and what version it is

$ which python
/usr/bin/python

The which  command shows the file which will be executed upon running the command $ python in shell. This is the default Python file pre-installed on the system. To check its version, run

$ python --version
Python 2.7.6

So, the default version that comes with Ubuntu 14.04 LTS is indeed 2.7.6 and is located in the /usr/bin directory.

OK, let us now start the installation process for the latest version 2.7.11. First, download the Python source archive file
$ wget https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz

Extract the files
$ tar -xf Python-2.7.11.tgz

Enter the directory
$ cd Python-2.7.11

Before we configure, we must install quite a few packages necessary to build Python successfully. Although not all of the packages below are required, I recommend getting them because they will be needed for installing packages later on, such as numpy, scipy, and matplotlib, which I will cover in the next post.
$ sudo apt-get install -y build-essential gcc-multilib g++-multilib libffi-dev libffi6 libffi6-dbg python-crypto python-mox3 python-pil python-ply libssl-dev zlib1g-dev libbz2-dev libexpat1-dev libbluetooth-dev libgdbm-dev dpkg-dev quilt autotools-dev libreadline-dev libtinfo-dev libncursesw5-dev tk-dev blt-dev libssl-dev zlib1g-dev libbz2-dev libexpat1-dev libbluetooth-dev libsqlite3-dev libgpm2 mime-support netbase net-tools bzip2 libblas-dev liblapack-dev gfortran

Now, configure
$ ./configure

Then build
$ sudo make install

It will take some time to build and install. When complete, we verify whether Python is now updated to 2.7.11 version
$ python --version
Python 2.7.11

Great. So, the question is, did we just overwrite the system's default python? The answer is no; we actually installed the new version on the system side by side from the old one, so the old version is still intact. How do we figure? Simply run

$ which python
/usr/loca/bin/python

The output shows that the command $ python will now execute /usr/local/bin/python, which is different from the old one /usr/bin/python. To verify this, see if we can still run the old one

$ /usr/bin/python --version
Python 2.7.6

So, the old one is still there; if we need to run the old one, we just need to execute it by the full path. This is because the default $PATH environment looks something like this

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

The directory /usr/local/bin, where version 2.7.11 is installed, precedes the directory /usr/bin, where version 2.7.6 is installed. Thus, when we run $ python command, it will execute the Python file in /usr/local/bin, thus running the 2.7.11 version.

Thursday, January 21, 2016

Setup Latex Environment for Ubuntu

I personally use TexWorks for creating and editing TEX documents. I also often use IEEEtran.cls package along with many other related scientific packages, such as subfig, etc. This post will go over how to install these packages successfully in the Ubuntu environment.

First off, add the TexWorks repository
$ sudo add-apt-repository ppa:texworks/stable -y

Install TexWorks and lots of latex-related packages that are frequently used
$ sudo apt-get install -y texworks texlive-fonts-recommended texlive texlive-latex-extra texlive-math-extra texlive-pstricks texlive-science latex-beamer

Download IEEEtran.zip file from http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/ by
$ wget http://mirrors.ctan.org/macros/latex/contrib/IEEEtran.zip

Unzip the file to the latex package location
$ sudo unzip IEEEtran.zip -d /usr/share/texmf/tex/latex/IEEE

Let tex know that the package has been added manually
$ sudo texhash


Now, you should be able to run TexWorks and compile TEX files!

Tuesday, January 19, 2016

Installing Muliple Desktop Environments in Ubuntu

I love to try new things out. In that respect, I love trying different various desktop environments on Ubuntu.

Ubuntu comes default with Unity desktop, but you can install other desktop environments as well. To install Ubuntu desktop environment, simply run
$ sudo apt-get install -y --no-install-recommends desktop_environment_to_install

where desktop_environment_to_install can be any of the following corresponding to the desktop environment in parentheses:
ubuntu-gnome-desktop (GNOME)
kubuntu-desktop (KDE)
ubuntu-desktop (Unity)
ubuntu-mate-desktop (MATE)
xubuntu-desktop (Xfce)
lubuntu-desktop (LXDE)

When you have multiple desktop environments installed, you may be able to choose one at the log-in screen.

Sunday, January 17, 2016

How to Find Configuration Files in Linux (GNOME example)

When you change the computer's setting or configuration, the change must be saved as a file somewhere. This means that if you search through your files system for files that have been created or modified after you have applied the change, you will be able to locate exactly where your setting or configuration is stored on the system.

For example, assume you want to change the font size of the default terminal profile. Well, when you apply the change, this must have changed some file on the system that saves this setting, but what is this file and where is this file even located?

In this post, we will look into ways to locate exactly which files have been modified and how using find command in terminal. The basic idea is pretty simple: you simply look for all files on the system that have been modified since you have applied the change. Here is step by step instructions:

First, create a timestamp file:
$ touch timestamp

When you type
$ ls -l timestamp
you will see the file's modified time, which should be just a second ago.

Next up, you change any setting or configuration. It is recommended that you do one change at a time unless you know what you are doing. I am going to change the font size of the default terminal profile.

Then, look for files that have been modified after the timestamp file by
$ find / -type f -newer timestamp -not -path "/proc/*" -not -path "/run/*" -not -path "/dev/*" 2> /dev/null

The command is very long and intimidating at first glance. Let us break down the command and look at each option one by one:

find / command will search all files and folders in the root directory. Note that if you change the configuration that pertains to a specific user, the file is saved in the user directory, i.e., ~ or /home/username. In this case, you may want to type in find ~ instead.

-type f will search for files only, not directories; this is because the configuration or setting will be saved as a file.

-newer timestamp will only find those that have been modified after that of timestamp file, which we have created in the first step.

-not -path "/proc/*" option will exclude those in the /proc directory from the result; we are doing this because /proc directory contains process information, which change constantly, so we want to opt this directory out. Note that this option only pertains to find /, since find ~ will not search for /proc directory anyway.

-not -path "/run/*" option will exclude those in the /run directory from the result; /run directory contains run-time variable data. This directory may change regardless of the configuration change, so we will opt this directory out as well.

-not -path "/dev/*" option will exclude those in the /dev directory from the result. Note that it is extremely unlikely that the configuration change is saved in either /proc, /run, or /dev directories.

2> /dev/null will ignore any error messages, such as those that complain for permission denied. You may get rid of this option by using sudo. More on this later.

When you prompt the aforementioned command, you will get output something similar to
/home/unixnme/.config/dconf/user
where it is obvious that the first file ~/.config/dconf/user is the configuration setting file that is responsible for terminal profile. Unfortunately, this file is not a simple plain-text configuration file, so it is a bit difficult to open up and read the file. However, you can try the following:

Open up the file with vim
$ vim ~/.conf/dconf/user

Within vim, search the text 'Monospace' by entering
/Monospace
You will most likely see a string Monospace, the default terminal font, within this file. As a matter of fact, this is actually the right file; you will see the font size in number right next to the string. I will cover this in more details in the later post. Quit vim by pressing ESC key and entering
:q!

Although in most cases, the configuration file should be readable by the user, it is not the case all the time. In this case, one will need to invoke sudo
$ sudo find / -type f -newer timestamp -not -path "/proc/*" -not -path "/run/*"

I will cover more details of this ~/.conf/dconf/user file in the later post.

Wednesday, January 13, 2016

How to Permanently Change Locale to Default in Ubuntu

To find the current locale of the system, simply type in the terminal
$ locale

This will display the current system's locale variables. In order to change the locales to the default values and set the language as US English, you will need to edit the /etc/default/locale file by
$ sudo vim /etc/default/locale

Edit the file with the following text:
LANG="en_US.UTF-8"
LC_ALL="C"


You can reboot to see it works!
For further info, please refer to https://help.ubuntu.com/community/Locale

Monday, January 11, 2016

VirtualBox Commands in Shell (Mac OS X & Linux)

If you would like to run virtual machines from terminal, this is what you will need to type in, assuming that your virtual machine has been already set up.

** For Linux, use vboxmanage **
** For Mac OS X, use vBoxManage **

To find out what virtual machines you have, type
$ vBoxManage list vms

To start the virtual machine, type
$ vBoxManage startvm "name_of_virtual_machine"

If you are actually running a server that does not have GUI interface, then you might want to add the option --type headless at the end of the aforementioned command.

To view the status of the virtual machine, type
$ vBoxManage showvminfo "name_of_virtual_machine"

You may want to add | less at the end.

Also, below command will let you pause, resume, reset, power off, or save state the virtual machine:
$ vBoxManage controlvm "name_of_virtual_machine" control_option
where control_option will be one of the following: pause, resume, reset, poweroff, or savestate.

For more info, please refer to https://www.virtualbox.org/manual/ch08.html

Saturday, January 9, 2016

File & Directory Searches in Linux Command Line using find

Let's learn how to search files & directories in Linux command line. The syntax is very simple:
$ find /location/to/find -name 'name_to_search'

Example 1: find all files and directories with the name etc
$ find / -name 'etc'

Example 2: find files and directories inside /home directory with the name .bashrc
$ find /home -name '.bashrc'


OK. Let's add some complication. Assume that you want to find files and directories that contain some given name. You can do this by:
$ find /location/to/find -name '*name_to_search*'

Example 3: find all files and directories that contain the name etc
$ find / -name '*etc*'

Example 4: find files and directories inside /home directory that contain the name bash
$ find / -name '*bash*'

Here, the * symbol acts as a wildcard that can be any number of any character including none. Since we put the search keyword in between the wildcard *, we can find all instances of files and directories that contain the given keyword regardless of its position.

Another wildcard is ?, which can be any single character. Let's do some examples:

Example 5: find all files and directories whose names are four letters with the last three being ate
$ find / -name '?ate'

Example 6: find all files and directories whose names are four letters with the first three being dum
$ find / -name 'dum?'


We can mix the * and ? wildcards too.

Example 7: find all files and directories whose names end with three letters, of which the first and last are n and x, respectively
$ find / -name '*n?x'


We can add restricted wildcard by using []. Let's go straight to the example.

Example 8: find all files and directories whose names are three letters and start with b, c, or r and end with at
$ find / -name '[bcr]at'

Example 9: find all files and directories inside the current directory whose names are three letters and start with any character but b, c, nor r and end with at
$ find . -name '[!bcr]at'

Example 10: find all files and directories whose names ends with w through z
$ find / -name '*[w-z]'

Example 10: find all files and directories whose names end with anything but alphabet letters or numbers
$ find / -name '*[!a-z,A-Z,0-9]'

The - character indicates all letters or numbers in between, whereas the ! indicates the negation of what is in [].


You must precede with \ when searching for special characters.

Example 11: find all files and directories whose names contain one or more spaces
$ find / -name '*\ *'

Example 12: find all files and directories whose names contain "
$ find / -name '*\"*'

Example 13: find all files and directories whose names contain  \
$ find / -name '*\\*'


Example 14: find all files and directories whose names contain ?
$ find / -name '*\?*'


Example 15: find all files and directories whose names contain *
$ find / -name '*\**'


By the way, you may see a bunch of Permission denied outputs. In this case, run with root privilege
$ sudo find / -name 'name_to_search'
or you can suppress standard error messages by
$ find / -name 'name_to_search' 2> /dev/null

Lastly, one can use -iname instead of -name to search with the case-insensitive option.

Avoiding Duplicate UUID Error in VirtualBox

Sometimes you may encounter a strange error saying that it failed to open hard disk because UUID of the disk already exists in the media registry. In this case, what you need to do is to change the UUID of the virtual disk file using the following command in terminal

$ vboxmanage internalcommands sethduuid /location/of/virtual/disk.vdi

This command will simply change the UUID and you can successfully add the virtual disk to your virtual machine! (tested on VirtualBox 5.0.10)

By the way, if you are using Mac OS X, the command will be
$ VBoxManage internalcommands sethduuid /location/of/virtual/disk.vdi

Tuesday, January 5, 2016

Mount Partitions at Startup in Ubuntu 14.04 LTS

When your system uses more than one partition (excluding swap partition) it is desired to have them mounted at startup. For example, assume that you have two data partitions: one for the operating system with the root mount point / while the other for personal data with mount point of /home. You could configure these options when installing Linux, but what if you want to do it after installations are already complete? Well, here is how you can do this!

The file system table file /etc/fstab contains all the necessary configuration for mounting partitions at startup. Let's take a look at this file
$ vim /etc/fstab

The main ingredient of the file is organized as follows

1. <file sytem> can be either UUID or device path such as /dev/sda1. For robustness, UUID may be used. In order to find out UUID of your partitions, run
$ ls -l /dev/disk/by-uuid

2. <mount point> is where you would like to mount it as. The root partition will be / whereas user data partition could be /home.

3. <type> is the format of the partition, such as ext3, ext4, fat, ntfs, swap, etc

4. <options> specifies the mount options list separated by commas. For example, the system partition may be optioned with errors=remount-ro option, while default option is indicated as defaults. A swap partition can be specified with sw.

5. <dump> specifies whether to backup or not. Unless you know what you are doing, leave this as 0.

6. <pass> specifies disk check option. For root partition, leave this as 1. For other data partitions, leave this as 2. For partitions that do not require check, such as swap partition, leave this as 0.


With these in mid, you may want to first create a backup file,
$ sudo cp /etc/fstab /etc/fstab.bk

edit the file,
$ sudo vim /etc/fstab

and reboot for the system to take effect.
$ sudo reboot 
 
For more info, please refer to https://help.ubuntu.com/community/AutomaticallyMountPartitions