domingo, 11 de septiembre de 2016

Install automatically email notification after finished a script in Linux

In this example we will work with gmail account. We must go click the gear icon in gmail page, go to Settings, go to the tab Forwarding POP/IMAP, and click the Configuration instructions link in IMAP Access row.
Then click Enable IMAP.  Note the mail server information and use that information for further settings as shown in the next section.


Before proceeding further visit https://www.google.com/settings/security/lesssecureapps and check Turn on radio button.


Install mutt
Ubuntu
sudo aptitude install mutt
sudo apt-get install mutt
Configure Mutt
We need create:
mkdir -p ~/.mutt/cache/headers
mkdir ~/.mutt/cache/bodies
touch ~/.mutt/certificates
A mutt configuration file  called muttrc
touch ~/.mutt/muttrc
We modify mutter file:
vim ~/.mutt/muttrc
Add following configurations
set ssl_starttls=yes
set ssl_force_tls=yes
set from = "user@gmail.com"
set realname = "Your name"
set imap_user = "user@gmail.com"
set imap_pass = "password"
set folder = imaps://imap.gmail.com/
set spoolfile = "+INBOX"
set postponed ="+[Gmail]/Drafts"
set header_cache =~/.mutt/cache/headers
set message_cachedir =~/.mutt/cache/bodies
set certificate_file =~/.mutt/certificates
set smtp_url = "smtp://user@smtp.gmail.com:587/"
set smtp_pass = "password"
set move = no 
set imap_keepalive = 900
Send a email
Now  is ready to start sending emails.
mutt -s "Subject" user@icloud.com < message.txt
And if you want to send an attachment
mutt -s "Subject" user@icloud.com < message.txt -a /tmp/photo.jpg
-s => is the subject
-a => is the attachment 
message.txt  => is the message itself
This is another way to write the test, having the body in the same line.
echo "body" | mutt -s "Subject" user@icloud.com -a /tmp/photo.png
And you can add those command in any file or script.

viernes, 3 de junio de 2016

Install SGE (Sun Grid Engine) on Linux

Install SGE on Ubuntu 16.04

0. Install some package:

deimos@deimos:~$ sudo apt-get install xfs*
deimos@deimos:~$ sudo apt-get install libguestfs-xfs
deimos@deimos:~$ sudo apt-get install xfonts-75dpi xfonts-100dpi
deimos@deimos:~$ sudo apt-get install t1-xfree86-nonfree ttf-xfree86-nonfree ttf-xfree86-non
deimos@deimos:~$ xset +fp /usr/share/fonts/X11/75dpi
deimos@deimos:~$ xset fp rehash


1. Install SGE on master node:

deimos@deimos:~$ sudo apt-get install gridengine-client gridengine-common gridengine-master gridengine-qmon gridengine-exec

  • Remove gridengine-exec from the list if master node is not supposed to run jobs
  • During the installation, we need to set the cluster CELL name (such as 'default')

2. Install SGE on other nodes:

deimos@deimos:~$ sudo apt-get install gridengine-client gridengine-exec


  • The CELL name is set the same as that of the master node

However, you can install all in the same machine (master and client), but you must set:
deimos@deimos:~$ sudo apt-get install gridengine*








3. Set SGE_ROOT and SGE_CELL environment variables:

We need edit three files /etc/profile, /etc/bash.bashrc and ~/.bashrc, and add the following two lines:
  • export SGE_CELL=default
Set this line if it is necessary:
  • export SGE_ROOT='path of SGE'

4. Set domain:

Edit /etc/hosts with your domain


5. Configure SGE

We need launch qmon as superuser:
deimos@deimos:~$ sudo qmon

5.1 Configure host
  • Host Configuration => Administration Host => add master node and other administrative nodes
  • Host Configuration => Submit Host => add master node and other submitted nodes
  • Host Configuration => Execution Host => add slave nodes
  • Click Done
5.2 Configure user
In this configuration, you can add or delete users that are allowed to access SGE.
  • User Configuration => Userset => Highlight userset 'arusers' and click on 'Modify' => input user name in 'User/Group' field
  • Click Done
5.3 Configure queue
Queue Control define ways to connect hosts and users.
  • Queue Control => Hosts => Confirm the execution hosts show up there.
  • Queue Control" => Cluster Queues => Click on "Add" => Name the queue, add execution nodes to Hostlist
  • Use access => allow access to user group arusers;
  • General Configuration => Field 'Slots' => Raise the number to total CPU cores on slave nodes (ok to use a bigger number than actual CPU cores).
  • Queue Control" => Queue Instances => This is the place to manually assign hosts to queues, and control the state (active, suspend ...) of hosts.
5.4 Configure parallel environnent
  • Queue Control => Cluster Queues => select a queue that it will run parallel jobs => click on 'Modify' => 'Parallel Environment' => click on 'PE' below the right and left arrows => click on 'add' => name the PE, slot = 999, start_proc_args = $SGE_ROOT/mpi/startmpi.sh $pe_hostfile, stop_proc_args = $SGE_ROOT/mpi/stopmpi.sh, allocation_rule= $fill_up, check 'Control slaves' to make this variable checked

6. Check SGE hosts

  • The system info from all nodes
deimos@deimos:~$ qhost
  • The hostnames of nodes
deimos@deimos:~$ qconf -sel
  • List the queues
deimos@deimos:~$ qconfi -sql
  • Check master daemon
deimos@deimos:~$ ps aux | grep sge_master
  • Check execute daemon
deimos@deimos:~$ ps aux | grep sge_execd
  • If sge_master or sge_execd is not running, you can start the service
deimos@deimos:~$ sudo service gridengine-master start
deimos@deimos:~$ sudo service gridengine-exec start


Reference
N1 Grid Engine 6 Administration Guide
https://web.njit.edu/topics/HPC/basement/sge/817-5677.pdf