Tuesday, September 13, 2016

How to Retrieve Emails from Gmail Periodically and Save as Files in Linux

In this tutorial, I will go over my own attempt to automate a process for fetching gmail and saving as files. This post is heavily based on the tutorial here with some modifications.

IMPORTANT:
In order for this to work, you will need to enable POP in Gmail setting.

You will also need to go to Google Account Security page and disable 2-step verification and turn on the option that allows less secure app.


First, install some necessary software:
$ sudo apt-get install -y fetchmail procmail openssl ca-certificates

Next, create ~/.fetchmailrc file and edit it with the following content
poll pop.gmail.com
with proto POP3
user "GMAIL_USERNAME@gmail.com"
there with password "YOUR_PASSWORD"
is "LINUX_USERNAME" here
mda "/usr/bin/procmail -f %F -d %T"
options
no keep
ssl
sslcertck
sslcertpath /etc/ssl/certs

Note that LINUX_USERNAME should be exactly what you would get from the following command:
$ echo $USER

Since your password is saved as a plain text, you will need to make sure to set read/write flags only for the owner:
$ chmod 600 ~/.fetchmailrc

Now, we need to configure procmail. Create ~/.procmailrc file and edit it with the following content:
# Environment variable assignments
PATH=/bin:/usr/bin:/usr/local/bin
VERBOSE=off                   # Turn on for verbose log
MAILDIR=$HOME/Mail            # Where Procmail recipes deliver
LOGFILE=$HOME/.procmaillog    # Keep a log for troubleshooting.
# Recipes
:0:
* ^(From).*(SOMEBODY)
$MAILDIR/SOMEBODY

This configuration will have procmail filter out any mails from SOMEBODY and save the message by appending to the file ~/Mail/SOMEBODY. Make sure to create the folder where it will be saved to:
$ mkdir ~/Mail

What about any other messages? They should be saved to path pointed by $MAIL, which we will need to specify. Edit ~/.bashrc to append the following line:
MAIL=/var/spool/mail/$USER && export MAIL

Let the change take effect immediately
$ source ~/.bashrc

Well, looks good. Let's test out. Run
$ fetchmail
2 messages for xxx@gmail.com at pop.gmail.com (3998 octets).
reading message xxx@gmail.com@gmail-pop.l.google.com:1 of 2 (3383 octets) flushed
reading message xxx@gmail.com@gmail-pop.l.google.com:2 of 2 (615 octets) flushed
You have new mail in /var/spool/mail/linuxnme

If your message is sent from SOMEBODY, it will be saved into ~/Mail/SOMEBODY file. If not, it will be saved to /var/spool/mail/$USER file. Open up these files and see if you are receiving the messages.

Finally, to schedule fetchmail every 5 minute, do the following:
$ crontab -e

Append the following line:
*/5 * * * * /usr/bin/fetchmail &> /dev/null

By the way, if you want fetching to run system-wise, edit /etc/crontab file instead.

No comments:

Post a Comment