The Ultimate Emailing Agent with Mu4e and Emacs

Posted by: mistan

1. Mu4e in Arch Linux
2. Installation
3. Use getmail to retrieve your email from all your account
4. Install davmail for exchange server access
5. Process emails with Procmail
6. Filter and protection with Spamassassin and Clamvirus
7. MSMTP for sending emails, follow instruction here.
8. Use GnuPG to encrypt password in a file instead of clear password
9. Sending email notification with cron link
10. Creating org-mode links to emails for reminder and GTD purpose
11. Other Features
12. The keybindings and cheatsheet from mu4e official site.

1 Mu4e in Arch Linux

2 Installation

l mu and config mu4e for Emacs mu4e will be installed with mu, or grab it from github.

# get from git (alternatively, use a github tarball)

$ git clone git://github.com/djcb/mu.git

$ cd mu

$ autoreconf -i && ./configure && make

# On the BSDs: use gmake instead of make

$ sudo make install

Move the mu4e folder to your load-path (in my case, “~/.emacs.d/dotEmacs/”), OR, soft-link it.

1 2	ln -s /usr/local/share/emacs/site-lisp/mu4e ~/path/to/org-mode/

Then configure .emacs as the following.

(add-to-list 'load-path "~/path/to/org-mode/mu4e")

(require 'mu4e)

(require 'org-mu4e) ;;create org-mode links

mu4e-multi and mu4e-maildirs-extension are the simplest way to set up mu4e setting.

mu init --maildir=/mnt/2nd_BAY/mail

mu index # to fill the database with your messsages

3 Use getmail to retrieve your email from all your account

3.1 The email directoy system

I prefer maildir directory system here – a directory that stores email messages as files. The folders inside the mail directory are subdirectories such as .Drafts .Trash and .Sent. Each of these folders contain three additional subdirectories called tmp, new and cur. tmp – This subdirectory stores email messages that are in the process of being delivered. It may also store other kinds of temporary files. new – This subdirectory stores messages that have been delivered but have not yet been seen by any mail application, such as webmail or Outlook. cur – This subdirectory stores messages that have already been viewed by mail applications, like webmail or Outlook.

3.2 Install and configure getmail to retrieve emails

NOTE: use getmail from this link if you want passworddeval capability. or install from AUR (en) – getmail-passwordeval AUR for Arch Linux. getmail (http://pyropus.ca/software/getmail) with “passwordeval” receiver option so that the password can be received from an external program instead of being stored clear text in the config file. use separate rc file for each account (two gmails and one exchange server) set destination MDA-external and path to procmail rc file for each account to further process emails. create maildir directories within each account.

1 2	$ mkdir -p ~/mail/{cur,new,tmp}

Following is getmail rc file for one account.

[retriever]

type = SimpleIMAPSSLRetriever

server = imap.gmail.com

# mailboxes = ("[Gmail]/All Mail",)

## optional - leave this line out to just grab inbox

mailboxes = ("Inbox", "[Gmail]/Sent Mail")

username = yuanshengtan

passwordeval = gpg --no-tty -d ~/.password-store/.msmtp-gmail-XX.gpg

[destination]

type = MDA_external

path = /usr/bin/procmail

arguments = ("-f", "%(sender)", "-m", "~/.procmail/.procmailrc-tan")

[options]

# retrieve only new messages

# if set to true it will re-download ALL messages every time!

read_all = false

#very verbose and tell us the status of each message

verbose = 2

message_log = ~/.getmail/log

# do not alter messages

delivered_to = false

received = false

3.3 Automate the process –using a script file.

emailing setting http://pyropus.ca/software/getmail/getmailrc-examples

touch ~/usr/bin/get_mail

chmod +x ~/usr/bin/get_email

[taurus ~/usr/bin] $ nano get_mail

#!/bin/bash

/usr/bin/getmail -rcfile first_getmailrc -rcfile second_getmailrc

[taurus ~/usr/bin] $ crontab -e

*/20 * * * * /home/$USER/bin/get_email

4 Install davmail for exchange server access

use the correct exchange server url (davmail needed with owa: https://remote.xx.com/owa/ in my case). check with your IT if needed. use locallost and davmail port# in getmail.

If encounter following issue, modify getmail script:

Solving getmail protocol error when used with DavMail | My Experience 101

It worked but the e-mails was being duplicated everytime I pull it with getmail despite the fact that I have “read_all = false” in my configuration file to only pull new messages. After painful debugging I saw the below error, which occur when after pulling the last e-mail and seems it ignore saving the state so next time it pull everything from the begining getmailrc: protocol error (command: CLOSE => socket error: EOF)

Debugging further, I found in the file getmail, which is a python script, around line 342 that when this protocol error occurs the retriever abort and does not save the stat. So simply replacing it with retriever.quit() solved the problem! except (poplib.error_proto, imaplib.IMAP4.abort), o: errorexit = True #retriever.abort() retriever.quit() # My Hack to solve duplicate messages

UPDATE: for office 365:

Office 365 Office 365 EWS endpoint is now standard for all customers: https://outlook.office365.com/EWS/Exchange.asmx Also make sure you set mode to EWS in settings

-— from here

5 Process emails with Procmail

Each procmail rc corresponds to each account Note:

Comment out path to prevent mu -index to /root/.mu/xipian
good instruction here for Spamassassin, and isbg for IMAP folder filtering.

You may need to install docopt if you do not have it in your system.

PATH=/bin:/usr/bin

MAILDIR=/mnt/2nd_BAY/mail

PMDIR=$HOME/.procmail

DEFAULT=${MAILDIR}/Account1/

#trailing slash is needed above for emails to be placed in 'new' folder

LOGFILE=${MAILDIR}/procmail.log

SHELL=/bin/sh

6 Filter and protection with Spamassassin and Clamvirus

6.1 Spamassassin installation and setting

Download and install from apache official site, compile and install
or install from arch repos

1
2
3

sudo pacman -S spamassassin
sudo systemctl enable spamassassin

check with sample test mail to make sure it running.

Add following recipes in procmail rc file

# By using the f and w flags and no condition, spamassassin is going add the X-Spam headers to every single mail, and then process other recipes.

# No lockfile is used.

:0fw

| /usr/bin/vendor_perl/spamc

# Messages with a 5 stars or higher spam level are going to be deleted right away

# And since we never touch any inbox, no lockfile is needed.

* ^X-Spam-Level: \*\*\*\*\*

/dev/null

# If a mail with spam-status:yes was not deleted by previous line, it could be a false positive. So its going to be sent to an spam mailbox instead.

# Since we do not want the possibility of one procmail instance messing with another procmail instance, we use a lockfile

:0:

* ^X-Spam-Status: Yes

$MAILDIR/Spam

That should be it. You can play with the config files such as adding blacklist and whitelist etc follow this link.

6.2 ClamAV installation and setting -base one official Arch site.

Install from arch repos

1
2
3

sudo pacman -S clamav
freshclam # update
Confirm it is working

1
2

wget -O- http://www.eicar.org/download/eicar.com.txt | clamscan -

The output must include:

1
2

stdin: Eicar-Test-Signature FOUND

copy following into procmail rc settings

# We make sure its going to use the sh shell. Mostly needed for mail-only account that have /sbin/nologin as shell

SHELL=/bin/sh

# We will scan the mail with clam using the standard input, and saving the result on the AV_REPORT variable

AV_REPORT=`clamdscan --stdout --no-summary - |sed 's/^stream: //'`

# We check if the word FOUND was in the result and save "Yes" or "No" according to that

VIRUS=`echo $AV_REPORT|sed '/FOUND/ { s/.*/Yes/; q }; /FOUND/ !s/.*/No/'`

# formail is a filter that can alter a mail message, while keeping the correct format. We use it here to add/alter a header called

# X-Virus with either value Yes or No

:0fw

| formail -i "X-Virus: $VIRUS"

# And if we just added "X-Virus: Yes", we will also add another header with the scan result, and alter the subject, again, with the scan result.

# Since we are using the f flag, the mail is going to be delivered anyway.

:0fw

* ^X-Virus: Yes

| formail -i "X-Virus-Report: $AV_REPORT" -i "Subject: [Virus] $AV_REPORT"

6.3 Add more personal flavor to your recipe

I just modify following based on each of accounts, like it so far.

:0:

${MAILDIR}/commercials

7 MSMTP for sending emails, follow instruction here.

configure each account info with credentials.

# Set default values for all following accounts.

defaults

logfile ~/.msmtp.log

aliases /etc/aliases

# Gmail-Acount1

account Account1

host smtp.gmail.com

auth on

tls on

tls_trust_file /etc/ssl/certs/ca-certificates.crt

port 587

from your@gmail.com

user yourusername

passwordeval gpg --quiet --for-your-eyes-only --no-tty --decrypt ~/.password-store/.msmtp-gmail-t$

# MIS

account Account2 #same name as folder

host smtp.gmail.com

auth on

tls on

tls_trust_file /etc/ssl/certs/ca-certificates.crt

from tan@memoryimprintstudio.com

user tan@memoryimprintstudio.com

passwordeval gpg --quiet --for-your-eyes-only --no-tty --decrypt ~/.password-store/.msmtp-gmail-m$

# account3 -exchange server3

account Account3

auth login

tls off

host localhost

port 1025

protocol smtp

from name@adfas.com

user yourname

passwordeval gpg --quiet --for-your-eyes-only --no-tty --decrypt ~/.password-store/.msmtp-21cm.g$

# Set a default account

account default : Tan

Dot-emacs setting for signature for each account After all the fiddling, it turn out to be this is the easiest way for me. link here

(defun my-mu4e-choose-signature ()

"Insert one of a number of sigs"

(interactive)

(let ((message-signature

(mu4e-read-option "Signature:"

'(("formal" .

(concat

"Joe Bloggs\n"

"Department, Company Name, Country\n"

"W: http://www.example.com\n"))

("informal" .

"Joe\n")))))

(message-insert-signature)))

(add-hook 'mu4e-compose-mode-hook

(lambda () (local-set-key (kbd "C-c C-w") #'my-mu4e-choose-signature)))

8 Use GnuPG to encrypt password in a file instead of clear password

Install GnuPG and PGP-agent. It is pretty straight-forward with GnuPG installation, just follow Arch Official Site and informative tutorial, or Gentoo Info. Note: By default $GNUPGHOME isn’t set and your $HOME is used instead, thus you will find a ~/.gnupg directory right after the install.
gpg-agent is mostly used as daemon to request and cache the password whenever it needs the password. This is useful if GnuPG is used from an external program like a mail client. It can be activated by adding following line in gpg.conf:

1
2
3
4

~/.gnupg/gpg.conf
---------------------------
use-agent

Add the following entry to your .xinitrc or .bash_profile to autostart it.

envfile="$HOME/.gnupg/gpg-agent.env"

if [[ -e "$envfile" ]] && kill -0 $(grep GPG_AGENT_INFO "$envfile" | cut -d: -f 2) 2>/dev/null; then

eval "$(cat "$envfile")"

else

eval "$(gpg-agent --daemon --enable-ssh-support --write-env-file "$envfile")"

export GPG_AGENT_INFO # the env file does not contain the export statement

export SSH_AUTH_SOCK # enable gpg-agent for ssh

Logout and log back in, check if pgp-agent activated:

1
2

pgrep gpg-agent

The gpg-agent can be configured via ~/.gnupg/gpg-agent.conf file, such information as cache interval and how to ask user to input password (pinentry is used here since it installed along with GnupG in Arch Linux).

default-cache-ttl 3600

# PIN entry program

# pinentry-program /usr/bin/pinentry-curses

# pinentry-program /usr/bin/pinentry-qt4

# pinentry-program /usr/bin/pinentry-kwallet

pinentry-program /usr/bin/pinentry-gtk-2

Note: an empty gpg-agent.env was also created to remove the complain at initial login.

Reload the agent

1
2

echo RELOADAGENT | gpg-connect-agent

The shell should print OK.
Install getmail with “passwordeval” branch, download from here, unload the default getmail FIRST Navigate to download folder and unzip; Do the following as a regular user:

1
2

$ python2 setup.py build

Then (probably as root), do:

1
2

# python2 setup.py install
Generate password file for each of your email account, adapted from here.

1
2

echo -e "password\n" | gpg --encrypt -o .msmtp-gmail.gpg # enter id (email...)

“passowrd” is your email account password, use single quote if you have special character in your password (e.g. ! @ etc). Input your email used for GnuPG key generation, and RET again to finish.
Confirm your password with following:

1
2

gpg --decrypt .msmtp-gmail-tan.gpg
Add following to your .msmtprc (for sending emails) and .getmailrc files (for retrieving emails):

1
2

passwordeval "gpg --quiet --for-your-eyes-only --no-tty --decrypt ~/.msmtp-gmail.gpg"

delete your password line.
Confirm you that you can still retrieve emails with getmail.

1
2

getmail --rcfile ~/.getmail/getmailrc
That is it!!

9 Sending email notification with cron link

install msmtp-mta which creates a symbolic link from /usr/bin/sendmail to /usr/bin/msmtp
create /etc/msmtprc and append

1
2

aliases /etc/aliases
and create /etc/aliases:

1
2
3
4

your_username: your@email.com
# Optional:
default: your@email.com
Then modify the configuration of cronie daemon by replacing the with email notification link ExecStart command with:

1
2

ExecStart=/usr/bin/crond -n -m '/usr/bin/msmtp -t'
verified working, and yes, it is working fine, kind of annoying though with email notification.
Disable it

1
2

crontab -e

disable the cron job by “#” to comment out the line
set up mu4e in dot-emacs file to check and index emails at desirable interval

1
2
3
4

(setq
mu4e-get-mail-command "~/usr/bin/get_mail" ;;use script file
mu4e-update-interval 900) ;; update every 5 minutes

10 Creating org-mode links to emails for reminder and GTD purpose

activate in dot-emacs configuration file

1
2
3

(require 'org-mu4e)
(require 'org-old-mu4e) ;;for org-mode <8.x
in Headers View or Message View, C-c l
initialize org-mode file, C-c C-l, TAB to choose stored links
C-c C-o or RET to open at point
C-c C-l with cursor on existing link, to edit link
Official site for more

11 Other Features

☐ Email attachment reminders in mu4e see code here.
☐ Email alias for group emailing here

12 The keybindings and cheatsheet from mu4e official site.

12.1 The cheatsheet for mu is here.

12.2 The list of useful queries

refer to man mu-find, man mu-easy and man mu-query man pages and here.

Function	Notes
from:john flag:attach panda	messages with attach about panda from john
from:jim AND NOT flag: attach	all messages from jim without attach
Contact: alice	messages with alice in contact fields (to, from and cc)
date: 2w..now subject:mu4e	last 2 weeks emails about mu4e subject
file: pic*	all emails with attachments with filenames starting with pic
mime:Application/pdf	emails with pdf attachments
mime:image/*	emails with image attachments
mime:application/msword	emails with attachments in Word Format (.doc)
mime:application/vnd.openxmlformats-officedocument.wordprocessingml.document	(.docx)
maildir:/MIS	emails under maildir Note: slash
flag:attach	emails with attachments

Note:

☐ press s to start search in header view.
☐ man mu-query or mu-easy for more examples.
☐ The query is case-insensitive.
☐ to or from use email address instead of name; it works with names without quote.
☐ refer to here for a list of Incomplete list of MIME types – HTTP | MDN.
☐ BUGS? subject:“Google Alerts” does not work, it works without “”; WORKS with the new version now
☐ sth like PR-132 need to use regex //

☐

Other formats:

Extension	Kind of document	MIME Type
.xls	Microsoft Excel	application/vnd.ms-excel
.xlsx	Microsoft Excel (OpenXML)	application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
.tar	Tape Archive (TAR)	application/x-tar
.tiff .tif	Tagged Image File Format (TIFF)	image/tiff
.ppt	Microsoft PowerPoint	application/vnd.ms-powerpoint
.pptx	Microsoft PowerPoint (OpenXML)	application/vnd.openxmlformats-officedocument.presentationml.presentation
.jpg .jpeg	JPEG images	image/jpeg
.mpeg	MPEG Video	video/mpeg
.wav	Waveform Audio Format	audio/wav

12.3 Navigate The Views of Mu4e:

[C] +--------+ [RFCE]

--------> | editor | <--------

/ +--------+ \

/ [RFCE]^ \

/ | \

+-------+ [sjbB]+---------+ [RET] +---------+

+-------+ [q] +---------+ [qbBjs]+---------+

[sjbB] ^

[.] | [q]

+-----+

| raw |

+-----+

Default bindings

----------------

R: Reply s: search .: raw view (toggle)

F: Forward j: jump-to-maildir q: quit

C: Compose b: bookmark-search

E: Edit B: edit bookmark-search

12.4 The keybindings

12.4.1 key description

n,p	view the next, previous message
],[	move to the next, previous unread message
y	select the message view (if it’s visible)
RET	open the message at point in the message view

12.4.2 searching

s	search
S	edit last query
\|	narrow the search
b	search bookmark
B	edit bookmark before search
j	jump to maildir
M-left,\	previous query
M-right	next query
O	change sort order
P	toggle threading
Q	toggle full-search
V	toggle skip-duplicates
W	toggle include-related

12.4.3 marking

d	mark for moving to the trash folder
=	mark for removing trash flag (’untrash’)
DEL,D	mark for complete deletion
m	mark for moving to another maildir folder
r	mark for refiling
+,-	mark for flagging/unflagging
?,!	mark message as unread, read

u	unmark message at point
U	unmark all messages

%	mark based on a regular expression
T,t	mark whole thread, subthread
<insert>,*	mark for ’something’ (decide later)
#	resolve deferred ’something’ marks

x	execute actions for the marked messages

12.4.4 composition


R,F,C	reply/forward/compose

E	edit (only allowed for draft messages)

12.4.5 misc

a	execute some custom action on a header
\	pipe message through shell command
C-+,C–	increase / decrease the number of headers shown
H	get help
C-S-u	update mail & reindex
q,z	leave the headers buffer

12.4.6 Useful keybindings mu4e’s editor view

C-c C-c	send message
C-c C-d	save to drafts and leave
C-c C-k	kill the message
C-c C-a	` attach a file (pro-tip: drag & drop works as well)
(mu4e-specific)
C-S-u	update mail & reindex

dsdssd sd

7
JUN

Timothy Fry October 23, 2016

Thanks for putting this together! I find myself very tired of most Linux mail clients and am allured by the simplicity and “usability” of having my mail right in Emacs.

Your overview is far more complete and valuable than anything else I’ve found. Just seeing how you wrote the rc files answered many questions I had with greater clarity than the documentation or the Arch Wiki was able to provide.

I appreciate your time and effort. (-:

mistan March 9, 2017

You are very welcome!

zac October 22, 2017

I do think it’s a great post, especially the navigate table, easy to be understood. thanks for your time.

Blog