Difference between revisions of "Mu"

Revision as of 12:39, 5 November 2011

Mu is a multi-purpose tool shipped with Mailutils. It can be used for various mail- and database-related tasks, as well as an auxiliary tool for compiling and linking programs with Mailutils.

Syntax

Mu is a command line tool. Its invocation syntax is:

mu [options] command [args]

where options are options that affect the behavior of mu as a whole, command instructs it what it is to do and args are any arguments the command needs in order to be executed.

The commands are:

2047: Decodes or encodes email message headers.
acl: Tests Mailutils access control lists.
cflags: Shows compiler options needed to compile with Mailutils.
dbm: Invokes a DBM management tool.
filter: Applies a chain of filters to the input.
help: Displays a terse help summary.
imap: Invokes an IMAP4 client shell (in development).
info: Displays information about Mailutils compile-time configuration.
ldflags: Constructs a ld(1) command line for linking a program with Mailutils.
logger: Logs information using Mailutils log facility.
pop: Invokes a POP3 client shell.
query: Queries configuration values.
wicket

Scans wickets for matching URLs

help

The mu help command lists all available options and command names along with short descriptions of what each of them does. It is similar to the mu --help option.

A command name can be supplied as an argument to help, in which case it will display a help page for that particular command, e.g.:

 mu help ldflags

will output help for the ldflags command. It is synonymous to the --help option used with that particular command, e.g.: mu ldflags --help.

info

The mu info command displays information about Mailutils compile-time configuration. In normal form its output lists a single configuration flag per line, e.g.:

 $ mu info
 VERSION=2.99.93
 SYSCONFDIR=/etc
 MAILSPOOLDIR=/var/mail/
 SCHEME=mbox
 LOG_FACILITY=mail
 IPV6
 USE_LIBPAM
 HAVE_LIBLTDL
 WITH_GDBM
 WITH_GNUTLS
 WITH_GSASL
 ...

A configuration flag can consist either of a single word, indicating that a particular capability has been enabled at compile time, or of a keyword/value pair delimited by an equal sign, which indicates a particular value used by default for that feature. For example, IPV6 means that Mailutils was compiled with support for IPv6, whereas SYSCONFDIR=/etc means that the default place for configuration files is in /etc directory.

Such short output is convenient for using mu info in scripts to decide whether it is possible to use a given feature. To assist human users, the --verbose (-v) option is provided. It prints a short description next to each flag:

 $ mu info --verbose
 VERSION=2.99.93           - Version of this package
 SYSCONFDIR=/etc           - System configuration directory
 MAILSPOOLDIR=/var/mail/   - Default mail spool directory
 SCHEME=mbox               - Default mailbox type
 LOG_FACILITY=mail         - Default syslog facility
 IPV6                      - IPv6 support
 USE_LIBPAM                - PAM support
 HAVE_LIBLTDL              - a portable `dlopen' wrapper library
 WITH_GDBM                 - GNU DBM
 WITH_GNUTLS               - TLS support using GNU TLS
 WITH_GSASL                - SASL support using GNU SASL
 ...

cflags

The mu cflags command shows compiler options needed to compile a C source with Mailutils. It is intended for use in configuration scripts and Makefiles, e.g.:

 CFLAGS=-g -O2 `mu cflags`

ldflags

The mu ldflags command is a counterpart of cflags used for linking. It constructs a ld(1) command line for linking a program with Mailutils.

When used without arguments, it outputs ld arguments which would link only with the core Mailutils library libmailutils:

 $ mu ldflags
 -L/usr/local/lib -lmailutils

This command accepts a number of keywords which allow to select a particular subset of Mailutils libraries to link with. In particular, the argument all instructs it to link in all available libraries:

 $ mu ldflags all
 -L/usr/local/lib -lmu_mbox -lmu_mh -lmu_maildir -lmu_imap -lmu_pop \
 -lmu_mailer -lmu_compat -lmailutils -lmu_auth -lgsasl -lgnutls -lgcrypt \
 -lldap -lgnuradius -lpam -ldl

Other available keywords are:

mbox: Link in the UNIX mbox format support.
mh: Link in the MH format support.
maildir: Link in the Maildir format support.
imap: Link in the IMAP protocol support.
pop: Link in the POP protocol support.
nntp: Link in the NNTP protocol support.
mailer: Enable support for mailers.
sieve: Link in the support for Sieve mail filtering language.
dbm: Link in the support for DBM databases (libmu_dbm library).
compat: Provide a compatibility layer for Mailutils 2.x.
auth: Link in the Mailutils authentication library.
guile: Provide Guile language bindings.
python: Provide Python language bindings.
cfg: Link in the Mailutils configuration library.
argp: Link in the library for command line parsing.

query

The mu query command queries values from Mailutils configuration files. It takes one or more configuration paths as its arguments. On output, it displays the values it found, each value on a separate line. If the requested value is a block statement it is displayed in full. For example, if main configuration file contained:

  logging {
    syslog yes;
    facility mail;
  };

Then:

 $ mu query .logging.syslog
 syslog yes;
 $ mu query .logging.syslog .logging.facility
 syslog yes;
 facility mail;
 $ mu query .logging
 logging {
   syslog yes;
   facility mail;
 };

Several command line options allow to modify output format. The --value option instructs the command to output only values:

 $ mu query --value .logging.syslog
 yes

The --path option instructs it to print full path names for each value:

 $ mu query --path .logging.syslog
 logging.syslog: yes

The --program option instructs mu to behave as if it was called under another program name. For example, the following command:

 $ mu query --program=pop3d .server.transcript

will return the value of the server.transcript statement for pop3d utility.

By default, mu query operates on the main configuration file. Another configuration file can be supplied using the --file (-f) option:

 $ mu query --file /usr/local/etc/file.conf .pidfile

2047

The mu 2047 command is a filter for decoding or encoding email message headers formatted in accordance with RFC 2047. By default, it operates in encode mode and assumes the iso-8859-1 encoding. If arguments are supplied in the command line, they are treated as the text to operate upon. Otherwise the command acts as a UNIX filter, reading lines from the standard input and printing results on the standard output.

For example:

 $ mu 2047 'Keld Jørn Simonsen <keld@dkuug.dk>'
 =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>

The decode mode can be requested via the --decode (-d) option:

 $ mu 2047 --decode '=?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>'
 Keld Jørn Simonsen <keld@dkuug.dk>

The --charset (-c) option changes the default character set. It is meaningful both in decode and in encode modes. In decode mode it instructs the utility to convert the output to the given character set. In encode mode it indicates the encoding of the input data, which will be reflected in the resulting string:

 $ mu 2047 --charset=utf-8 'Keld Jørn Simonsen <keld@dkuug.dk>'
 =?utf-8?Q?Keld J=C3=B8rn Simonsen <keld@dkuug.dk>?=

The --encoding (-E) option can be used in encode mode to change the output encoding. Valid arguments for this option are: quoted-printable (the default) or base64.

The --newline (-n) option prints an additional newline character after each line of output.

filter

The mu filter command applies a chain of filters to the input. The filters to apply and their arguments are given in the command line. The full invocation syntax is:

 mu filter [option] filter-chain

The syntax for filter-chain in Backus-Naur form follows:

 <filter-chain> ::= <filter> | <filter-chain> "+" <filter>
 <filter> ::= <filter-spec> <ARG>* 
 <filter-spec> ::= <WORD> | "~" <WORD>

where <WORD> stands for the filter name and <ARG> represents filter arguments. To obtain a list of known filter names, run:

 mu filter --list

Filters are applied in the order of their appearence, from left to right and operate in encode mode. The plus sign has the same meaning as pipe in shell. The default mode can be changed using the --decode (-d) and --encode (-e) options. Whatever the default mode is, the ~ before filter name reverts the mode for that filter alone.

For example, to encode the contents of file file.txt in Base64 run:

 mu filter base64 < file.txt

To convert it to base64 and use CRLF as line delimiters, run:

 mu filter base64 + crlf < file.txt

The following command will decode the produced output:

 mu filter --decode crlf + base64

It can also be written as

 mu filter ~crlf + ~base64

The following example converts the input from ISO-8859-2 to UTF-8, quotes eventual From occurring at the beginning of a line, encodes the result in Base64 and changes line delimiters to CRLF:

 mu filter iconv iso-8859-2 utf-8 + from + base64 + crlf

This final example removes UNIX-style comments from the input and joins continuation lines:

 mu filter --decode inline-comment -S '#' + linecon

Such invocation can be useful in shell scripts to facilitate configuration file processing.

acl

The mu acl command tests Mailutils access control lists. By default it reads ACL from the Mailutils configiration file section "acl". The command takes a list of IP addresses as its arguments, applies the ACL to each of these addresses in turn and prints the result.

To select the ACL to test, two options are provided. The --file (-f) option supplies the name of configuration file to read instead of the default one. The --path (-p) option supplies the path to the ACL section to use instead of the default ".acl". For example, to test ACL in section server 213.130.1.232 of file /etc/pop3d.conf use:

 mu acl --file=/etc/pop3d.conf --path=/server="213.130.1.232"/acl address

As an example of its use, consider file test.conf with the following contents:

acl {
        deny from 10.10.10.1;
        deny from 10.10.1.0/24;
        log from any "Connect from ${address}";
        allow from 10.0.0.0/8;
        allow from 192.168.1.0/24;
        deny from any;
}

Then, running mu acl --file=test.conf 127.0.0.1 you will get:

 Testing 127.0.0.1:
 mu: Connect from 127.0.0.1
 127.0.0.1: deny

More examples:

 $ mu acl --file=test.conf 127.0.0.1 10.10.10.1 10.10.1.3 10.5.3.1 192.168.1.0 192.168.2.0
 Testing 127.0.0.1:
 mu: Connect from 127.0.0.1
 127.0.0.1: deny
 Testing 10.10.10.1:
 10.10.10.1: deny
 Testing 10.10.1.3:
 10.10.1.3: deny
 Testing 10.5.3.1:
 mu: Connect from 10.5.3.1
 10.5.3.1: accept
 Testing 192.168.1.0:
 mu: Connect from 192.168.1.0
 192.168.1.0: accept
 Testing 192.168.2.0:
 mu: Connect from 192.168.2.0
 192.168.2.0: accept

The mu option --debug-level will give you a deeper insight into the address matching algorithm:

 $ mu --debug-level=acl.trace9 acl --file test.conf 127.0.0.1
 Testing 127.0.0.1:
 mu: Checking sockaddr 127.0.0.1
 mu: 1:deny: Does 10.10.10.1/255.255.255.255 match 127.0.0.1? no; 
 mu: 2:deny: Does 10.10.1.0/255.255.255.0 match 127.0.0.1? no; 
 mu: 3:log: Does any match 127.0.0.1? yes; 
 mu: Expanding "Connect from ${address}"
 mu: Expansion: "Connect from 127.0.0.1". 
 mu: Connect from 127.0.0.1
 mu: 4:accept: Does 10.0.0.0/255.0.0.0 match 127.0.0.1? no; 
 mu: 5:accept: Does 192.168.0.0/255.255.0.0 match 127.0.0.1? no; 
 mu: 6:deny: Does any match 127.0.0.1? yes; 
 127.0.0.1: deny

wicket

The mu wicket command looks up matching URLs in the Mailutils ticket file (by default, ~/.mu-tickets) and prints them. The URLs to look for are supplied in the command line.

Consider the following ticket file as an example:

smtp://foo:bar@*
smtp://bar:baz@gnu.org
*://baz:qux@*
*://quux:bar@gnu.org

Now, running mu wicket smtp://bar@gnu.org will show:

smtp://bar@gnu.org: /home/user/.mu-tickets:2

which means that this URL matches the line 2 in your .mu-tickets file. The wicket command does not show the actual matching line to avoid revealing eventual security-sensitive information. You can instruct it to do so using the --verbose (-v) option:

 $ mu wicket -v smtp://bar@gnu.org
 smtp://bar@gnu.org: /home/user/.mu-tickets:2: smtp://bar:***@gnu.org

As you see, even in that case the tool hides the actual password part by replacing it with three asteriscs. If you are working in a secure environment, you can tell mu wicket to show passwords as well, by giving the -v option twice.

A counterpart of --verbose is the --quite (-q) option, which instructs wicket to suppress any output, excepting error messages. This can be used in scripts, which analyze the mu exit code to alter the control flow.

The mu wicket tool exits with code 0 if all URLs were matched and with code 1 if some of them were not matched in the ticket file. If an error occurred, the code 2 is returned.

dbm

The mu dbm tool manages DBM files using libmu_dbm. The invocation syntax is:

 mu dbm command [options] file [keys]

or

 mu dbm [options] command file [keys]

where command selects the operation mode, options modify the tool behavior and file specifies the DBM file to operate upon. Some commands allow for optional keys to be specified.

The file argument can be either a DBM file name or a Database URL.

Create a Database

The create command and its synonym load instruct the tool to create a new database:

 mu dbm create file.db

If the argument file already exists, it will be truncated prior to adding new records to it.

The data to populate the database with are read from the standard input. The dbm tool supports several formats for these data, which are discussed later. In the simplest case (a so called format 0.0) each input line must consist of two fields separated by any amount of whitespace. The first field is treated as a key and the second one as the corresponding value.

The usual way to read data from a file is, of course, by redirecting the file to the standard input as in:

 mu dbm create file.db < input.txt

There is also a special option for that purpose: --file (-f). Thus, the following command is equivalent to the one above:

 mu dbm create --file input.txt file.db

The --file option has the advantage that it allows, in conjunction with another options, for copying input file metadata (owner UID, GID and file mode) to the created database. For example, the following command ensures that the created database file will have the same metadata as the input file:

 mu dbm create --file input.txt --copy-permissions file.db

The --copy-permissions (-P) option is one that does the job.

There are also other ways to control mode and ownership of the created database, which are described below.

More advanced dump formats (e.g. 1.0) carry additional information about the file, including its original name, ownership and mode. If input is in one of these formats, the file name argument becomes optional. If it is not supplied, the name stored in the input stream will be used. For example, supposing that the file users.dump is in format 1.0, the following command suffices to restore the original filename, ownership, mode and, of course, data:

 mu dbm create --file users.dump

Add Records to a Database

The add command adds records to a database. Records are read from the standard input and must be formatted as for create command.

 mu dbm add file.db

If the argument file does not exist, it will be created.

Adding a record with a key which is already present in the database produces an error. To replace existing records, use the replace command instead.

The same options that affect the behavior of create apply to add and replace as well, e.g.:

 mu dbm replace --file input.txt --copy-permissions file.db

Delete Records

To delete records, use the delete command. It reads a list of keys to delete to be specified as arguments in the command line:

 mu dbm delete file.db foo bar

The command above will delete from file.db records with keys foo and bar.

It is not an error to attempt to delete a key that does not exist in the database, although such use will produce a warning message.

By default, keys are matched literally. It is also possible to use various pattern matching techniques, depending on the option specified.

The --glob (-G) option instructs the tool to use UNIX globbing pattern matching. For example, the command below will delete all keys starting with foo and ending with a decimal digit:

 mu dbm delete file.db 'foo*[0-9]'

(note the quoting necessary to prevent shell from interpreting the metacharacters itself).

Another option, --regex (-R) instructs mu to treat keys as extended regular expressions:

 mu dbm delete --regex file.db 'foo.*[0-9]{1,3}'

Both options are affected by the --ignore-case (-i) option, which turns on case-insensitive matching.

Using pattern matching to delete records can be a risky operation as selecting a wrong pattern will lead to removing wrong records. It is recommended to first use the list mode described below to verify that the patterns match the right keys.

List the Database

The list command lists the content of the database:

 mu dbm list file.db

By default, entire content is listed on the standard output in format 0.0.

If supplied more than one command line argument, this mode treats the rest of arguments after the database file name as the keys to look for and lists only records with these keys:

 $ mu dbm list file.db foo bar
 foo 1
 bar 56

The --glob and --regex options instruct the tool to use UNIX globbing or extended regular expression matching, correspondingly. These are described in detail above.

Dump the Database

The dump subcommand dumps the database to the standard output in a format suitable for backup or sending over the network (a version 1.0 format).

 mu dbm dump file.db > file.dump

The produced file is suitable for input to the create (load) command. Among other uses, this provides for an easy way to convert databases between various formats supported by Mailutils. For example this is how to convert the database file file.db to the GDBM database new.db:

 mu dbm dump file.db | mu dbm create gdbm://new.db

Both list and dump subcommands share the same set of options. In fact, they are pretty similar, except that use different defaults. The list subcommand is oriented to produce a human-readable output, whereas the dump subcommand is oriented towards backup purposes.

Dump Formats

As of version 2.99.93, mu dbm supports two formats for dumping DBM databases. Both formats are line-oriented. Comments are introduced with a sharp (#) sign in the column 0 of a line, followed by at least one white space character (space or tab). Sharp sign followed by a colon (#:) introduces a paragmatic comment, which carries some additional information to the loader.

The version 0.0 format is suitable for databases whose records contain only ASCII data. In this format, each record occupies a separate line, which consists of the key and value separated by a single TAB character. Empty lines are ignored. For example:

 $ mu list /etc/mail/users.db
 root    guessme
 smith   pAssword
 qed     fooBar

Output in version 0.0 format is human readable and can be used as input to the popauth utility. However, version 0.0 has serious drawbacks. First of all, it is not suitable for databases that contain binary data. Secondly, it cannot properly handle keys beginning with a sharp sign or containing TAB characters. The version 1.0 format is free from these drawbacks.

The version 1.0 format begins with a header containing important information about the file, such as its file name, ownership and file mode. This information is stored in pragmatic comments and allows mu dbm load to easily recreate an exact copy of the file. The following comments are defined:

#:version=1.0: Indicates that the data that follow are in version 1.0 format.
#:filename=s: Original database file name, without directory parts.
#:uid=n: Owner UID.
#:user=s: Owner name.
#:gid=n: Owner GID
#:group=s: Owner group name.
#:mode=o: File mode in octal

Following this header are actual data. Each record is output in two parts: key and value. Each part begins with a #:len=n construct on a line by itself, where n is the length of the data in decimal. This line is followed by one or more lines of the actual data, encoded in base64. The data are formatted so that each line does not exceed 76 bytes in length (not counting the terminating newline). An example of this format follows:

# Database dump file created by GNU Mailutils 2.99.93 on Tue Nov  1 13:28:03 2011
#:version=1.0
#:file=users.db
#:uid=0,user=root,gid=25,group=mail,mode=640
#:len=6
c21pdGgA
#:len=9
cEFzc3dvcmQA
#:len=5
cm9vdAA=
#:len=8
Z3Vlc3NtZQA=
#:len=4
cWVkAA==
#:len=7
Zm9vQmFyAA==

Exit Codes

The table below summarizes exit codes used by mu dbm:

Code	Symbolic name	Meaning
0	EX_OK	Successful termination
64	EX_USAGE	Command line usage error
65	EX_DATAERR	Error in user-supplied data: the input file is badly formatted, or some of the data supplied in the command line are invalid (e.g. user name, uid or the like), etc.
66	EX_NOINPUT	Cannot open input file
67	EX_NOUSER	No such user or UID when trying to set output file ownership
69	EX_UNAVAILABLE	Operation cannot be performed due to some kind of problem (e.g. access to the file denied, etc.)
70	EX_SOFTWARE	Internal software error
74	EX_IOERR	Input/output error

logger

The mu logger tool logs information using Mailutils log facility.

Syntax:

 mu logger [options] [message]

The message argument, if supplied, gives the text to log. If not supplied, the utility reads lines of text from standard input or a file (if the --file option is given) and sends them to log:

 # Send text to log
 $ mu logger I am here
 # Log each line from file.txt
 $ mu logger --file file.txt
 # Read stdin and log it:
 $ mu logger

The default logging channel is bound to standard error. To bind it to syslog, use the --syslog command line option. In that case mu uses facility user and priority err. You can change this by using the --priority (-p) option. Its argument is either a syslog facility name or facility and severity names separated by a dot. Thus, the following invocation will use facility auth, severity info:

 mu logger --priority auth.info

The syslog tag can be set using the --tag (-t) option:

 mu logger --tag myprog

The default tag is mu-logger.

The --severity (-s) option sets the Mailutils severity level. Its argument can be any of the following strings: debug, info, notice, warning, error, crit, alert, emerg.

Finally, the --locus (-l) option binds log messages to a location in a file. Its argument has the following syntax:

 file:line[:col]

where file is the file name, line is the line number and optional col is the column number in that file.

For example, the following invocation:

 mu logger --locus mailutils.rc:34 Suspicious statement

will send the following to the log:

 mu-logger: mailutils.rc:34: Suspicious statement

pop

The mu pop command invokes an interactive POP3 client shell. It reads commands from the standard input, executes them and displays the results on the standard output. If the standard input is connected to a terminal, the readline and history facilities are enabled (provided that Mailutils is configured with GNU Readline).

The mu pop commands form two major groups. The POP3 protocol commands interact with the remote POP3 server and display responses obtained from it. These commands are named after their POP3 equivalents. Another group, internal commands, are used to configure the shell itself.

POP3 protocol commands

connect [-tls] hostname [port]: Open connection to hostname. If the -tls option is given, TLS encryption (also known as POPS protocol) will be used. If port argument is not given, the command uses port 110 for a plain POP connection or 995 for POPS (if -tls is given).

stls: Start TLS negotiation. This command is valid only after successful unencrypted connection has been initiated (using connect without -tls argument).

user name: Send user name to the server. The pass command must follow.
pass [password]: Send password. This command is valid only after user. If the password argument is omitted, the shell will ask you to enter it. While entering, both echoing and history recording will be disabled. Use this to avoid compromising your password.

apop user [password]: Authenticate with APOP. If the password argument is omitted, you will be asked to supply it. While entering, both echoing and history recording will be disabled.

capa [-reread] [name...]: List server capabilities. Any number of name arguments is accepted. If given, the shell will display only the named capabilities, otherwise it displays entire list. By default capa reuses the response of its previous invocation (if there was any), instead of resending the CAPA command to the server. To force it do so, use the -reread option.

noop: Send a no operation.

stat: Get the mailbox size and number of messages in it.

uidl [number]: Shows unique message identifiers. Without arguments, shows identifiers for each message in the mailbox. If number is given, the command returns the UIDL of that particular message only.

list [number]: Lists messages. See above for the meaning of number. Each line of the produced listing contains describes a single message and contains at least the message number and size in bytes. Depending on the POP3 server implementation, additional fields may be present. For example, Mailutils pop3d can also output number of lines in the message in the additional third field.

retr number: Retrieve a message.

top msgno [number]: Display message headers and first number (default 5) lines of its body.

dele number: Mark message for deletion.
rset: Remove deletion marks.

quit: Quit pop3 session.

disconnect: Close existing connection.

Internal commands

verbose [on|off|mask|unmask] [secure [payload]]

Control output verbosity. Without arguments the verbose command shows current settings. The argument off (the default) turns off all additional output. The verbose on command enables POP3 protocol tracing output. Additional arguments can be used to provide more verbosity. The secure argument enables display of user passwords in the trace output and the payload enables showing payload data (e.g. response body sent in the reply to RETR command, etc.) Thus, the full diagnostics output is obtained by

 verbose on secure payload

The mask and unmask argument allow to disable and enable such additional verbosity. For example, supposing the command above is in action, the following command will suppress the display of user passwords in the traces:

 verbose mask secure

Similarly, verbose unmask secure will turn it back again.

prompt string: Set command prompt. The argument can contain variable references in the following forms:

 $name
 ${name}

where name is the variable name. Such references are expanded to the actual value of the variable at the time of expansion. The following variables are defined:

Variable	Expansion
user	Login name of the authenticated POP3 user. If the session is not authenticated yet, expands to [nouser].
host	Name of the remote host, or [nohost] if no connection is established.
program-name	Name of the program, as typed on the command line to invoke it.
canonical-program-name	mu
package	Mailutils
version	Mailutils version number
status	Session status. One of: disconnected, connected or logged in.