If you are upgrading from an existing version of XMail it is strongly suggested that you read the ChangeLog.txt file to check for important modifications that can change the way XMail works.

If you want to run XMail without reading the whole manual then go to chapter "Part 7 - Configuration".

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.

This server is born because I needed a free and stable Mail Server for my old company, which was using a Windows network. I don't like to reinvent the wheel but the need of some features pushed me to start a new project. If I could have used a Linux server on my network, I would probably have not had to write any code, but this was not my case. I needed it to be portable. XMail runs on Windows NT/2000/XP and on the following Unix versions: Linux, Solaris and FreeBSD. 

Another reason that drove me to write XMail is the presence of the same tedious steps when setting up a typical mail server, ie: sendmail + qpopper + fetchmail if one needs SMTP, POP3 and external synchronization, or: sendmail + qpopper for only SMTP and POP3 (I've quoted sendmail, qpopper and fetchmail, but there are many other packages You can use to fulfill those needs). With XMail you get an all-in-one package with a central administration that can simplify the above common steps.

The first version of XMail Server was created on Windows NT and Linux, and now, FreeBSD and Solaris versions are available. The supported compilers are gcc for Linux, FreeBSD and Solaris and M$ Visual C++ for Windows NT/2000/XP.

Linux and NT/2000/XP ports are stable, while Solaris and FreeBSD ports have not been tested.

- Any version of Linux that has glibc

- Windows NT with Winsock's ws2_32.dll (the TCP/IP stack) correctly installed and configured

- A working DNS and gateway to the internet (if You plan to use XMail over the Web)

- To compile on Linux you need gcc and glibc

- To compile on Windows you need MS Visual C++ (a project file is provided) or any other working compiler that supports the Win32 SDK

Always get the source code from the XMail home page to get the latest version:

http://www.xmailserver.org 

Use the correct distribution for your system and don't mix Unix files with Windows files -this is one of the most common cause of XMail bad behavior. When You unzip the package you have to check that the MailRoot directory contained inside the package itself is complete (look at the directory tree listed below) because some unzippers do not create empty directories.

There is a Visual C++ project for Windows NT. You have make files for each Unix version:

# make -f Makefile.lnx     (Linux)

# make -f Makefile.slx     (Linux on SPARC)

# make -f Makefile.plx     (Linux on PPC)

# gmake -f Makefile.bsd  (FreeBSD -You need GCC and GMAKE to compile XMail)

# make -f Makefile.sso    (Sun/Solaris on SPARC -You need GCC to compile XMail)

# make -f Makefile.ssx    (Sun/Solaris on Intel -You need GCC to compile)

As soon as the project reach a higher maturity I plan to supply a configure script.

Under Linux an init.d startup script is supplied (xmail) to allow you to run XMail as a standard rc? daemon. You must put it into /etc/init.d (it depends on which distro You're using) directory and then create K??xmail - S??xmail links into the proper directories.

Under Windows NT/2000/XP the XMail executable is a Win32 service by default and if you want to build it as an application instead of as a service then you have to comment the statement "#define SERVICE" in MainWin.cpp.

 [ Linux/Solaris/FreeBSD ]

 [ Windows NT/2000/XP ]

If you just want to test XMail then setup an environment variable called MAIL_ROOT that point to the XMail Server root directory.

• Linux:                         export MAIL_ROOT=/var/XMailRoot

• Windows NT/2000/XP:   set MAIL_ROOT=C:\MailRoot

The Mail root directory contains the following files:

aliases.tab <file>

aliasdomain.tab <file>

domains.tab <file>

dnsroots <file>

extaliases.tab <file>

mailusers.tab <file>

message.id <file>

pop3links.tab <file>

server.tab <file>

smtpgw.tab <file>

smtpfwd.tab <file>

smtprelay.tab <file>

smtpauth.tab <file>

smtpextauth.tab <file>

userdef.tab <file>

ctrlaccounts.tab <file>

spammers.tab <file>

spam-address.tab <file>

pop3.ipmap.tab <file>

smtp.ipmap.tab <file>

ctrl.ipmap.tab <file>

finger.ipmap.tab <file>

And the following sub-directories:

bin <dir>

cmdaliases <dir>

tabindex <dir>

dnscache <dir>

mx <dir>

ns <dir>

custdomains <dir>

filters <dir>

logs <dir>

pop3locks <dir>

pop3linklocks <dir>

pop3links <dir>

spool <dir>

local <dir>

temp <dir>

0 <dir>

0 <dir>

mess <dir>

rsnd <dir>

info <dir>

temp <dir>

slog <dir>

lock <dir>

cust <dir>

froz <dir>

...

...

userauth <dir>

pop3 <dir>

smtp <dir>

domains <dir>

And for each domain "DOMAIN" handled a directory (located under .\domains):

DOMAIN <dir>

userdef.tab <file>

Inside which reside, for each account "ACCOUNT" (under .\domains\ACCOUNT):

ACCOUNT <dir>

user.tab <file>

mlusers.tab <file>       [ mailing list case ]

mailproc.tab <file>      [ optional ]

pop3.ipmap.tab <file> [ optional ]

and

mailbox <dir>

And for each mailbox structure, a Maildir structure:

Maildir <dir>

tmp <dir>

new <dir>

cur <dir>

You can use external modules (executable files) to perform user authentication instead of using XMail mailusers.tab lookups. In the userauth directory you will find one directory for each service whose authentication can be handled externally (currently only for POP3).

Supposing that we have to authenticate USERNAME from DOMAIN, XMail will first try to lookup (from userauth/pop3) a file named:

DOMAIN.tab

else:

.tab

If one of these files is found, XMail will authenticate USERNAME - DOMAIN using this file.

The authentication file is a TAB file (see "configuration files" in Part 7) which has the following structure:

"auth-action"[TAB]"command"[TAB]"arg-or-macro"[TAB]...[NEWLINE]

Each argument can be:

@@USER will be substituted with the USERNAME to authenticate

@@DOMAIN will be substituted with the DOMAIN to authenticate

@@PASSWD will be substituted with the user password

@@PATH will be substituted with the user path

The values for "auth-action" can be:

userauth = executed when user authentication is required

useradd = executed when a user need to be added

useredit = executed when a user change is needed

userdel = executed when a user deletion is needed

domaindrop = executed when all domain users need to be deleted

The first line that stores the handling command for the requested action is executed as:

command arg0 ... argN

that in case of success must return zero. Any other exit code will be interpreted as authentication operation failure, which, in the 'userauth' case means that this user will be not authenticated.

If the execution of the command fails for a system reason (command not found, access denied, etc.) then the user will not be authenticated.

If none of these files is found then usual authentication is performed (with mailusers.tab).

Note: The external authentication does not replace the purpose of the user entry in the mailusers.tab file (which is mandatory).

When a message is to be sent through a SMTP server that requires authentication, XMail provides a way to handle this task by setting up the userauth/smtp subdirectoryproperly.

Suppose that a mail is to be sent through the SMTP server mail.foo.net, this makes XMail to search for a file named (in userauth/smtp):

mail.foo.net.tab

then:

foo.net.tab

then:

net.tab

If one of these files is found then its content is used to authenticate the SMTP client session. The structure of this file, is the TAB format used for most of the XMail configuration files. The first uncommented line (not begining by the '#' character) is used to choose the authentication method and the following lines have the following format:

"auth-type"[TAB]"param1"...[TAB]"paramN"[NEWLINE]

Valid lines are:

"plain" "username" "password"

or

"login" "username" "password"

or

"cram-md5" "username" "password"

or

"external" "auth-name" "secret" "prog-path" "arg-or-macro" ...

Where "auth-name" can be any symbolic name and "arg-or-macro" can be a program argument or one of the following:

@@CHALL = server challenge string

@@SECRT = authentication secret

@@RFILE = output response file path

Ex:

"external" "RSA-AUTH" "mysecret" "/usr/bin/myrsa-auth" "-c" "@@CHALL" "-s" "@@SECRT" "-f" "@@RFILE"

XMail will send a line like: "AUTH RSA-AUTH" to the SMTP server, and wait for a line like: "3?? base64-challenge".

Then XMail will decode the base64-challenge and invoke the external program to get the response to send to the SMTP server. The external program must return zero in case of success and must put the

response into the file specified by "@@RFILE" (without new line termination).

If a message that has as target domain sub1.sub2.domain.net comes to XMail then XMail will decide if this domain will require a custom domain processing by trying to lookup the following file:

sub1.sub2.domain.net.tab

.sub2.domain.net.tab

.domain.net.tab

.net.tab

.tab  in the 'custdomains' directory.

If one of these files is found then the incoming mail will get a custom domain processing by executing

commands that are stored in this file.

The .tab file format is:

"command"[TAB]"arg-or-macro"[TAB]...[NEWLINE]

store commands (internals or externals) that have to be executed on the message file. The presence of this file is optional an if it does not exist then the default processing is to send the message via SMTP.

Each argument can be:

@@FROM will be substituted with the sender of the message

@@RCPT will be substituted with the target of the message

@@FILE will be substituted with the message file path (the external command _must_ only read the file)

@@MSGID will be substituted with the (XMail unique) message id

@@MSGREF will be substituted with the reference SMTP message id

@@TMPFILE will create a copy of the message file to a temporary file

It can be used with "external" commands but in that case this is the external program responsibility to delete the temporary file.

Supported commands are:

CmdAliases implements aliases that are handled only through commands and can be considered a user level implementation of custom domain processing commands.

The command set is the same than the one described above in the custom domain processing section and it won't be explained again here (see "Part 10 - Custom domain mail processing").

For every handled domain (listed in the domains.tab file) a directory with the name of the domain is created in the 'cmdaliases' subdirectory. This directory is automatically created and removed when you add/remove domains through the CTRL protocol (or CtrlClnt).

When a mail from USER@DOMAIN is received by XMail and the domain DOMAIN is to be handled locally, if the standard users/aliases lookup fails, then a file named USER.tab is searched in the folder $MAIL_ROOT/cmdaliases/DOMAIN.

If this file is found then commands listed in this file (the file format is described in the previous section) are executed by the server in order to process the mail message.

Note: All domains and user names, when applied to the file system, must be lower case.

Use of the command [SMTP] must be done with care because it can create mail loops in XMail.

This feature offers a way to filter incoming emails by providing the ability to execute external programs, such as scripts or real executables, that examine (modify or both) the message and give a response by the means of a return value:

  96: the email is OK and can continue its trip
  97: the email is rejected without notification and is deleted
  98: the email is rejected without notification but kept on disk
  99: the email is rejected and a notification is sent to the sender
100: the email is modified and can continue its trip

When a message is received by the SMTP server for user foo@xyzw.abc then XMail searches ins the filters subdirectory to find a file named (user processing):

foo@xyzw.abc.tab

If this file is not found then XMail will look for (domain processing):

xyzw.abc.tab

If this file is not found then XMail will search a file called ".tab" in the filters subdirectory. This offers a way to specify a default mail filtering active for all the domains. If this file is not found then the message will continue its travel, otherwise the message will be submitted to all the filters specified in the *.tab file.

Note: To create a file called ".tab" under Windows open a DOS box and type: echo . > .tab

         (Windows will not allow you to create such a file from Explorer -the file manager)

The syntax of the *.tab file is:

"command"[TAB]"arg-or-macro"[TAB]...[NEWLINE]

Ex:

"C:\MailRoot\bin\MyFilter.exe"[TAB]""C:\MailRoot\bin"[TAB]"@@FILE"

This will invoke the program MyFilter.exe to check all incoming emails.

And arg-or-macro can be:

@@FROM will be substituted with the sender of the message

@@RCPT will be substituted with the target of the message

@@FILE will be substituted with the message file path (the external command may modify the file if it's going to return 100 as command exit value)

@@MSGID will be substituted with the (XMail unique) message id

@@MSGREF will be substituted with the reference SMTP message id

Here "command" is the name of an external program that must process the message and return a code. If it returns 99 then the message is rejected and a notification message is sent to the sender. By returning 98 the message will be rejected without notification while by returning 97 the message is rejected without notification and without being frozen (a 98 response could lead to a frozen message if the SERVER.TAB configuration file enabled this).

Note: If a filters returns a value different from 99, 98 or 97 then the message can continue its trip.

The filter command may also modify the file and return 100, having the ability to change the file content (AV scanning, content filter, message rewriting, etc.).

If the filter changed the message file then it MUST keep the message structure and it MUST terminate all the lines with a <CR><LF>.

The spool files have the following structure:

SmtpDomain         [ 1st line ]

SmtpMessageID    [ 2nd line ]

MAIL FROM:<...>   [ 3th line ]

RCPT TO:<...>      [ 4th line ]

<<MAIL-DATA>>    [ 5th line ]

...

After the "<<MAIL-DATA>>" tag (5th line) the message follows. The message is composed by a header section and, after the first empty line (CR-LF), the message body.

To get the C/ C++ source code of ready-to-use email filters visit:

http://www.twd-industries.com/en/xmail.htm      // To patch attachments and HTML Tags

http://software.dolist.net/xscanner.asp             // To scan emails with a list of keywords

A full implementation of the SMTP protocol includes the ability to perform mail routing bypassing DNS MX records by using the the "RCPT TO: <>" request.

If an email from xuser@hostz directed to @hosta,@hostb:foouser@hostc is received by @hosta then it will be sent to @hostb using        "MAIL FROM:<@hosta:xuser@hostz>"

                                    and "RCPT TO: <@hostb:foouser@hostc>" and then:

it will be sent to @hostc using "MAIL FROM:<@hostb,@hosta:xuser@hostz>"

                                    and "RCPT TO: <foouser@hostc>".

The new spool tree has been designed to enable XMail to handle very large queues. Instead of having a single spool directory (like versions older than 0.61) a two layer deep splitting has been introduced:

0 <dir>

0 <dir>

mess <dir>

rsnd <dir>

info <dir>

temp <dir>

slog <dir>

cust <dir>

froz <dir>

...

...

When XMail needs to create a new spool file a spool path is chosen randomly way and a new file with the format mstime.pid.hostname is created in the temp subdirectory.

When the spool file is ready to be committed, it is moved in the mess subdirectory that holds newer spool files. If XMail fails to send a new message (the ones located in the mess subdirectory) then it creates a log file (with the name of the message file) in the slog subdirectory and moves the file from mess directory to the rsnd folder.

During the message processing the message itself is locked by creating a file inside the lock subdirectory (with the name of the message file). If the message has permanent delivery errors or has expired and if the option "RemoveSpoolErrors" of the SERVER.TAB file is off, then the message file is moved in the froz subdirectory.

The following commands are understood by the XMail ESMTP server:

RFC 821 requires that every server support the HELO, RSET, NOOP, MAIL, RCPT, DATA and QUIT verbs ( verbs are not case sensitive but it is much safer to use UPPERCASE verbs since many clients and servers will close the connection if you send 'Hello' or 'hEllO'). 

The following commands are understood by the XMail POP3 server :

Most of the XMail configuration settings can be changed from the XMail command line:

It is possible to administrate remotely XMail: a "Controller server" is waiting for TCP/IP connections on port 6017 (change it via the command line switch "-Cp nport" documented in "Part 19 - Command line").

The XMail Admin Server "speaks" a protocol that can be used by external GUI utilities written with the more disparate scripting languages to administer remotely the mail server. The protocol is based on the idea of sending formatted strings of commands and waiting for formatted server responses and error codes. All the lines, commands and responses, are terminated by a <CR><LF> pair.

XMail has the ability to deliver locally prepared mail files. Just put them in the spool/local directory.

The format of these files is strict:

mail from:<...>[CR][LF]

rcpt to:<...>[CR][LF]

...

[CR][LF]

message text with [CR][LF] line termination

All lines must be [CR][LF] terminated, with one mail-from statement, one or more rcpt-to statements, an empty line and the message text.

Mail files must not be created directly in the /spool/local directory but instead in the /spool/temp directory. When the file is ready then only it can be moved in /spool/local.

The file name format is:

stime-seqnr.pid.hostname

where:

stime = system time in sec from 01/01/1970

seqnr = sequence number for the current file

pid = process or thread id that generated the local mail

hostname = the host name of the machine where the process that created the mail is running

Ex:

97456928-001.7892.home.bogus

XMail has a number of LMAIL threads that periodically scan the /spool/local directory watching for locally generated mail files. You can modify the number of threads with the "-Ln nthreads" command line option. The suggested number ranges from three to seven.

You can use the CtrlClnt.exe tool to send administrative commands to XMail.

These commands are defined in a previous section (see Part 20 - XMail's administration protocol).

The syntax of the CtrlClnt utility is:

CtrlClnt [-snuptf] ...

-s server = set server address

-n port = set server port [6017]

-u user = set username

-p pass = set password

-t timeout = set timeout [60]

-f filename = set dump filename [stdout]

with commands and parameters using the following syntax:

CtrlClnt -s mail.foo.org -u davide.libenzi -p ciao useradd home.bogus foouser foopasswd U

will execute the command useradd with the parameters "home.bogus foouser foopasswd U".

If the command is successful then CtrlClnt returns 0 else it returns a value different from zero.

If the command is a query then the result will be redirected to stdout.

This command line utility allows you to create user accounts (in a file called mailusers.tab) by providing a formatted list of users as a parameter (or by making a formatted text file).

The syntax of the list (or file) is:

domain;username;password;real-name;homepage[NEWLINE]

where the character '#' -if used for the fist column of a line- can be used to comment the entire line.

This utility can be also used to create a random number users, which is useful for me for testing the server performance.

The MkUsers command line parameters are ("{}" enclose the default value if any):

-a numusers = number of users to create in auto-mode

-d domain = domain name in auto-mode

-f inputFile = input file name {stdin}

-u username = radix user name in auto-mode

-r rootdir = mail root path {./}

-s mboxsize = mailbox maximum size {10000}

-i useridbase = base user id {1}

-m = create Maildir boxes (use this command ONLY for Unix!!)

-h = show this help

MkUsers creates the following file structure under the specified root directory: 

If a file called mailusers.tab already exist in the mail root path then MkUsers exits without overwriting the existing copy of the file mailusers.tab. This protects you from accidentaly overwriting your users file when playing inside the real MAIL_ROOT directory.

So if you want to setup the root directory (-r ...) as MAIL_ROOT then you must delete manually the existing file file first (you must know what you are doing).

If you setup the root directory (-r ...) as MAIL_ROOT then you MUST stop XMail before running MkUsers.

Existing files and directories will not be overwritten by MkUsers so you can keep your users database in the formatted text file (or generate it from a database dump for example) and then run MkUsers to create the XMail users structure.

Note: Remember that you have to add new domains in domains.tab file manually.

MkUsers is intended to be used as a bulk-mode utility, not to be used in order to create single users

because CtrlClnt.exe (or any other GUI/Web configuration utility) is far more appropriate.

When building XMail an executable called sendmail.exe is also created. This is a replacement of the sendmail program used mostly on Unix systems. It use the XMail local mail delivery feature to send emails to the server.

The only supported sendmail options are (other options are simply ignored):

-f{mail from} = Set the sender of the email

-F{ext mail from} = Set the extended sender of the email

-t = Extract recipients from the "To:"/"Cc:"/"Bcc:" header tags

The syntax is:

sendmail [-t] [-f...] [-F...] [--input-file fname] [--xinput-file fname] [--rcpt-file fname] [--] recipient ...

The message content is read from the standard input and must be RFC compliant.

The following parameters are XMail extensions expected to be used with mailing lists managers (using

sendmail as a mail list exploder):

--input-file fname = takes the message from the specified file instead of from stdin (RFC format)

--xinput-file fname = takes the message from the specified file instead of from stdin (XMail format)

--rcpt-file fname = adds recipients listed in the specified file (list exploder)

Being RFC compliant means that the message MUST be formatted like this:

[Headers]

NewLine [CR-LF]

Body

So, suppose you have your message in the file "msg.txt", that you are "xmailuser@smartdomain" and

that you want to send this message to "user1@dom1" and "user2@dom2" then the syntax is:

sendmail -fxmailuser@smartdomain user1@dom1 user2@dom2 < msg.txt

or

sendmail -fxmailuser@smartdomain --input-file msg.txt user1@dom1 user2@dom2

Version 0.1 (Alpha-1): 

0.1-001 Linux (FIXED)

SMail threads don't wake up upon a release semaphore issued by SMTP threads.

0.27-001 Windows

Using XMail in a network that uses MS Proxy Server 2.0 will cause XMail to fail sending UDP packets for DNS MX queries (due to a bug of WS2_32.DLL linked to MS Proxy 2.0). I don't know if greater versions of MS Proxy will fix this bug. To makes XMail work in such an environment you can use the "DefaultSMTPGateways" option in the SERVER.TAB file to use smart SMTP hosts. Or better, strip away MS Proxy server and setup a cheap PC running Linux and IP-Masquerading -that will cost exactly $0 and works great. Or use the "SmartDNSHost" option to redirect recursion queries to a smart DNS host which supports TCP and recursion.