How do I use LDIF for message routing and switching?

Issue

How do I use LDIF for message routing and switching?

Solution

Installations without an LDAP server can use locally managed LDIF (LDAP Data Interchange Format) files to manage information about message routing and mailbox switching (but not user authentication). Even with LDAP services, LDIF can be useful for domain-based routing.

This article describes how an LDIF file interacts with the Mirapoint or RazorGate appliance. Sections below describe message routing, new Administration Protocol commands, and interaction with the user database, distribution lists, and LDAP server.

Overview of Flatfile Message Routing

Mirapoint or RazorGate appliances accept an LDIF file that contains routing information used by the SMTP service, POP and IMAP front-end processors, and HTTP-based administration or Webmail redirect agents. These proxies use information from your LDIF input to obtain the routing address and mailhost for a given user.

When asked for the routing address or mail host of a user, the appliances first checks to see if that user exists in the LDIF flatfile database, whether or not LDAP is configured. If LDAP service is configured, the appliances then contacts the directory server(s) to retrieve the user's routing address or mail host.

The mailhost may be entered as a hostname or in dotted quad notation (IP address). If there is a mailhost: field in a record, it specifies the target machine for SMTP routing, POP/IMAP switching, and HTTP redirects. If there is no mailhost: field, but there is a mailroutingaddress: field, the target machine is extracted from the domain in that field, and the target user name is rewritten. If both fields are present, then the target machine is defined by the mailhost: field, however the username gets rewritten according with the mailroutingaddress rather than being preserved as is the case for mailhost: only.

Domain-based Routing

If the value of a mail: record is of the form @domain with no user specified, the entry is used as a routing entry for all users in that domain who do not have an explicit entry in the LDIF file. It is possible to direct all mail to all users in a given domain to a specific host by specifying that host as the mailhost. It is also possible to direct all mail to any user in a domain (but not subdomains) to a single user address by specifying that address as the mailroutingaddress.

When configuring the LDIF flatfile, you can tell the Mirapoint or RazorGate appliance to replace the current flatfile database with the one you are currently providing, or apply a set of deltas to the existing flatfile database with new data you are providing.

The Ldap Set Ldif command replaces the flatfile database and applies edits to the current database. The contents of the first line of the input file determine whether the LDIF data is to replace the existing database, or modify the existing database.

To add entries to the existing database, the LDIF input starts with a line containing <add>. For example:

<add>dn: uid=u1,dc=example,dc=com

mail: u1@example.com

mailroutingaddress: u1@h1.example.com

dn: uid=u2,dc=example,dc=com

mail: u2@example.com

mailroutingaddress: u2@h2.example.com

mailhost: mailhub.example.com

To delete entries from the existing database, the LDIF input starts with a line containing <delete>. For example:

<delete>dn: uid=u3,dc=example,dc=com

mail: u3@example.com

mailroutingaddress: u3@h1.example.com

dn: uid=u4,dc=example,dc=com

mail: u4@example.com

mailroutingaddress: u4@h2.example.com

To replace the existing flatfile database with a new database, the LDIF input should not start with a <add> or <delete> line. For example:

dn: uid=u6,dc=example,dc=com

mail: u6@example.com

mailroutingaddress: u6@h2.example.com

mailhost: mailhub.example.com

dn: dc=example,dc=com

mail: @example.com

mailhost: mailhub.example.com

It is expected that the administrators will update the Mirapoint or RazorGate appliance often, and less frequently (say once a week) completely synchronize the flatfile database by totally replacing it.

Limitations

Interpretation of the LDIF file is not very forgiving about formatting. The LDIF file is often computer-generated rather than hand-generated in a text editor. Consequently, where a space is expected, there must be one and only one space. LDIF records are delimited by one and only one blank line, and that line must not contain white space created by pressing the Space bar or Tab.

Each LDIF record must contain a mail: field. If the record is being added to the database, it must contain a mailroutingaddress: or mailhost: field. Records that are being deleted may contain only the mail: field.

When you retrieve the LDIF value from the Mirapoint or RazorGate appliance, it contains the last flatfile database that was given to the Mirapoint or RazorGate appliance, with all the addition and deletion records appended to the end of the file.

NOTE: The Mirapoint or RazorGate appliance does not really contain the truth about routing information, but it is instead merely a copy of routing information from some other source.

Fields using the base 64 notation, where the attribute name delimiter is a double colon (::) instead of a single colon (:), are not supported. Your LDIF file can contain these entries, but they are not recognized if used in the mail:, mailhost:, or mailroutingaddress: fields.

Continuation lines are supported. The continuation line may start with a space or tab character, but there must be one and only one of these characters starting the line.

For size constraints, it is expected that there will be no more than one million entries handled by the flatfile mechanism. There is not an explicit limit, but each one million records consumes about 50 MB of memory and requires 40 seconds of startup time to parse.

The LDIF flatfile information is not saved by any Mirapoint backup method.

Error Handling

Before the appliance incorporates a new LDIF file into the database, it must successfully parse the entire LDIF file (given the constraints above). If the file fails to parse correctly, then the contents of the database will be the same as before the LDIF was provided. If the LDIF file is to replace the existing data, then the existing database is overwritten with the new contents. If the new LDIF data is delta information, these are appended to the existing database, and in-memory data are overwritten to reflect the LDIF changes.

New Protocol Commands for LDIF

To support the LDIF flatfile support, the Mirapoint administration protocol was extended with the following commands:

tag Ladp Set Ldif Ldif-values:

tag OK Completed

tag Ldap Get Ldif

tag {500000}

...

tag OK Completed

The Ldif-value is an IMAP literal string that contains a modified LDIF file format. Sending an LDIF file exported from an existing LDAP server usually works, although most of the attribute-value pairs are ignored.

The Ldif-value can be a vanilla LDIF file, or it can be an LDIF file that begins with a command (either <add> or <delete>). If the Ldif-value is a vanilla LDIF file, the system's current Ldif-value is replaced by the new LDIF-value. If the Ldif-value begins with the Add or Delete command, the LDIF records are added or deleted to the system's current value. Here is an example Ldif-value:

<add>

dn: uid=fred,dc=example,dc=com

mail: fred@example.com

mailroutingaddress: fred@host1.example.com

dn: uid=barney,dc=example,dc=com

mail: barney@example.com

mailroutingaddress: barney@host2.example.com

mailhost: mailhub.example.com

Not all fields of the LDIF record are used. In fact, only the mail, mailhost, and mailroutingaddress attributes are used; other attributes are ignored. Since the distinguished name (dn:) is ignored, the mail attribute is used as the record's unique key. This means that if you have two users with the same e-mail address, the last record with this e-mail address will win over previous records.

Interaction of Flatfile with Other Components

The following section describes how the flatfile mechanism interacts with the IMAP/POP front-end processors, the inbound message routers, and the SMTP authentication capability of the SMTP relayer.

Overview

The table below describes the database mechanism (LDAP or flatfile) versus a particular service:

01 LOGIN Administrator
The user database is modified with the User command. For example, a typical session to add a user could look something like the following (you can try this by telneting to port 10143 on the Mirapoint or RazorGate appliance):

admin

01 23423423

01 OK User logged in

02 User Add "fred" "fredspassword" "Fred Flintstone"

02 OK Completed

03 User Add "barney" "barneyspassword" "Barney Rubble"

03 OK Completed

04 User List "" "" ""

* 04 "barney" "Barney Rubble"

* 04 "fred" "Fred Flintstone"

04 OK Completed

To use Smtp Auth on the IMR or OMR, all the users in the entire system must be added to each router where SMTP authentication is expected to function.

Flatfile

The flatfile is modified using the Ldap command. A typical session to set the flatfile looks as follows:

01 LOGIN Administrator admin

01 23423423

01 OK User logged in

02 Ldap Set Ldif {50234523}

dn: uid=fred,dc=bedrock,dc=com

mail: fred@bedrock.com

mailroutingaddress: fred@h1.bedrock.com

dn: uid=barney,dc=bedrock,dc=com

mail: barney@bedrock.com

mailroutingaddress: barney@h2.bedrock.com

mailhost: mailserver.bedrock.com

02 OK Completed

The flatfile can be incrementally edited using the same Ldap Set Ldif command. To trigger an addition, the <add> field needs to be the first line of the LDIF file. To trigger a deletion, the <delete> field needs to be the first line of the LDIF file. An example:

03 Ldap Set Ldir

{1342}<add>dn: uid=wilma,dc=bedrock,dc=com

mail: wilma@bedrock.com

mailroutingaddress: wilma@h3.bedrock.com

03 OK Completed

04 Ldat Set Ldif

{2043}<delete>

dn: uid=barney,dc=bedrock,dc=com

mail: barney@bedrock.com

mailroutingaddress: barney@h2.bedrock.com

mailhost: mailserver.bedrock.com

04 OK Completed

LDAP

LDAP is controlled with the Ldap command. Use the Ldap Add command to add LDAP server names. Use the Ldap Setquery commands to supply the baseDN, query, and attribute names to be used for the mailhost and mailroutingaddress properties used by the front-end processors, mail routers, and authentication. For example:

03 Ldap Add "ldap.example.com"

03 OK Completed

04 Ldap Setquery "user:mailhost" "$(dcmap)""(|(mail=$(login))(mailaltaddress=$(login)))""mailhost" ""

04 Completed

05 Ldap Setquery "user:routingaddr" "$(dcmap)""(|(mail=$(login))(mailaltaddress=$(login)))""mailroutingaddress"""

05 Completed

06 Ldap Setquery "user:publishedname" "$(dcmap)""(|(mail=$(login))(mailaltaddress=$(login)))""mail" ""

06 Completed