Can you explain the Mirapoint message store, for use with selective restore from an image?

Issue

Can you explain the Mirapoint message store, for use with selective restore from image?

Solution

The Mirapoint message store is one file system. All files and directories available for backup and restore are found under /usr/store in MOS 3.x releases (/mira/usr/store in MOS 4.x).

Figure 1 Example: /usr/store Directory Structure

Figure 1 is a high-level view of a possible /usr/store directory structure, showing only a few of the directories. Some store directories are standard on every appliance, and some exist only when the appliance is licensed for the specific feature.

Table 1 /usr/store Directories

The appliance automatically restores everything in the imap and mta directories that is important to the successful completion of a restore.

About the spool Directory

The spool directory contains all user data for the Mirapoint appliance's primary mail domain. Data for users in delegated domains is stored in the corresponding subdirectory under domains (see "About the domains Directory" in this article).

Figure 2 Example: store/spool/user Directory Structure

When a new user is created using the Administration Suite or CLI (User Add user, and Mailbox Add userinbox), the directory for that user is created and stored under spool/user; this is the inbox level directory. For example, if the user John Doe is created as jdoe@example.com, the user directory (inbox) jdoe is created under spool/user, with the pathname /usr/store/spool/user/jdoe.

All mailbox subfolders created automatically for, or manually by, an individual user are stored in directories under the inbox level (for example, Sent, Trash, Junkmail, Draft, etc.).

In most cases, the name for the individual user directory is the same as the name given when created (for example, jdoe@example.com has the directory jdoe). If a user name is created using upper and lowercase characters (jDoe), the uppercase character is converted to lowercase in the user's directory name. For more information on how the Mirapoint message store handles directory names, see "About Directory Names" in this article.

About the Inbox Level Directory

Each user inbox level directory contains numerous files, subdirectories, and the individual message files for that user. Of the files that might appear in the inbox directory, four recur in all directories at all levels (hdr.db, var.db, .rc, and .hc). The remaining files occur only in the inbox directory. The inbox subdirectories are referred to here as level1 directories.

Figure 3 Example: Inbox Level Directory Structure

Table 2 details the recurring and other possible files of individual user inbox level directories.

Table 2 Inbox Level Directory Files

Inbox Level1 Directories

If you are using WebMail, or any email client that uses default subfolders (for example, Sent, Draft, Junkmail, Trash), those mailbox subfolders appear as level1 directories (subdirectories) to the Inbox level. If a user creates his own mailbox subfolders, those are stored as level1 directories as well.

Message Files

Each message file contains exactly one(*) email message. An email message delivered to multiple recipients is stored on disk on the Mirapoint appliance only once, though there is overhead in tracking it in multiple folders for purposes such as NDMP backup.

Message filenames consist of some number of digits, never beginning with zero, followed by a dot "." (for example, 1021.).

About the domains Directory

If the appliance is configured with delegated domains, the user directories and files for those delegated domains are stored in the domains directory.

Each delegated domain (deldom) includes the imap and spool directories. Each individual deldom spool directory contains the same file and directory structure as the spool directory associated with the main mail domain for the appliance (see "About the spool Directory" in this article).

Figure 4 Example: Domain Directory Structure

Directory names are always lowercase. If a user is created and given a name that uses both upper and lowercase letters, the directory name given to that user is only in lowercase characters.For MOS 4.x, instead of having user directories directly under /mira/usr/store/spool/user or /mira/usr/store/domains/domain.example.com/spool/user, there is a hashed structure that holds user directories with a symbolic link for each user pointing to those directories.About Directory Names

About Character Sets

Most computers use ASCII code to represent text, allowing data transfer between computers. ASCII (American Standard Code for Information Interchange) is a code representing English characters as numbers from 0 to 127. It uses 7 bits for each character. The Latin-1 character set is a superset of the ASCII characters recognized by the computer hardware and software. The International Organization for Standards (ISO) developed this standard character set, and both the HTTP and HTML protocols on the World Wide Web are based on this ISO Latin-1 standard. Therefore, to represent non-ASCII characters on a web page, you need to use the corresponding ISO Latin-1 code.

Unicode is used to represent text for non-English languages (for example Greek, Japanese, Chinese, and the like). Unicode is a standard for representing characters as integers. It uses 16 bit, which means it can represent more than 65,000 unique characters. These double-byte character sets cannot be represented by single-bye codes such as ASCII.

UTF-8 and modified UTF-7 (mUTF-7), short for Universal Transformation Format, is a method of converting Unicode characters from 16-bit into 7- or 8-bit characters; mUTF-7 converts Unicode into ASCII for transmission over 7-bit mail systems, and UTF-8 converts Unicode to 8-bit bytes.

Figure 5 Non-Latin-1 Character Conversion Process

When users are created using non-Latin1 characters (such as those used in Asian or European languages), the email client software converts them to mUTF-7 before sending it to the Mirapoint appliance. The Mirapoint applianceconverts the mUTF-7 characters to UTF-8, then stores the content of the folders and messages in the message store. When the DMA performs a backup of the Mirapoint message store, the Mirapoint appliance produces the NDMP file history and sends it to the DMA that then creates a file index (see Figure 5). When the data management application (DMA) browser presents the file index, one of the items following occurs:

• It understands the converted non-Latin-1 characters, converts them back to their original condition and displays them on screen.
• It understands the converted non-Latin-1 characters, converts them back to their original condition, but cannot display them on the screen.
• It does not understand the converted non-Latin-1 characters, cannot convert them back to their original condition, and cannot display them on screen.

When the DMA cannot display non-Latin1 character sets, or cannot understand the converted non-Latin-1 characters it receives in the data stream from the Mirapoint appliance, they might display in any of the following ways:

• A string of random ASCII characters (for example abc/de2f)
• A string of random ASCII characters proceeded by an ampersand, then ending with a dash (for example &bc/de2f-)
• A string of random ASCII characters proceeded by and/or ending with black rectangles (for example zzzabc/de2fzzzz)
• A string of random ASCII characters proceeded by and/or ending with question marks (for example ???abc/de2f???)
• A sequence of characters similar to the non-latin1 character set used in European languages where the ASCII character is followed by its diacritic mark (for example % displays as %)

The DMAs that can understand and display converted non-Latin-1 characters in their original condition can be used with Mirapoint's selective restore from image feature.

DMAs that cannot understand the converted non-Latin-1 characters or that do not convert them back to their original form and display them properly cannot be used with the selective restore from image feature. On these DMAs, the difficulty in identifying which directory corresponds to a given user hinders the ability to complete a successful selective restore. Mirapoint recommendeds performing a full or incremental image restore rather than attempting a selective restore.