Considerations for Setting Up IMAP and POP3 Servers in PMDF on OpenVMS

What resources the servers need will depend on:

  • total number of users, number of users you want to support at any time,
  • whether you have POP only, (POPSTORE only, POPSTORE and native VMS mail or VMS mail only) IMAP only or both POP and IMAP, and how many users of each.
  • for IMAP users, the number of folders they keep open. (depending on the client used)
  • size of users' mailboxes, do they get lots of messages with multi-megabytes of attachments?

Quotas

You can control the resource usage by setting the quotas for the server processes. Process quotas for the IMAP/POP servers are taken from: (most important ones are PGFLQUOTA, WSQUOTA, WSEXTENT)

  1. defaults within the Dispatcher,
    PGFLQUOTA=81920
    ASTLM=500
    BIOLM=100
    BYTLM=200000
    DIOLM=100
    TQELM=500
            
    these are overridden by
  2. what you specified in the Dispatcher configuration (in dispatcher_mailbox_servers.cnf under each service)
    PGFLQUOTA=n
    WSQUOTA=n
    WSEXTENT=n
            
  1. the Dispatcher specifies the quotas used to create the process, and these numbers are further adjusted by

  2. system minimums and system defaults in SYSGEN. relevant SYSGEN parameters are:
    PQL_MASTLM, PQL_DASTLM
    PQL_MBIOLM, PQL_DBIOLM
    PQL_MBYTLM, PQL_DBYTLM
    PQL_MCPULM, PQL_DCPULM
    PQL_MDIOLM, PQL_DDIOLM
    PQL_MFILLM, PQL_DFILLM
    PQL_MTQELM, PQL_DTQELM
    PQL_MENQLM, PQL_DENQLM
    PQL_MJTQUOTA, PQL_DJTQUOTA
    PQL_MPGFLQUOTA, PQL_DPGFLQUOTA
    PQL_MWSDEFAULT, PQL_DWSDEFAULT
    PQL_MWSQUOTA, PQL_DWSQUOTA
    PQL_MWSEXTENT, PQL_DWSEXTENT (limited by WSMAX)
    VIRTUALPAGECNT
    CHANNELCNT (suggest sum(MAX_CONNS)+sum(MAX_CONNS*MAX_PROCS)+ 
    30 if using UCX.
    Multinet or TCPware sites should set UCX_HOLD=0 and leave
     CHANNELCNT unchanged
    from your currently operating value) 
              
  3. quota of the PMDF or SYSTEM account are NOT used!

    Relevant Parameters in the Dispatcher Configuration

    • MIN_PROCS (default 1) MIN_PROCS specifies the minimum number of server processes which will be resident. You can set this to 0 so no process slot will be taken up if no connections were ever made to it.
    • MAX_PROCS (default was 5, later based on answer to the configuration utility) This sets the absolute maximum number of server processes for the service.
    • MAX_SHUTDOWN (default 2) This is the maximum number of servers which can be in the "shutting down" state due to MAX_LIFE_TIME or MAX_LIFE_CONNS so the service will not be completely unavailable. You should adjust this to be about half of MAX_PROCS when you are not using the default MAX_PROCS for the service.
    • MIN_CONNS (default 3)
    • MAX_CONNS (default 5) For native VMS MAIL store with either POP or IMAP, the maximum is 31 due to callable mail limits. MAX_CONNS*MAX_PROCS determines how many total concurrent connections you can have for the service. For POP, MAX_CONNS*MAX_PROCS would be the total number of active users you can support at any instant, for IMAP, this is the total number of connections, since each user may have more than one connections, the number of users can not be determined exactly.
    • MAX_LIFE_TIME (default to Dispatcher default in dispatcher.cnf) This sets the maximum elapsed time (NOT CPU time) a server process can stay active before being told to shutdown. You probably don't want this to be too low.
    • MAX_LIFE_CONNS (default 100) You should not set MAX_LIFE_CONNS arbituarily large, so server processes can be retired after it had this number of connections. This is used to avoid running into memory leaks in callable mail.
    • MAX_IDLE_TIME (default to Dispatcher default in dispatcher.cnf)
    • The MAX_IDLE_TIME should not be too large on a resource-poor but active system. Suggest 300 seconds. Large number means an idle server (not having any connections) will stay around longer and take up system resources like physical memory, pagefile space and process slots. A smaller number means more process creation but also ensures that resources are returned promptly.

    • PGFLQUOTA (default to 150000)
    • BYTLM

      Suggest using a number bigger than 5120*MAX_CONNS + 20480

    • FILLM

      FILLM should be set to be at least 5*MAX_CONNS

    • ENQLM

      should be at least 10*MAX_CONNS.

    • WSQUOTA
    • WSEXTENT

    System Resources:

    • number of processes
    • The number of process slots you have configured for the entire system (MAXPROCESSCNT) must be large enough to accomodate the number of processes you configured for the POP or IMAP service plus everything else you already have.

    • number of threads per server process

      This is determinted by your MAX_CONNS configuration parameter for the service in pmdf_table:dispatcher_mailbox_servers.cnf file. More threads means more VIRTUALPAGECNT may be needed and higher process quotas will be needed.

    • pagefile

      you need to have enough real pagefile space to back up the pagefile usage of the server processes you have configured. e.g. if you have configured maximum of 10 POP3 servers, and each has a PGFLQUOTA of 150000, then you could at a worst case need 150000*10 more pages in your pagefiles. (worst case because it is not likely that all the virtual memory used need to be actually written to the pagefile). You should monitor the usage on your system with the PMDF PROCESS/MEMORY command (available since PMDF 5.2) periodically.

    • physical memory

      POP3 and IMAP servers are memory intensive. If you only have 64MB of real memory, then the performance of 10 concurrent IMAP servers with 5 connections each serving an average of 10 MB of mail messages (10MB*5*10=500MB just for the message data, almost twice as much is actually needed due to internal buffering) is going to cause a lot of paging. (and 10 MB is not much nowadays).

      We recommend a MINIMUM of 1 MB of physical memory for each active POP3 user connections at the same time. So if you think you will have 100 concurrent POP3 users at any instant out of a population of 1000, then you should have AT LEAST 100 MB of physical memory in addition to whatever else you need on the system.

      For IMAP users, the requirement will be more. IMAP users tend to connect and stay connected for a long time. It is possible for a connection to stay up ALL day, or even ALL WEEK. Since each IMAP user may have more than one connections to the server, the number of IMAP users you can serve may be smaller than the number of connections. Let's say you have configured MAX_CONNS=10 and MAX_PROCS=10, that is 10*10=100 concurrent connections, this could mean 50 or fewer real users depending on client implementations and usage patterns.

    • working set quota/extent

      Do not have more WSEXTENT/WSQUOTA for the servers than you have physical memory to support them comfortably. But if you have the memory, do give them a large WSEXTENT.



    Notes

    • POP/IMAP servers are more memory and I/O intensive than CPU intensive.
    • The disk(s) mail reside on should be fast! We recommend you use RAID arrays and spread out onto multiple spindles.
    • Virtual memory is cheaper than other resources.
    • Disk space for pagefile is cheap. Be realistic and generous.
    • see also how to not leave mail on server to decrease work for the POP server.
    • You can also control how frequently the POP users can log in with the MIN_LOGIN_INTERVAL option.

    A rough calculation of how many POP3 users you can support

    Assumptions for this example (your assumptions are probably different):

    • Total user population of 10000, meaning you have this many accounts.
    • Active population of 1000. The active users are the ones who could be running a POP3 client at a given instant. If you are a service provider, you would use the number of dialup modems. You may have to estimate.
    • They are VMS MAIL users.
    • Each user has configured the POP3 client to check mail every 10 minutes.
    • Each user gets an average of 10 messages per day, with a total size of 2 MB for the day.

    This means ON AVERAGE, in every hour, each user will have tried to connect to the server 6 times, for a total of 6000 connection attempts from your active population. 6000 per hour = 100 per minute = 1.66 per second. Let's round it to 2 per second. This is a pretty low demand on your system, meaning ON AVERAGE, you will probably only have 2 concurrent connections at any one time. Even with a VAXstation, you can probably support this many connection easily. If the users check mail more frequently, then obviously the load on your system will increase proportionally.

    You can have a worst case when 1000 users are all trying to connect at the exact same time, you have the choice of having the connections refused or you can try to size up your system to support it. If you were to support this case, you would have to have MAX_CONNS*MAX_PROCS=1000, let's say you have MAX_CONNS=20, then you would need MAX_PROCS=50.

    Proceeding on the worst case, assuming all the users are getting their 2MB of messages at this time, then each server process will need a minimum of 2*20*2 MB = 80 MB (or 163840 512-byte pagelets) of virtual memory for message data plus some small amount for per thread data, (2* because we memory map the message itself, but also need another buffer to format the message to have CRLF's at the end of each line, when not all these messages need to be served at once, the requirement will be lower because some memory will be reused depending on size, fragmentation of memory and timing of when the memory is needed.)

    So if we round the requirement up to about 180000 pages, then that means the PGFLQUOTA for the server process will need to be at least 180000 to serve this scenario. With 50 such servers needing the space at once (not likely), you would potentially need 50*180000/2=4500000 blocks of pagefile space (about 2.3GB). (divide by 2 because half the buffer is memory mapped and does not require space in the page file). I stress the word "potentially" because some pages will be in memory and will not need to be written to the pagefile. And if you have 2 GB of real memory for this (remember the MINIMUM of 1MB per connection? that's 1GB for your 1000 connections), and sized up the WSEXTENT properly, then you will not likely to need the pagefile space.

    In terms of physical memory usage, if your WSEXTENT for the servers are set too high, then all memory could be used up in this case.

    So what if you don't have the resource? Don't set MAX_PROCS and MAX_CONNS that high, and if users all happen to connect at once, they will be denied service. Typically, they will just try again, and succeed.

    A rough calculation of how many IMAP users you can support

    With IMAP, the story is a little different, basically because the usage characteristics are different. IMAP connections tend to stay around for a while, and when they do, it is somewhat like having that many interactive users all using mail at the same time. This is worse than interactive use because when you are using VMS MAIL or PMDF MAIL you normally only have ONE folder open, with PMDF PINE, you can have two folders open (the INBOX, or NEWMAIL folder, and another one if you so choose). With IMAP, if a client opens a connection to EACH folder you happen to click on, then the potential resource usage on the server could be a lot higher. Different IMAP clients behave differently, and you would have to find out what your client does.

    The resource used by each active IMAP connection is pretty much the same as what's needed for an active POP3 connection (when the number of messages are the same). To calculate the number of users you can support, you need to divide the connections by the number of average connections per user, I would probably divide it by 2 to start with.

    So using our worst case POP3 scenario of 1000 connections, with the same resource, you would be able to support 500 ACTIVE IMAP users. If 1/10 of your population have an IMAP client running at once, you will only be able to support a population of 5000. If half of your population will have an active client running, then you will only be able to support 1000 users.

    See http://www.process.com/techsupport/bytlm.html for a BYTLM calculator.



Search: