We've been experimenting with a groupware server, and we've identified a set of needs that we have for a small business server, to form a pattern for departmental and general purpose servers inside our organisation. For a standard office, we need the following:

  • Email in and out
  • Calendaring
  • Address books (personal, team and organisation-wide);
  • File sharing
  • Printer management
  • Non-technical user administration frontend

Underlying all this, we want

  • Centralised user management database (ie. LDAP)

How do we get there?

Technology assessment

It's worth pointing out that none of the open source tools that call themselves "groupware servers" really cover all of these things. Kolab is the least bad, but does not provide the file sharing. Since it uses LDAP and there is the ability to drive Samba via LDAP, at least in theory you can get the two using the same LDAP server. In practice, we haven't been able to manage it, not least because Kolab is a swine to install.

This highlights another requirement: we want this all nicely integrated, but we don't want it too integrated that it all falls apart when you try to customize it. This is why we're not using SME server. It's all so tightly integrated that to get it to do something that it hasn't been designed for (for instance, putting everything into LDAP) the pain of customization is greater than the benefit of using an integrated system. Similarly, Kolab, and the other "groupware servers", are pretty good in that they use standard open source tools for most of their functionality. So the feeling is, since we're going to customize it all already, there is probably very little benefit in using one of these servers, and instead we just use all the individual components and tie them together ourselves.

We haven't looked into the possibility of a Microsoft solution in great detail, except to note that (a) Microsoft servers are too expensive for our users outside the US and Europe and (b) most IT professionals who have come into the organisation are familiar with Linux systems, particularly Debian and Red Hat.

Mail

Mail means mail in, SMTP, and IMAP.

For mail in, we need to fetch mail from external POP3 accounts, defined by the user. We'd like to store the user's POP3 details in the LDAP repository as well, which means that we may need to create our own schema to store that information, and then write a wee Perl script which walks the LDAP tree and calls fetchmail with each user's details. It's worth checking whether or not we can repurpose the Kolab schema to store mail information, regardless of whether or not we actually deploy Kolab. (Have checked: we can't) Otherwise we need a user interface to modify the POP3 details.

For SMTP, it doesn't really matter which one we use. We don't need any particular SMTP server; for instance we don't need to look-up the users of incoming mail in LDAP, because we know we're only fetching mail for users in LDAP! We do want antispam and antivirus. There are several players in this market, Mailscanner being the main one, and that requires sendmail or exim. However, you can also use postfix with ClamAV and Spamassassin. Heck, you can probably even use qmail if you want. My (Simon's) personal recommendation is exim and Mailscanner, because they're very well integrated with Ubuntu, and because I know them the best.

For IMAP, there's a similar array of choices. We would like to do shared folders, both for sharing of information and possibly for calendaring and address books. Courier and Cyrus both support shared folders. Courier's support of shared folders is detailed at http://www.inter7.com/courierimap/README.sharedfolders.html - it requires lots of symlinks. It's not clear how we'd create a user interface to that, so the current feeling is to go with Cyrus and websieve which is a package to allow user configuration of shared folders (as well as filter scripts) over the web. Since we're going with Cyrus, we must not forget to configure our mail server with LMTP delivery to get the mail into the Cyrus store.

Calendaring

Calendaring on Open Source has always been a big lacuna. Really, there just isn't anything there yet. WebCAL/CalDAV might be an option in a few years time but it's not there yet. For now, you basically have to treat calendaring as file sharing of .ics files. For client access, we're going with Lightning (http://wiki.mozilla.org/Calendar:Lightning) since we know that everyone has Thunderbird anyway. In terms of publishing calendars, we think DAV is the future-proof way forward (things seem to be moving to CalDAV, and we need to support DAV for file sharing, SVN etc.).

Sharing of free/busy information will need to be set up for a DAV-based solution.

The only realistic alternative is using Kolab-style calendar sharing - where calendars are maintained in special shared IMAP folders. This doesn't require Kolab itself, but requires synchronisation on both Thunderbird (Sync Kolab extension) and Outlook (Toltec connector). Accessing Free/Busy status is also likely to be a problem here.

Address Books

Ideally we'd like to do this via LDAP. The problem is that LDAP directory servers are generally read-only on mail clients. Novell have a directory server called miru, and there's a Thunderbird extension ("AddTo miru directory server") which adds contacts to an LDAP directory. It does it by POSTing a specially formatted XML request to a PHP script included in the miru server. We looked at all this, and thought it wasn't very impressive.

So our standard address book system will actually be CiviCRM. It's a nice drupal-based system which is designed for the sort of community/activist/fundraising uses we have. We will use LDAP's SQL backend to export the CiviCRM contacts info as an LDAP directory. This means that people can edit their contacts over the web. We will write Thunderbird extensions to add contacts to CiviCRM from the mailer.

Mark says: We have at least 3 possible address books

  • the organisation-wide email address book accessible by LDAP and hosted in Australia. These are only personal addresses on the organisation-wide email system and cannot be modified. Functional addresses are supposed to be added to this system in future... however no timescales are available.
  • UK addresses - both functional and individual - corresponding to accounts in the LDAP database.
  • Address book(s) containing external contacts. This is done in CiviCRM.

Shares

Yes, Samba, obviously, with the LDAP integration we've already done. But also DAV so that we can authenticate external users using server and client SSL certs in Apache.

Perhaps the usershares functionality in Samba >= 3.0.23 will allow end-users to manage their own shares - though DAV shares won't be able to benefit from this.

DAV is hard: because the web server runs as www-data, and needs to read and write to directories owned by all sorts of users. See http://ask.slashdot.org/article.pl?sid=03/07/25/0045222

Our plan of attack will be: have a separate Apache instance just running mod_auth_ldap (included in the Apache2 package in Ubuntu) and mod_dav. That'll run as www-data, but www-data will be in everyone's group. Both Apache and Samba will need to be run with umask 002, so that files created in one can be overwritten in the other. Additionally, we wrote the Apache::AuthzUnix? module which provides Unix-style authorization from the point of view of the logged-in user rather than from www-data's point of view.

As far as clients, http://support.microsoft.com/kb/307560 suggests that the Windows client can speak SSL with client certs, even if Office can't open the files directly!

Printing

Samba.

User management frontend

Our user management frontend needs to be easy enough for non-technical users to use. It needs to modify LDAP, create Samba users (home directories, etc.) and set up the IMAP mail store.

We've come up with three different "levels" of user management, depending on how much you need to know about what you're doing.

  • The Windows 2000 user account manager program will work for adding users, deleting them, changing password (check this).
  • A web-based system for adding and deleting users and self-service interface to changing passwords - this is just a CGI front-end to smbldap-tools.
  • LDAP Account Manager (http://lam.sourceforge.net/) for low-level LDAP manipulation.

We have modified the smbldap-tools useradd script such that when a new user is created, a user SSL cert is created, made into a PKCS12 bundle with a generated password, and the bundle placed in their home directory. Also the web server user is added to the new user's group.

Questions we have answered

  • Management of common samba shares - done, we'll just make 'em users.
  • There is also a very nasty interaction between ldapauth and civicrm: users need valid email addresses to enter CiviCRM, but ldapauth doesn't fill out email addresses for users it creates. So the user has to edit their profile before they're allowed to get to CiviCRM, which is suboptimal. Hackaround: 
    CREATE TRIGGER email_hack BEFORE INSERT ON users FOR EACH ROW SET NEW.mail = concat(NEW.name, "@whatever");

Questions we haven't yet answered

  • Outlook users.
  • Drupal gets mad if you turn on modules behind its back, so we can't automate that last part
    • User must turn on civicrm and ldapauth modules separately
    • Then configure resource URL in civicrm config
  • Cyrus administrator user should be admin
  • How do groups get modified? - more specifically, how does the User Administrator (or end-user) enable other users to access a share by adding them to its group (perhaps the usershares mechanism?

Where to go from here

Simon's todo list...

  • Finish web frontend to smbldap-utils - Administrator can make changes but other users cannot; something wrong with the user Samba is becoming to call the scripts?
    • Also, adding users to groups should be in there.
  • fetchmail or similar
  • MUST fix the Drupal module setup thingy