War FTP Daemon™

This document is published by Jarle Aase, Jgaa’s Internet (http://www.jgaa.com)

Copyright © 1998 by Jarle Aase.

Email:jgaa@jgaa.com

* * *

Jarle Aase is the author of the War FTP Daemon. He originally wrote 90% of the source code. The last 10-% is modified from public available source code.

Part of the source code is copyright 1989, 1993, 1994 "The Regents of the University of California. All rights reserved.".

Part of the source code is copyright "Sun Microsystems, Inc"

Part of the source code is copyright "1991, 1992, 1993 Free Software Foundation, Inc." after the GNU public licensing policy.

War FTP daemon™ is a trademark of Jarle Aase.

I started writing War FTP daemon back in November 1995, after realizing that there was no decent freeware FTP server for Windows. In fact, at that time there was no freeware FTP server at all, - only a number of crippled shareware servers, that really didn’t impress me too much. When the server was released about half a year later, it became very popular, and got a large number of very impressive reviews. One year later, the War FTP Daemon was generally regarded as the #1 FTP server for Windows. When reviewing other servers, the War FTP daemon is usually used as a reference, and the only scores given to other servers these days are simplicity. War FTP daemon obviously has all imaginable features, and hence, is a bit tricky to use for some users/reviewers. (Well – there is room for improvements, as you will see.)

In October 1996 I started to work on War FTP Daemon version 2. In December the source code for the alpha (pre-release) version was released, - and I continued to work on the code in order to release a beta. One year after I started on version 2, I decided to discontinue the development. The code was growing too complex, and the design had some major drawbacks. I made some research on a new design, and came up with some brilliant ideas for version 3. The time frame for version 3 was at least one year, so I decided to release a version 1.70, partially based on the design of the original server, and partially on the ideas for version 3. As I worked on 1.70, more and more code was written from scratch, or stolen from version 2 (the design had drawbacks, but after 1 year of development, version 2 indeed had some very useful and well debugged source code modules). Version 1.70 is in fact a bright new server, as almost no source code is left from version 1.66.

You might ask, what are the main changes since 1.66? Well – first of all – the server module and the user interface is split into different programs. The server is written using portable code, code that will soon be ported to Linux/Unix. The user interface is built on top of MFC – providing a state of the art Windows95/98 user interface. Most of the problems reported with previous versions have been related to users having problems figuring out how to configure the server correctly. When I designed the new user interface, I tried to meet the requirements of the most sophisticated user needs, while also making the configuration an easy task for novice users. The server code is also more robust, and has intelligent interpreting/error checking of configuration options. The user interface communicate with the server using ONC RPC (former SUN RPC), which is a well-established platform-independent protocol. The use of ONC RPC makes it possible for the server and the user interface to run on different machines, using different operating systems and even different microprocessor families (Intel/Alpha/Motorola). A Web server running on an enterprise Unix server can easily communicate with the War FTP Daemon, running on an NT server – just to mention one big benefit. The imagination of the users (and the skills of their programmers/software development departments) is the only thing that limits the possibilities with this new design.

The specs of 1.70 looks very much like the specs of version 2 – what’s the difference between these versions? The main differences are that 1.70, from the outside, looks more like 1.66 than 2.0. 1.70 also lacks the plugin API interface promised for version 2. On the inside, 1.70 looks very much like version 2.

Ok – the server is very neat, flexible and easy – but to whom is it targeted? The only answer to that question is - to you! The server is scalable, making it the best choice for any FTP server need, no matter if you just want to share files between your work and home PC, or run a large-scale FTP server on Internet. The server is designed to meet any imaginable need’s The only drawback for professional use is that the CPU/memory usage is a bit higher than the simplest competitors, such a MS FTP service or wftpd. But hardware is cheap – and this server takes full advantage of multi-CPU hardware for the largest imaginable FTP servers. While 1.66 was developed on a 10mbit network, and is unable to provide much more than 1 MB/sec throughput, 1.70 is developed on a 100mbit network (using a dual Pentium II motherboard on the development PC), and is only limited by the networking hardware in the machine!

When I told my friends that I was going to make a new FTP server, they asked me why? What would I gain? I already made the best FTP server ever written. There was no way I could get better ratings, as version 1.x already had top ratings on all major software sites on the net! To tell you the truth – the only reason for the top-ratings is that the other FTP servers are crap! War FTP daemon 1.x actually sucks, as it have major design drawbacks, poor performance, serious bugs, and even lack the possibility to access the maintenance display when the server runs as a NT service! When I first started my goal was to make the first decent freeware FTP server for Windows. My goal with 1.70, and later on version 3, is perfection. My goal is not just to make the best FTP server (that’s no big deal with today’s competitors), - but making the best Internet server ever. That means writing c++ code that is poetry, taking maximum advantage of the hardware where the server is running, and being compatible with all related software, such as FTP clients, operating systems, and administrative tools.

At last – why did 1.70 take so long? Well – there are several reasons. I changed my mind about the scale of the changes. I also had some periods, lasting from weeks to months, where I was unable to do any coding at all due to personal problems. (I was the victim of a false conviction back in 1994. Sometimes the rage against the society and the so called ‘justice system’ is so strong that all I can focus on is figuring out very painful ways to kill judges and police officers.) I also wanted to introduce some new features in 1.70. After almost two years without any major improvements, I needed to take the time to test and debug as much as possible.

What new features can be expected from the War FTP daemon? There are limitations to the FTP protocol. In 1997 I tried to make some suggestions to the FTP working group in regards of the next version of the FTP protocol. I was however pretty much ignored in that forum; something I found strange, as the War FTP daemon is among the most used FTP servers in the world. Anyway, I decided to work on my own server, rather than challenging the egos of people I’ve never heard about before, and just implement the new FTP specs in the server when (or rather if) they ever reach to a conclusion. War FTP daemon will be 100% compliant with the FTP protocol specification, but I will add my own new features to the protocol, that can be enabled by any FTP clients that support them. Since this server is so widely used, I hope that FTP client writers will take advantage of the extensions I make, such as the interactive chat protocol, enhanced directory listings, asynchronous response messages, encryption, and enhanced file information options. In the end, it’s up to the users do decide what FTP clients, and what features, they want.

There are however a small number of significant programs, that leads the way. Among FTP servers, you have wu-ftpd for Linux/Unix – probably the mostly used FTP server in the world, and wftpd for Windows. Wftpd was the first Winsock FTP server ever, and the author, Alun Jones, also helped giving birth to other ftp servers, such as serv-u, and has always been very active in the Winsock newsgroups, helping newbie programmers (and also the more experienced ones) out. Alun Jones is btw, the only programmer ever that have received any significant part of the pre 1.70 War-ftpd source code.

Among FTP clients for Unix, you have the original ftp client, which is rather primitive, and ncftp, which is my favorite command line FTP client. Ncftp is also available for 32 bit DOS (DOS running from Windows95 or NT). Among Windows FTP clients there are two very significant (and popular) choices, ws-ftp written by John A. Junod (early versions are available with full source code), and cute-ftp, written by Alex Kunadze (both are crippleware/shareware). Before I wrote my own FTP client (which was really not ready for mass distribution yet), Cute Ftp was my personal choice. Among the more fancy Windows clients, I could mention FTP explorer.

I take all the usual precautions, and I am not responsible for any damage you might experience using my software.

I do, however take my software seriously. There are no backdoors, traps, bombs, viruses or any other shit put into the software. My computers run anti-virus programs and all versions, released as well as internal, are scanned for viruses. If serious problems regarding the security is reported, I usually post a patch very soon, and broadcast information about the patch in mailing lists and in the alt.comp.jgaa newsgroup.

The quality of the software is at a commercial level, and several large, worldwide corporations, use the software. When you use the War FTP daemon, you can expect bulletproof security, the very best performance, and reliable operation. If you experience problems, there are free as well as commercial support available. No company back the software, but I do, and I don’t have to take commercial considerations. If I need to spend two weeks in order to implement a neat feature that no one ever thought about, and even fewer will need, that’s what I’ll do. My motivation is not money, it’s the satisfaction of making something that is of world class quality, and the satisfaction of being the very best in the world at what I do. I’m not a very skilled programmer, I just use my imagination and take the time I need to get the job done.

The current version lacks the final polishing of a normal application and there are some steps to take in order to make the server work.

First of all, there is no installation program. To install the server, unzip the distribution archive to an empty directory. If you want to upgrade from a previous version if War FTP Daemon, copy FtpDaemon.dat and FtpDaemon.ini from the current War FTPd server directory to the new directory before the server is started for the first time. Then start the server by executing WarFTPdv1.exe. To start to configure the server, start WarDaemonManager.exe.

When the daemon manager is started, the first thing you should do is to configure the internal DNS server variable.

Open the Advanced Server properties dialog and select

Sock_DNSserver. Type in the IP address to your DNS server and press the [SET] button. If you don’t know the IP address to your DNS server, you can use a utility like nslookup.exe to get that. If you use an ISDN or modem dialup connection, you might have to call your ISP to get the DNS server address.

Now, shut down the server by pressing the STOP button .

Exit the WarDaemonManager and restart first the server, and then the manager.

Now it’s time to configure the file permissions.

Press the User Manager button

and select the Security - File Access settings.

If you have upgraded from a previous version, you might have some references to VfSys in the file access list. VfSys is not yet implemented, so you will have to ‘convert’ the VfSys paths to file paths before they will be accessible for users.

Double click on each of the paths you need to convert and select

File as file system.

Make sure that the paths are valid. If you want to change the logical mapping (how the user see the path), you can disable the auto-arrange feature by checking the ‘freeze’ option and type in the mount point you prefer.

If you want to add new paths, the fastest way is to drag the paths from Windows Explorer into the file access list and then set the permissions. If you right click on a path or a group of marked paths, you can set common permission flags from a right-click menu.

Go over the user accounts and check that the users you want to give access to the server have FTP login access.

When the server is configured, shut it down, exit the manager, and start it again, - first the server, then the manager.

The performance of the current server has not yet been optimized for speed. I have performance readings from 300 to 6500 kb/s. It seems like system calls are slowing down the performance. If the buffer size is increased from 16 KB to 2 MB, the average transfer speed (on read) is raised from 500 Kb/s to 2100 KB/s. If I use the War FTP Client with 1 MB buffers and high priority, I can get readings up to 1,500,000 kb/s on 1 MB files.

The manager should now be reasonable stable. It will crash if the connection to the server is lost, while a dialog is open. It will also freeze if you click in the manager when the server is shutting down.

All the standard FTP commands are implemented. The only significant features that are missing at this time are:

• Virtual file system
• CD-ROM file system
• Disk quota restrictions

The new features and the security are not 100% debugged. Please try normal (and odd) FTP operations, and try to break trough the security.

Jarle

The session manager keeps track of all activity in the server. It is also responsible for the security. The session manager has 4 important lists to manage:

1. Hosts
2. Users
3. Servers
4. Sessions

The Hosts list contains all current and recent hosts that has connected or tried to connect to the server. When a connection is established, the session manager looks for the connecting host in the Hosts list. If it is unresolved, a reverse DNS lookup is initiated, and the session denied. When the DNS lookup is complete, the Host entry set a flag that indicate that logins are allowed to proceed. The following scheme illustrate this:

The user list contains one entry for each user account that is online or was online very recently (up to a few seconds ago). This list is used by the session manager to limit the number of concurrent sessions from one user, and to store information about the user. This ‘cache’ function makes it unnecessary to query the user database about user information all the time.

The Server list contains one entry for each virtual domain that is active. Normally you will have only one virtual domain active, because one domain can listen to all the IP numbers assigned to the machine. But if you are in the business of WWW/FTP hosting, you might have several virtual domains in this list, where each virtual domain handle one IP number and one eventually handle the rest of the IP numbers assigned to the machine. The server list is used by the session manager to control the virtual FTP servers. Do you remember the server/client design mentioned earlier? Since the Session Manager acts as a server module for the FTP server, it can’t contact the FTP server modules to inform about important events, like a system shutdown. Instead, it relays such information in the server list, and waits until the virtual domain’s FTP server module asks about the current states to pass the information on. In the occasion of a system shutdown, the FTP server module will consider the nature of the shutdown (shutdown when ready or immediately) and either throw all users off and resign from the server list, or throw each user off as he complete the current file transfer.

The session list contains unique information about each user, batch and administrator connection to the server. This list is used to control the idle time of each connection, status information for the War Daemon Manager, and counters used by different features like the up/download ratio.

The user database and session manager is designed to run in one thread. Other modules might share this thread, or run in their own thread. The session manager performs no disk access at all, and the user database performs only occasional disk access, when auto-saving the user database. A multithreading design in these modules would most likely add so much system overhead, compared with the current design; that the total performance could decrease – even on a multi CPU system.

The user database is kept in memory. It is structured as a binary tree, where the actual user properties are linked from a tree-node. The design is inspired from the UNIX file system, and allows several tree nodes to link to the same physical user properties. (This ‘hard link’ design is not 100% finished, and therefore there are no high-level functions to actually link a user to another tree node.) The user database is based on the WarTree, WarTreeNode and WarTreeInode base classes, - originally designed for the virtual file system.

Many user properties can be recursive. When the server needs some information about a node (user, class, group, virtual domain) – special functions are used to search the tree recursively for the information, until a valid value or a default value is found. These functions are called through the WarUserNode class, or, more conveniently, trough the WarUserHandle high-level interface.

It’s too early to write a full set of guidelines for those of you working on server extensions. There will be a well-defined plugin interface, and there will be a well-defined API for external programs. At this time I can say the following:

1. If you need to access the user database/session manager – use the WarUserHandle class. This class is thread-safe, and will handle communications in an optimal manner, no matter if you use the same thread as the user database or another. Note: All FTP user sessions or WarFile objects owned by an FTP user session has valid WarUserHandle objects you can use in most situations.
2. Use the SesMgr.GetOption() call to access global variables. This call is faster than WarUserHandle(), but still thread-safe.
3. Don’t make any assumptions about threads. A client connection might or might not run in it’s own thread. Your code must be thread-safe. Avoid static variables.
4. Never hold on to the current thread more than a few milliseconds. If you need to perform timely operations, use worker threads and callbacks. If you stall on network traffic, use WarSocket’s and override the On*() callback functions. The WarSocket class will provide the best overall performance on TCP/IP connections. If you need to test time-out values or perform housekeeping, install your own external timers.

RPC (Remote Procedure Call) is used to communicate with the server. In order to make communications independent of vendors (like Intel or Microsoft), ONC RPC (former Sun RPC) is used. One major benefit from this choice is that servers running under Win95 can have full RPC server capabilities (Microsoft has only implemented the MS RPC server interface for NT).

The RPC interface is defined in resource files, ending on .x. A special program, wrpcgen, reads these files, and translate the interface into c code. Then the interface can be compiled with a c/c++ compiler and used by programs.

Wrpcgen is based on Sun’s original rpcgen utility. Changes have been made to interface the RPC source modules with the War server, and to make client calls thread-safe. The header files and source files used for the ONC RPC interface in the war server and other programs are not 100% compatible with the original files. If you are going to interface a program with the war server, the most convenient way will be to simply compile the source files for the war server, and then call high-level functions to do the job. Or – if you prefer it the ‘hard’ way – to compile the .x files with Sun’s original rpcgen and use the source modules produced by that. Note that there is a hack to implement reasonable security into the RPC interface. You have to study the WarUserHandle::RPCLogin() function in order to write your own login function to the user database/session manager.

The reason for the changes was the need to auto-generate source code modules that would fit into the War server framework, and the need to make the client modules thread-safe.