VXE, a Linux security tool


System Administration


[Photo of the Author]


original in en Serge Lozovsky 


Total 20 years of experience in information systems as programmer/manager. 5 years of Internet technology, 9 years UNIX, 4 years Artificial Intelligence (knowledge and data representation).


Serge Lozovsky presents a software package that he developed to make UNIX systems more secure. The software is available under a non free licence. It is possible to use it at no cost for noncommercial educational purposes.

A major problem with UNIX security is that the superuser can do with the system anything he wants. There are programs (daemons) which work with superuser privilegies, for example popd, sendmail, and are accessible from the network (Internet/Intranet). There could be bugs in any program, so the intruder connects to such a program via network, exploits existing bugs in it and gets control over all hosts.

VXE (Virtual eXecuting Environment) protects UNIX servers from such intruders, cracker attacks from the network and so on. It protects software subsystems, such as: SMTP, POP, HTTP and any other subsystem, already installed on the server. There is no need to change the configuration of existing software - just PROTECT it.

So, VXE solves the following problem: protects host and particular subsystems, which work as superuser and can have bugs. This is the situation we have in real life.




A Virtual eXecuting Environment (VXE) protects the host and particular subsystems, which work as superuser and could have bugs. When the program works in superuser mode, it can access all resources of the operating system (OS). VXE creates a virtual environment for each subsystem. In such an environment only the resources needed for normal operation are visible and available to the subsystem. Subsystem stands for, the startup program and all subprocesses initiated (forked) by it. Any subprocess runs in the same VXE as the parent. To affect any system resources, the program uses OS system calls (syscalls). VXE has means to describe what system calls, with what parameters are available for each subsystem. For example, it can be described (for file operation syscalls) that some files are readable and some executable, network operations unavailable (in case of POP server - it handles network connection, but doesn't make new ones) and these restrictions can't be broken even by a program with superuser privileges.

These restrictions can be as smart as needed. If intruder gets control over such subsystems, he can't use ordinary methods to sniff information or affect the system. Everything he can do in theory, using sophisticated methods, - is to affect the operation of the hacked subsystem, but not the OS itself, nor another subsystems. Here, ordinary methods, are those, where an intruder gets superuser privileges and runs a command interpreter (shell), and ordinary utilities, such as text editor, copy utility and so on. He can't do anything without such utilities. For example, a POP server doesn't need a text editor and a copy utility for it's work, so there is no such programs in VXE environment, created for POPD protection.

More exactly, VXE protects system and it's sybsystems from interference of hacked subsystem (which works under the control of VXE). And as a side effect, provides protection of the subsystem itself (in a way described above). For simplicity, in the following text, we will say that VXE protects subsystem.

VXE description and activation

VXE description (VXED) is a small LISP program (set of functions) which uses a declarative description of acceptable parameters for different system calls. This VXED is loaded to the kernel where it controls system call parameters from the specified subsystem. So VXEDs are dynamically loadable modules, handled by the small LISP interpreter, inserted into the kernel. In the current VXE version, this is vxelisp, derived from RefLisp  (Bill Birch bbirch@ctp.com). vxelisp has new internal bigstring representation, a full set of string and bit functions. The kernel version of vxelisp is reentrant, to handle different VXEDs simultaneously.

There are two methods to activate VXED. Explicit and implicit (automatic). Explicit activation is done using the vxe program. Parameters are VXED pathname, path and parameters of the executable, which will be run with restrictions. For the automatic method, the vxed utility preloads all needed VXEDs into the kernel. Each VXED has an activation pattern. During program start (exec), the kernel checks the executable path against patterns and corresponding VXED is activated. This method can be used, to activate protection at the start of any program in a specified directory (and all subdirectories). For example, to protect CGI scripts, supplied by users, VXEDs can be defined for each user subdirectory.

The VXE development system

Any sophisticated VXED can be created manually, using the full power of vxelisp. But VXE doesn't force administrators to learn and use LISP. One can think about VXE as of self-learning system. The VXE development system (DS) runs VXE in trace mode.

Such a run prints descriptions of permitted (used) system calls. Creation and modification of a VXED is done via WWW interface. The development system supports two types of VXED. Strict and filesystem types. The strict VXED describes all permitted syscalls explicitly. Filesystem VXED describes read, write, and execute permissions for a defined path. Specified restrictions apply to filesystem syscalls, all other syscalls are permitted. After the VXED has been created for particular subsystem, it works in soft mode. In this mode all violations of VXED are logged, but syscalls are performed. VXE DS can upgrade VXED automatically, using the logged information.

Surely, needed changes in VXED can be done manually using the VXED editor. Violations can be caused by an intruder activity or by deviation in the subsystem's behavior under various circumstances. The VXE administrator reviews the log with the help of the DS and makes the decision, if an upgrade is reasonable. If there are no violations, then the VXED can be switched to production mode. In this mode violations are logged and syscalls are blocked (fail). Once again, the log can be used for intruder detection or for VXED tuning.

For security reasons, all control actions over VXE can be done only by superuser and outside any VXE.


VXE affects performance in the following ways. If program runs outside any VXE, every syscall executes two assembler instructions more (checks if VXE is in effect for current process and jump if not). For every exec syscall a small C subroutine checks if there is a matching VXED already available in the kernel. For programs that run in VXE, a few lines of C code check if parameter verification is needed. Some syscalls can be marked in VXED as uncheckable (for example, by default, read and write operations). And only the rest of the syscalls are checked by very small LISP functions. These functions are located in VXED and can be easily observed by the administrator.

VXE support: vxe@intes.odessa.ua
VXE home page: http://www.intes.odessa.ua/vxe