Written by Bjorn Ekwall in the spring of 1995 (updated in January 1996) How to modularize ================= There are some guidelines for modularizing; some of them are easy to implement but there are a couple of tricky parts as well. The things to watch out for is that the module-to-be should really *be* a module. i.e. an entity that supplies a well defined *separate* functionality. It should also have a "clean" interface to the rest of the kernel, i.e. it should not demand that a whole bunch of new symbols should be added to "linux/kernel/ksyms.c". The "easy" parts: 1. Make sure that the first include is this line: #include Only or should be included before these lines (see "Note 2" for execptions). This will ensure that the "symbol versions" will work correctly, as well as supplying the necessary module definitions that you will need later on. Note 1: For newer kernels, is included by , so you don't have to. For older kernels, you will also have to include . If you get complaints about "kernel_version" being multiple defined, just remove the "#include ", or, alternatively, add "#define __NO_VERSION__" before you include , or add "-D__NO_VERSION__" to the Makefile CFLAGS. Note 2: If you compile a module that is _not_ included in the kernel tree, and you want to use versioned module symbols, you should also add the following to your CFLAGS in the module Makefile: "-DMODVERSIONS -include /usr/include/linux/modversions.h" Of course you will also have to have "-DMODULE" in CFLAGS as well. Some older versions of gcc had trouble with "-include"; in that case you will have to add "#include " to the module instead. In this case _only_: add this _before_ . 2. Identify the spots where the module functionality is requested and released. Usually you will find these spots in "open_*()" and "close_*()". Every time the module supplies a "resource", add: MOD_INC_USE_COUNT; and every time it releases the resource, add: MOD_DEC_USE_COUNT; Without these safeguards you will risk the sanity of the module as well as of the kernel. Check the logic so that the INC's and DEC's are balanced, since the "hidden" count must be zero for the module to be unloaded. 3. Lastly, add the "standard module definitions", using this skeleton: #ifdef MODULE ... /* driver specific module definitions */ ... int init_module(void) { ... /* driver specific inititialization, usually "register_*()" */ ... if (fail) return -EIO; ... return 0; } void cleanup_module(void) { ... /* driver specific cleanups, ususally "unregister_*()" */ ... } #endif /* MODULE */ Note that due to historical reasons, some modules might still include something like this in their "cleanup_module": if (MOD_IN_USE) printk("xxx: device busy, remove delayed\n"); else /* driver specific cleanups, ususally "unregister_*()" */ With recent kernels this is taken care of automatically, so you won't need it. If you feel adventurous, you can clean up your old modules by removing this historic check, but then of course you have also said goodbye to Linux 1.0.* ... 4. Update the Makefile so that the module will be compiled with the correct flags and so that it will be automatically linked to the correct place in the directory tree. You can take a look at "linux/drivers/net/Makefile" for an example. If you are lucky, you just add a line or two, where the magic line is: M_OBJS += my_module.o or (if you use "register_symtab"): MX_OBJS += my_module.o If your Makefile is not already prepared for modules, you will have to modify (or add) the relevant targets. As before: "the previous source is your friend" :-) The "hard" parts: Now you have a potential module that you can try out by doing "make modules" in the kernel root directory. The compilation will usually go without problems, but just wait... The command "insmod my_module.o" will more often than not protest and give you a list of undefined symbols. Whatever you do, *do not* start to add symbols to "linux/kernel/ksyms.c" !!! Often these "errors" come from the fact that the module doesn't have a "clean" interface to the kernel. Kernel resident drivers will get their symbols resolved from the complete set of kernel symbols, but the module only has the exported symbols available. Hidden interdependencies between parts of the kernel will now be seen as these missing symbols. The solution is to clean up the interface, instead of just making the dependencies permanent by adding a symbol to "linux/kernel/ksyms.c". Two examples: - In kernels before 1.1.85 there was a function in slhc.o, "ip_csum", that prevented that module to be cleanly loaded. Instead of just adding this symbol to ksyms.c, I noticed that "ip_csum" was *only* used by slhc.o, and that is was defined as an inline function in "linux/net/inet/ip.c". The "correct" solution was to move this "function" from ip.c to ip.h, since this header was included anyway by slhc.o! Problem solved! - In the 1.1.88 kernel, the nfs.o module complained about "socki_lookup" being undefined. Would you believe that this function was used externally *only* by nfs.o, and that it has later been cleaned up in "linux/net/socket.c" so that the function call can be replaced by a simple assignment!?! --- linux-1.1.88/fs/nfs/sock.c Mon Jan 23 09:38:29 1995 +++ linux/fs/nfs/sock.c Sun Feb 5 01:27:48 1995 @@ -37,9 +37,6 @@ * ***FIXME*** should probably put this in nfs_fs.h */ #define NFS_SLACK_SPACE 1024 - -extern struct socket *socki_lookup(struct inode *inode); - #define _S(nr) (1<<((nr)-1)) /* @@ -81,7 +78,7 @@ file = server->file; inode = file->f_inode; select = file->f_op->select; - sock = socki_lookup(inode); + sock = &inode->u.socket_i; if (!sock) { printk("nfs_rpc_call: socki_lookup failed\n"); return -EBADF; Problem solved! This should be your *first* step before you even *start* thinking of adding symbols to "linux/kernel/ksyms.c"... My own "snooper/sniffer" for these cases is just to do "nm -gu my_module.o" and compare with "ksyms -a". After having removed the "module interface": _Using_Versions _init_module _cleanup_module _kernel_version _mod_use_count the remaining undefined symbols can be searched for in all sources in the kernel tree. I'm lazy, so I have prepared two files, like this: find . -name '*.c' -print > Sources-c find . -name '*.h' -print > Sorces-h For every undefined symbol, do: grep symbol_name `cat Sources-c` grep symbol_name `cat Sources-h` With this information it is time to look through the sources and to check if there *really* is a need to export the symbol, or if a small rewrite will fix it... This is tedious work, but you only have to do it once :-) For every symbol :-( I promise you that you will learn a lot more of the kernel internals than you ever thought you needed! Anyway, the goal is to create clean interfaces. This will reduce the "bloat" in ksyms.c, but it will also, as a side-effect, clean up the kernel! There is yet another cleanup method that you should contemplate. Since 1.1.23 there has been support for stacked modules, where a module can supply symbols for use by later loaded modules. This is accomplished by allowing insmod to extract the global symbols in a module and to add an extra symbol table to the kernel. For modules that consist of only one source file, it is possible to selectively export symbols by using the "static" keyword on all *local* symbols. For other modules, that are pre-built by "ld -r", this is not possible, which means that the set of exported symbols might become *huge* :-( If the module isn't intended to be a "base" module in a module stack, you can always tell insmod not to export the symbols by doing "insmod -x module.o". But there is another way! You can use the kernel function: "register_symtab()" and select the symbols in your source instead! Actually, one can also explicitly *delete* all symbols from a module by including "register_symtab(0)" in the function "init_module". In "linux/drivers/net", take a look at "Makefile" and "ppp.c" for an example of the use of "register_symtab()". IMHO this is the right way to do it... Bjorn