\input texinfo @c $Id: lyskomd.texi,v 1.40 2001/11/19 23:25:03 ceder Exp $ @c %**start of header @setfilename lyskomd.info @include version.texi @settitle lyskomd @value{VERSION} Reference Manual @setchapternewpage odd @c %**end of header @iftex @parindent 0pt @begin tex \global\def\BB#1{\b{#1}} \global\def\II#1{\i{#1}} @end tex @end iftex @ifinfo @macro BB {text} '\text\' @end macro @macro II {text} /\text\/ @end macro @end ifinfo @ifinfo This is the reference manual for the lyskomd LysKOM server version @value{VERSION}. Copyright @copyright{} 1995-2001 Lysator ACS. Permission is granted to make and distribute verbatim copies of this specification provided the copyright notice and this permission notice are preserved on all copies. @end ifinfo @dircategory LysKOM @direntry * lyskomd: (lyskomd). lyskomd reference manual. @end direntry @titlepage @sp 10 @title lyskomd Reference Manual @sp 2 @subtitle Server version @value{VERSION} @sp 2 @author by the lyskomd developers @page @vskip 0pt plus 1filll Copyright @copyright{} 1995-2001 Lysator ACS Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies. Modified versions of this document may be redistributed with the added condition that all modifications not cleared with the LysKOM development group are clearly marked and that the entire modified work be redistributed under the same conditions as the original. Permission is granted to copy and distribute translations of this manual into another language under the same conditions as for modified versions. @end titlepage @ifinfo @node Top @top lyskomd lyskomd is a server for the LysKOM conferencing system. This info file documents version @value{VERSION} of lyskomd. @menu * Copying:: lyskomd is free software. * Overview:: Overview of LysKOM. * Installation:: How to install lyskomd. * Configuration:: How to configure lyskomd. * Running lyskomd:: How to run lyskomd. * Invoking updateLysKOM:: How to run updateLysKOM. * Invoking komrunning:: How to run komrunning. * Administration:: Administering a LysKOM server. * Bugs:: Known bugs in lyskomd. * DBCK Reference:: Checking and repairing the database. * splitkomdb:: How to backup the database. * Hacking:: Notes for server developers. * lyskomd Database Specification:: @end menu @end ifinfo @node Copying @chapter Copying lyskomd is free software. It is distributed under the Gnu General Public License version 2. The file COPYING in the top level of the distribution contains the text of the license. @node Overview @chapter Overview LysKOM is a conferencing system@footnote{Or in modern terms, enabling technology for Computer-Supported Cooperative Work (CSCW).}. Similar systems were QZ-KOM and PortaCOM@footnote{Also known as ``PottaKOM'' and ``BortaKOM''.}. The LysKOM system is copyrighted by Lysator Academic Computing Society and distributed under conditions of the GNU General Public License version 2. LysKOM and its documentation is provided ``as is'' without warranty of any kind. This reference manual documents version @value{VERSION} of the lyskomd LysKOM server. The lyskomd server is the work of several people. The main contributors have been Per Cederqvist @email{ceder@@lysator.liu.se}, Inge Wallin @email{inge@@lysator.liu.se}, Thomas Bellman @email{bellman@@lysator.liu.se}, David Byers @email{byers@@lysator.liu.se} and Peter Eriksson @email{pen@@lysator.liu.se}. @section History In 1990, Per Cederqvist @email{ceder@@lysator.liu.se} and Peter Eriksson @email{pen@@lysator.liu.se} and a few other persons started to write the server. It was operational in the summer of 1990, even though the members of Lysator discovered a thing called MUD. We started using RCS on 20 May 1991. The first release was made on 16 Sept 1991. Around that time we switched from RCS to CVS to handle our source code. @node Installation @chapter Installation Instructions for compiling and installing lyskomd are in the files @file{README} and @file{INSTALL}, located in the top level of the lyskomd distribution. Installation should be straightforward on most platforms. @node Configuration @chapter Configuration There are two configuration files for lyskomd. One defines the server options and the other defines aux-item types @ref{(protocol-a)The Aux-Item List,The Aux-Item List}. @menu * Server Configuration File:: The server configuration file. * Aux-Item Definition File:: The aux-item definition file. @end menu @node Server Configuration File @section Server Configuration File The server reads its configuration from a configuration file. The default configuration file is @file{/usr/lyskom/etc/config}. The location of the configuration file can be changed at run-time by supplying an argument to lyskomd. The configuration file is line oriented. Each line consists of a parameter name followed by a colon, and the value of the parameter. Empty lines and lines whose first non-blank character is a @samp{#} are ignored. @menu * Parameter Types:: Types of configuration parameters. * Parameters:: Valid configuration parameters. @end menu @node Parameter Types @subsection Parameter Types Every parameter has a type. The legal types are: @table @code @item bool The parameter can be true or false. Legal values are @code{on}, @code{true}, @code{yes} and @code{1} for true and @code{off}, @code{false}, @code{no} and @code{0} for false. @item locale-name The parameter is a locale name. The value must be a legal locale name of the system where lyskomd is running. @item path The parameter is a path name. The value must be a legal path on the system where lyskomd is running. Most paths you can specify can be either absolute paths (if they begin with a @samp{/}) or paths relative to the installation prefix which is specified at compile time or with the @samp{Prefix} parameter in the configuration file. @item portname The parameter is a TCP/IP port. It can be a symbolic port name (traditionally looked up in @file{/etc/services}) or a port number. @item int The parameter is a number of some sort. It can be a conference number, text number or perhaps a timeout. @end table @node Parameters @subsection Parameters @table @code @item Max conferences: @var{int} The maximum number of conferences possible in the server. This number must be larger than the number of conferences in the database. This parameter is required. There is no default. @item Max texts: @var{int} The maximum number of texts possible in the server. This number must be larger than the number of texts in the database. This parameter is required. There is no default. @item Locale: @var{string} Use @var{string} as the locale to run in. This parameter is only available om systems which support the @code{setlocale} call. If this parameter is not set, no call to @code{setlocale} will be made. The default is unset. @item Force ISO 8859-1: @var{bool} This option is provided for those with dysfunctional computers that cannot handle @code{setlocale} properly. If this is set, lyskomd will handle texts according to the ISO 8859-1 (latin1) alphabet. Default is off. @item Prefix: @var{path} All files that the server uses are found in sub-directories of this directory. The default value of this parameter is set at compile time. The default at compile time is @file{/usr/lyskom}. @item Send async: @var{bool} Do not send any non-requested messages. This disables the sending of messages about events in the server to all connections. Use of this parameter is not recommended. Default is on. @item Client host: @var{hostname} Specify which IP number the server should use when listening for new clients. @var{hostname} may be a FQDN (such as @samp{kom.lysator.liu.se}) or an IP number (such as @samp{10.0.0.1}). Default is to bind @code{INADDR_ANY}, which means that the server will listen to all IP numbers of the computer it is running on. @item Client port: @var{portname} Listen for new clients on port @var{portname}. The default is 4894, which is what all clients expect. Do not change this parameter without really good reason. @item Presentation of conferences: @var{int} The number of the conference where presentations should be sent. Defaults to 1. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the @ref{(protocol-a)set-info,set-info}. @item Presentation of persons: @var{int} The number of the conference where presentations should be sent. Defaults to 2. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the @ref{(protocol-a)set-info,set-info}. @item Motd-conference: @var{int} The number of the conference where "message-of-the-day" messages should be sent. Defaults to 3. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the @ref{(protocol-a)set-info,set-info}. @item News-conference: @var{int} The number of the conference where news of interest to the readers of this LysKOM server should be written. This is typically a conference with very low traffic which everyone shoule be a member of. Clients should offer new users to join it. Defaults to 4. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the @ref{(protocol-a)set-info,set-info}. @item Message of the day: @var{int} Default message-of-the-day of this server. The text will be shown automatically by conforming LysKOM clients when a user logs on. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the @ref{(protocol-a)set-info,set-info}. @item Garb: @var{bool} Should the database be automatically purged of old texts? The default is on. @item Never save: @var{bool} Completely disables saving the database. Do not set this to @code{true} unless you really know what you're doing. The default is @code{false}. @item Log accesses: @var{path} This parameter can only be set if the server has been compiled with @code{LOGACCESSES} defined. It will save a trace of all activity in the database to a file, for later use in simulations et c. Compiling with @code{LOGACCESSES} slows the server down quite a lot, so it is normally not defined. @item Data file: @var{path} The path relative to the installation prefix where part of the database is kept. The default is @file{db/lyskomd-data}. @item Backup file: @var{path} The path relative to the installation prefix where a backup of the database is kept. This file will always contain a complete database, but it may be a little out-of-date. Default is @file{db/lyskomd-backup}. @item Backup file 2: @var{path} The path relative to the installation prefix where a previous generation of the backup of the database is kept. This file may be needed if an error in the backup file is detected during the creation of the data file. Default is @file{db/lyskomd-backup-prev}. @item Lock file: @var{path} Name of the lock file that ensures that @code{dbck} and @code{lyskomd} never attempt to modify the database at the same time. It should always reside in the same directory as the @samp{Data file}. Default is @file{db/lyskomd-lock}. @item Text file: @var{path} The path relative to the installation prefix where the actual texts in the database are kept. Default is @file{db/lyskomd-texts}. @item Text backup file: @var{path} When @code{dbck} is run with the @samp{-g} option (@ref{(dbck)Invoking dbck,Invoking dbck}, it will store the previous contents of the text file in the file specified by this option. The path is relative to the installation prefix. This file is never used by @code{lyskomd} itself. Default is @file{db/lyskomd-texts-backup}. @item Backup export directory: @var{path} When @code{splitkomdb} is run, it will create a copy of the database in this directory. The copy will be split in a way that helps to keep incremental backups of that directory small. @xref{splitkomdb}. The path is relative to the directory specified by @code{Prefix:}. This directory is never used by @code{lyskomd} itself. Default is @file{exportdb}. @item Log file: @var{path} The path relative to the installation prefix where log messages from lyskomd are written. Default is @file{etc/server-log}. @item Log statistics: @var{path} Whenever lyskomd receives a SIGUSR1 it will append a timestamp and a count of how many different atomic calls have been made in this file. The path is relative to the installation prefix. Default is @file{etc/lyskomd-log}. @item Pid file: @var{path} When lyskomd is up and running it will write its pid in this file. The path is relative to the installation prefix. This file is used so the @code{updateLysKOM} script can easily find out what pid the LysKOM server has. Default is @file{etc/pid}. This file should be removed when the computer reboots, before @code{komrunning} or @code{updateLysKOM} is run. @item Memory usage file: @var{path} When lyskomd exits normally it appends some info on its usage of memory to this file. The path is relative to the installation prefix. Almost any memory leak bugs should be detectable by looking in this file. Default is @file{etc/memory-usage}. @item Aux-item definition file: @var{path} This file defines which aux-items the server should support and how it should handle them. You will find the details in @xref{Aux-Item Definition File}. The path is relative to the installation prefix. Default is @file{etc/aux-items.conf}. This file is re-read if a @samp{SIGWINCH} singal is sent to the server. @item Core directory: @var{path} The Directory where core dumps are written. This path is relative to the installation prefix. Default is @file{cores}. @item Status file: @var{path} This file is created by @code{komrunning} to indicate that lyskomd should currently not be running. When this file exists @code{updateLysKOM} will send it a @samp{SIGHUP} signal, so that it saves the database and dies. Default is @file{etc/status}. @item Nologin file: @var{path} If this file exists, the server will not allow any connections at all. Default is @file{/etc/nologin}. @item Idle timeout: @var{int} Number of milliseconds to sleep when there is nothing for lyskomd to do. Default is @code{120000} (two minutes.) @item Garb timeout: @var{garb} Number of milliseconds to sleep when the server is garbage-collecting texts, but has nothing else important to do. Default is @code{100} (0.1 seconds.) @item Sync timeout: @var{sync} Number of milliseconds to sleep when lyskomd is saving its database. Defaults to 0. @item Permissive sync: @var{bool} Turning this option on lets any session sync the LysKOM database. Turning it off restricts the operation to LysKOM administrators. Default is off. @item Garb interval: @var{int} Number of minutes between each garb sweep. Defaults to @code{1440}, that is, a garb sweep will be run once per day. @item Sync interval: @var{int} Number of minutes between syncs. The current version of lyskomd keeps changes to the database in memory until they are synced to disk. This parameter specifies the number of minutes the server waits before attempting to dump the database. The default is @code{5}. @item Sync retry interval: @var{int} If anything goes wrong while trying to dump the data base (such as if the disk is full), lyskomd will wait for this many minutes before trying again. Default is @code{1}. @item Saved items per call: @var{int} When the server is saving the database, it does so in the background. It serves one call from a client, saves a few items to the new database file, serves another call, et c. This parameter sets the number of items (texts, conferences, persons) that are saved after each call. Default is @code{5}. @item Max client data length: @var{int} The maxiumum allowed length for client name and version data. The default is @code{60}. @item Max conference name length: @var{int} The maximum length of conference names. The default is @code{60}. @item Max password length: @var{int} Only the first eight characters of the password are currently significant, even if this number is much larger. The default is @code{128}. @item Max what am I doing length: @var{int} The maximum length of the string permitted in the protocol A call @ref{(protocol-a)change-what-i-am-doing, change-what-i-am-doing}. The default is 60. @item Max username length: @var{int} The maximum length permitted for user names. Default is 128. @item Max text length: @var{int} The maximum length allowed for a text. The default is 131072 characters. @item Max aux_item length: @var{int} The maximum length allowed for a single aux-item. The default is 16384 characters. @item Max broadcast length: @var{int} The maximum length allowed for broadcast messges. The default is 1024 characters. @item Max regexp length: @var{int} The maximum length allowed for regexps in various calls. The default is 1024 characters. @item Max marks per person: @var{int} The maximum number of marks a person is allowed to have. The default is 2048. @item Max marks per text: @var{int} The maximum number of marks a text can have. The default is 1024. @item Max recipients per text: @var{int} The maximum number of recipients of a text. The default is 512. @item Max comments per text: @var{int} The maximum number of comments a text can have. The default is 128. @item Max footnotes per text: @var{int} The maximum number of footnotes a text can have. The default is 32. @item Max links per text: @var{int} The maximum number of misc info items that can be added to a text. @item Max mark_as_read chunks: @var{int} @c FIXME: What is this? @item Max super_conf loop: @var{int} @c FIXME: What is this? @item Max accept_async len: @var{int} Maximum length of list accepted in the accept_async call. Default is 128. @item Max aux_items deleted per call: @var{int} Maximum number of aux_items that can be deleted in one call. Default is 128. @item Max aux_items added per call: @var{int} Maximum number of aux_items that can be added at once. Default is 128. @item Default garb nice: @var{int} Each conference has a lifetime for texts written in it. The lifetime is counted in days, and can be set for each conference by the administrator of the conference. This is the default value assigned to new conferences. Default is 77 days. @item Default keep commented nice: @var{int} A text will not be removed if it has comments newer than a certain number of days. This number can be set for each conference. This parameter specifies the default value for that number of days. The default is 77. @item Max client transmit queue: @var{int} Max number of pending data blocks in the reply queue to a client. If there is ever more than this many data blocks in the queue the client will be disconnected. Each atomic question typically generates two data blocks. Default is 300. @item Max simultaneous client replies: @var{int} This is a performance tuning parameter of little real interest. Default is 10. @item Open files: @var{int} Try to persuade the operating system to allow lyskomd to have this many open file descriptors simultaneously. Each client that is connected to the server occupies one file descriptor, and lyskomd needs several file descriptors for internal purposes. Default is to not use this parameter. @item Anyone can create new persons: @var{bool} If this is set, anyone can create a new person, even if he lacks special bits for doing so. Default is on. @item Anyone can create new conferences: @var{bool} If this is set, anyone can create a new conferences, even if he lacks special bits for doing so. Default is on. @item Allow creation of persons before login: @var{bool} If this is set, persons can connect the the server and create a new person without logging in. This is how new users register in open environments. If this option is off, then new persons can only be created by existing users. The default is on. @item Default change name capability: @var{bool} If this is set, new users are created with the ability to change their own name. Default is on. @item Ident-authentication: @var{policy} Decide how strictly the server should use the IDENT protocol. The policy can take any of three values: @table @asis @item @code{off} or @code{never} Do not use the IDENT protocol. @item @code{on} or @code{try} Use it, but allow logins even if the lookup fails. @item @code{require} or @code{required} Disallow connections if the server cannot find a IDENT login name. @end table @item Log login: @var{bool} Should logins be logged to the log file? Default value is off. @item Cache conference limit: @var{int} How many conference statuses the server cache should hold in main memory. Default is 20. This parameter should be set to at least the number of expected simultaneous logins. @item Cache person limit: @var{int} How many person statuses the server cache should hold in main memory. Default is 20. This parameter should be set to at least the number of expected simultaneous logins. @item Cache text_stat limit: @var{int} How many text statuses the server cache should hold in main memory. The default is 20. This parameter should be increased on busy servers. @item Echo: @var{string} Write @var{string} in the log when the config file is read. @item Jubel: @var{pers_no} @var{text_no} States that @var{pers_no} is not allowed to create text number @var{text_no}. Default is unset. @item Jubel: @var{pers_no} @var{dividend} @var{remainder} States that @var{pers_no} is not allowed to create any text number @var{T} which meets the condition @var{T} % @var{dividend} == @var{remainder}. Default is unset. @item Add members by invitation: @var{bool} If this is set, then adding others as members to a conference sets the invitation bit of the membership. If this is off, the membership bit is set to whatever the caller specifies. The default is on. @item Allow secret memberships: @var{bool} If this is set, then memberships may be secret. Otherwise any attempt to create a secret membership or change an existing membership to a secret membership will fail. The default is on. @item Allow reinvitations: @var{bool} If this is set, then it is possible to set the invitation bit of a membership even after it has been cleared. If it is not set, then the invitation bit of a conference type can only be set when the membership is created. It can be cleared at any time. The default is off. @item lyskomd path: @var{path} Path to the @code{lyskomd} binary. This is used by @code{updateLysKOM} to find the right program to run. Defaults to @file{bin/lyskomd}. @item savecore path: @var{path} Path to the @code{savecore} program. If a file named @file{core} exists in the directory specified with @code{Core directory} when @code{updateLysKOM} is about to start @code{lyskomd}, this program will be called first. It could, for instance, move the core file so that it is available for later debugging. @item Normal shutdown time: @var{int} In a normal setup, @code{updateLysKOM} will be run from @code{cron} once every ten minutes or so. If it detects that it has taken @code{lyskomd} more than @var{int} minutes to shut down it will print a warning message. @item Mail after downtime: @var{int} @itemx Mail until downtime: @var{int} If @code{lyskomd} has been down for X minutes, where @code{Mail after downtime} <= X < @code{Mail until downtime}, @code{updateLysKOM} will send a mail message to the mail address found on the first line of the status file. Actually, it is the age of the status file (named with @code{Status file}) that is measured. @end table @node Aux-Item Definition File @section Aux-Item Definition File The default aux-item definition file should not be changed unless it is really necessary. The need to change the definitions will probably only arise at installations used for client or server development. The location of the aux-item definition file is specified by the @code{Aux-item definition file} option in the server configuration file. The default location is @file{/usr/lyskom/etc/aux-items.conf}. @subsection Syntax of the Aux-Item Definition File The aux-item definition file contains a sequence of aux-item definitions. Each definition specifies one type of predefined aux-item: its number, name, and properties. Empty lines and all characters from a # character to the end of the line are ignored. Each entry has the following format: @example tag : name (target, target, ... ) @{ field = value; field = value; ... @} @end example @var{tag} is an integer, the aux-item's tag. If a tag is defined more than once, the last definition is used. The @var{target}s specify what kind of objects aux-items with tag @var{tag} can be added to. Valid targets are: @table @code @item any Aux-items with the specified tag can be added to any object in the database. This is shorthand for @code{text,conference,letterbox,server}. @item text Aux-items with the specified tag can be added to texts. @item conference Aux-items with the specified tag can be added to conferences that are @emph{not} letterboxes. @item letterbox Aux-items with the specified tag can be added to conferences that are letterboxes. @item server Aux-items with the specified tag can be added to the server itself. @end table It is legal to add one of the keywords @code{create} or @code{modify} before any target except @code{server}. If @code{create} is specified, aux-items with the specified tag can only be added when an object is being created. They cannot be added later. If @code{modify} is specified, aux-items with the specified tag can only be added after an object has been created. They cannot be added when the object is being created. Each @var{field}/@var{value} pair specifies a property of aux-items with the specified tag. Most values are boolean or trillian. Legal values for either type are @code{true} and @code{false}. Boolean values have reasonable defaults; trillian values can be unset. @table @code @item author-only Boolean, default false. When true, only the author of a text or supervisor of a conference can create items with this tag. @item supervisor-only Boolean, default false. When true, only the supervisors of the author or letterbox can create items with this tag. In all likelihood, the implementation of this flag is screwed up in version 2.0 of lyskomd. @item system-only Boolean, default false. When true, only the server can initiate creation of items with this tag. This is normally used for items that are created automatically in response to events in the system. @item permanent Boolean, default false. When true, aux-items with this tag cannot be deleted once they have been created. (They will be deleted automatically when the object they are assigned to is deleted.) @item unique Boolean, default false. When true, there can only be one non-deleted item with this tag per creator. @item owner-delete Boolean, default false. When true, the owner of the object that this aux-item is attached to can always delete the aux-item. For a text, the owner is defined as the supervisor(s) of the author of the text. For a conference, the owner is defined as the supervisor(s) of the conference. @item inherit-limit Integer, default 0. The maximum number of times items with this tag can be inherited, plus one. Zero means an unlimited number of times, one means no times, 2 means once and so forth. This number overrides the inherit-limit set by the client only if that number is higher than this one. @item inherit Trillian. When set, the inherit bit on new items with this tag is forced to the specified value. @item secret Trillian. When set, the secret bit on new items with this tag is forced to the specified value. @item hide-creator Trillian. When set, the hide-creator bit on new items with this tag is forced to the specified value. @item dont-garb Trillian. When set, the dont-garb bit on new items will be forced to the specified value. @item reserved-2 @item reserved-3 @item reserved-4 Trillian. When set, these flags force the values of the three reserved bits in the aux-item flags field. These should only be used by lyskomd developers, and then only very carefully. @item validate String or function, default none. When set to a string, this specifies a regexp that must match the data field in newly created items with this tag. If the regexp fails to match, then the item will not be created. The syntax for strings is essentially the same as the syntax used in C files. When set to a function, this specified a built-in validation function to call. The following validator functions are currently implemented: @table @code @item existing-readable-text Creation is only allowed if the item contains the number of an existing text that the item creator has permission to read. @end table @end table There are a few fields which specify actions the server is to take when something happens to aux-items with the specified tag. Each of these values is a function specification, the name of a trigger function defined in lyskomd. The syntax for functions is the name followed by an empty pair of parens. It is not possible to pass arguments to the functions yet. @table @code @item add-trigger Function to call when an item with the specified tag is added to an object. @item delete-trigger Function to call when an item with the specified tag is scheduled for deletion. @item undelete-trigger Function to call when an item with the specified tag scheduled for deletion is unscheduled. It should undo the effects of the delete trigger. @end table The following trigger functions are currently defined: @table @code @item mark-text Increase the mark count for the text the item refers to. The item must contain the number of a text. This trigger should be combined with the existing-readable-text validation function. @item unmark-text Decrease the mark count for the text the item refers to. The item must contain the number of a text. This trigger should be combined with the existing-readable-text validation function. @item link-faq Create a faq-for-conf item linked to a faq-text item. This trigger is used exclusively for faq-text items. The item must contain the number of a text. This trigger must be combined with the existing-readable-text validation function. @end table @node Running lyskomd @chapter Running lyskomd This section explains how to run lyskomd, the files it uses and how it can be controlled while running. @menu * Invoking lyskomd:: How to run lyskomd. * Signals:: How to control lyskomd with Unix signals. * Files:: Files used by lyskomd. @end menu @node Invoking lyskomd @section Invoking lyskomd @example lyskomd [-d] [@var{config-file}] @end example The option @samp{-d} adds one to the debug level. The amount of output on stderr is increased for each time the option is specified on the command line. Furthermore, if this option is used, lyskomd will not run as a daemon, but will stay in forground mode. Using one @samp{-d} makes the process print a `>' for every timeout, a message for every person that is connecting or disconnecting and a message for every successful or unsuccessful communication to the process. The optional @var{config-file} argument can be used to specify the server configuration file. @xref{Server Configuration File}. @node Signals @section Signals It is possible to control some aspects of lyskomd using Unix signals. The following signals have special meaning to the server: @table @samp @item SIGHUP Logs out all sessions, saves the database and exits normally. @item SIGQUIT Saves the database and dump core. (This should only be used for debugging purposes.) @item SIGUSR1 Print statistics about how often different commands have been used since the process started. @item SIGUSR2 Forks a child that immediately dumps core. The main process just waits until the child is done and then continues. @item SIGWINCH Re-read the aux-item definition file. @end table @node Files @section Files Used by lyskomd All file names can be changed in the server configuration file. @xref{Parameters}. @table @file @item /usr/lyskom Default value of the @code{Prefix} parameter. The default of this value is set at compile time, but it can be changed in the server configuration file. @xref{Parameters}. @item @var{prefix}/db/lyskomd-data Half of the database: all status information. @item @var{prefix}/db/lyskomd-texts The other half of the database: the actual texts. @item @var{prefix}/db/lyskomd-backup A backup copy of @file{lyskomd-data}. Never, ever delete this file unless you know what you are doing, or you may lose the entire data base. Most of the time this is the only complete database file! @item @var{prefix}/etc/pid File with the pid of the lyskom-process. @item @var{prefix}/etc/memory-usage On normal exit, @code{lyskomd} will append some statistics to this file. It can be used for detecting memory leaks. @item @var{prefix}/etc/aux-items.conf This file contains definitions of the aux-items that the server should support. It is read by @code{lyskomd} at startup. @item /etc/nologin If this file exists, lyskomd will not allow anyone to connect to the server. This path can be set with the @code{Nologin file} parameter in the server configuration file. @end table @node Invoking updateLysKOM @chapter Invoking updateLysKOM @example updateLysKOM [-c @var{config-file}] [ -v ] [ -V ] @end example @code{updateLysKOM} determines if @code{lyskomd} should be running. It can start or stop @code{lyskomd} if needed. It uses the same configuration file as @code{lyskomd} (@pxref{Server Configuration File}). You can use @samp{-c @var{config-file}} to override the compiled-in default. Note, however, that this option is not passed along to @code{lyskomd} if @code{updateLysKOM} starts it, so the option should be used with extreme caution. @samp{-v} and @samp{-V} causes @code{updateLysKOM} to report its version number and exit. @code{updateLysKOM} is normally run from @code{cron}; @pxref{Administration}. @node Invoking komrunning @chapter Invoking komrunning @example komrunning [-c config-file] [start | stop] komrunning -v | -V @end example @code{komrunning}, when invoked with no arguments, reports whether @code{lyskomd} is currently running or not, and whether it should be running or not. @samp{komrunning start} attempts to start @code{lyskomd}. @samp{komrunning stop} attempts to stop @code{lyskomd}, and it will not return until the server has saved its database and exited. @code{komrunning} uses the same configuration file as @code{lyskomd} (@pxref{Server Configuration File}). You can use @samp{-c @var{config-file}} to override the compiled-in default. Note, however, that this option is not passed along to @code{updateLysKOM} if @code{komrunning} invokes it, so the option should be used with extreme caution. The @code{komrunning} can be installed in @file{/etc/init.d/}. Be careful, however, to ensure that the pid file is removed earlier during the boot sequence. @samp{-v} and @samp{-V} causes @code{komrunning} to report its version number and exit. @node Administration @chapter Administration The first thing you will have to do is to follow the instructions in the files @file{INSTALL} and @file{README}. This will set up the LysKOM system with a database containing a few necessary conferences and one person - the administrator. Once the LysKOM system is running, there is not much you will have to do to keep it that way. One thing to remember is that the current release of the server has an incomplete handling of garbage collection of the database. The database is split into two files, the information file and the text file. Newly written texts are concatenated to the text file and old texts are never removed. The information file contains information about conferences, users and where in the text file the texts are. This file is properly garbage collected, but not the text file. There is a program called @code{dbck} (Data Base Check) which is used to check the consistency of the LysKOM database. This program can also be used to shrink the text file. To do this, just type @samp{dbck -g}. @xref{(dbck)}. When @code{dbck} is to be run on the database, the LysKOM server @emph{must} be stopped, or unrepairable damage may result. See below for a description on how to stop the server. There is a program called @code{updateLysKOM} which is used to ensure continuous operation. This program should be run with certain intervals, for instance from @code{cron}. If the LysKOM server has died for some reason, @code{updateLysKOM} restarts it. If the server is still running properly, @code{updateLysKOM} sends a signal (@samp{SIGUSR1}) to it, which causes the server to write some statistics to a file named @file{etc/lyskomd-log} in the lyskom directory. Taking the server down cleanly can be done in two ways: through the use of the LysKOM protocol on a socket, preferably through the use of a suitable client, or by sending the signal @samp{SIGHUP} to it. This will cause the server to save the database and close all client connections. It will also create a file named @file{etc/memory-usage} in which the memory usage of the server is reported. To prevent @code{updateLysKOM} from restarting a server, create a file named @file{/usr/lyskom/etc/status}. The file should contain a valid mail address on the first line. @code{updateLysKOM} will not restart the server as long as that file exists. In addition, if the file is between 1 and 2 hours old (configurable) an email will be sent to the mail address found in the file. If the file is older than that, an error message will be printed on stderr and @code{updateLysKOM} will exit with a non-zero exit status. cron is expected to deliver the error message to an operator. The shell script @code{komrunning} can be used to start and stop the LysKOM server. With no arguments, it will report the status. @example komrunning stop @end example will (attempt to) shut down the server, creating the file @file{/usr/lyskom/etc/status}. If the user running @code{komrunning} doesn't have permission to send signals to @code{lyskomd} the actual shutdown will be delayed until the next time that @code{updateLysKOM} is run. @example komrunning start @end example will restart the server. The actual starting of the server will be done by @code{updateLysKOM} the next time it is run. @code{komrunning} only removes the @file{/usr/lyskom/etc/status} file. @node Bugs @chapter Known Bugs @itemize @bullet @item lyskomd should re-read the config file when a @samp{SIGHUP} is received. @item lyskomd should terminate when a @samp{SIGINT} or @samp{SIGTERM} is received. @item The security policy is vague and the implementation is frayed at the edges. @item The choice of asynchronous messages is not very good. @item The server uses too much memory. @end itemize @c ====================================================================== @c ====================================================================== @c == == @c == DBCK REFERENCE == @c == == @c ====================================================================== @c ====================================================================== @node DBCK Reference @chapter DBCK Reference dbck is a program that can is used for minor database maintenance tasks and for repairing a corrupt lyskomd database. @menu * DBCK Overview:: Overview of dbck. * Invoking dbck:: How to run dbck. * DBCK Notes:: Notes about running dbck. * DBCK Files:: Files used by dbck. * DBCK Bugs:: Known bugs in dbck. @end menu @node DBCK Overview @section Overview The dbck program is used for minor maintenance of the LysKOM database and for repairing corrupt databases. In brief it performs the following functions: @itemize @bullet @item Compact the text file to remove deleted texts. @item Repair inconsistent membership information. @item Repair invalid recipients @item Repair inconsistent comment links @item Correct invalid local text numbers @item Correct invalid text maps @item Set special conferences @item Convert between database formats @end itemize @node Invoking dbck @section Invoking dbck The functionality of dbck is controlled through command-line switches. These are documented below. If @code{dbck} is invoked without any options it will read the database and report on its integrity. No files will be modified. @menu * General Options:: Controlling the overall behavior of dbck. * Database Repair Options:: Repairing errors in the LysKOM database. * Format Conversion Options:: Converting the database file to a new format. * Database Maintenance Options:: Options for database maintenance. * Reporting Options:: Options controlling status reports. @end menu @node General Options @subsection General Options These options control the general behavior of lyskomd. @table @asis @item @samp{-h} or @samp{--help} Give a usage message (which includes the version number and the compiled-in default location of the config file) and exit immediately. @item @samp{-v} or @samp{--verbose} Verbose mode. Report not only errors but the status of the database. @item @samp{-F} or @samp{--force-output} This option forces dbck to write the database file. Normally @code{dbck} will only write a new database file if changes have been made for some other reason. If you want to simply convert a database from one version to another, you will probably have to give this option. @end table @node Database Repair Options @subsection Database Repair Options The following options control database repair. @table @asis @item @samp{-i} or @samp{--interactive} Run interactively. If any inconsistency is found, a remedial cure will be suggested and the user must confirm the correction. @item @samp{-r} or @samp{--auto-repair} Repair simple errors without asking. @item @samp{-c} or @samp{--set-change-name} Consider it an error if the @code{change-name} capability of a person is not set. Due to a bug that capability was never set for newly created persons in release 1.6.1 of lyskomd. This option can be used to repair the damage. @end table @node Format Conversion Options @subsection Format Conversion Options dbck can be used to conver the LysKOM database from one storage format to another. This is necessary only when moving the database to a new server version. @table @asis @item @samp{-F} or @samp{--force-output} This option forces dbck to write the database file. Normally @code{dbck} will only write a new database file if changes have been made for some other reason. If you want to simply convert a database from one version to another, you will probably have to give this option. @item @samp{-o} or @samp{--output-version} This option is used to set the output version of the database. This option will normally be used in conjunction with the @samp{-F} option. Version 1.9 of @code{lyskomd} requires database version 1; version 2.0 requires database version 2. Versions of @code{lyskomd} prior to 1.9 requires database version 0. Note that information is irrevocably lost when converting from a higher to a lower database version. This options requires an argument: the output format version. @end table @node Database Maintenance Options @subsection Database Update Options dbck can be used to update certain aspects of the database that either were impossible to update in early versions of protocol A or that are inconvenient in all protocol versions. @table @asis @item @samp{-g} or @samp{--compact-text-mass} Do garbage collection on the texts part of the database. This removes all unreferenced texts from the database. @item @samp{-P} or @samp{--clear-password} Clear the password of a specified user. This option is silently ignored if the user does not exist. This option requires an argument: the ID of the person whose password is to be cleared. @item @samp{-G} or @samp{--grant-all} Grant all privileges to the specified user. This option is silently ignored if the user does not exist. This option requires an argument: the ID of the person who is to be granted all privileges. @item @samp{--pers-pres-conf} Set the person presentation conference of the server to the specified conference. Since version 1.9 of lyskomd the @code{set-info} call can be used to do this. @item @samp{--conf-pres-conf} Set the conference presentation conference of the server to the specified conference. Since version 1.9 of lyskomd the @code{set-info} call can be used to do this. @item @samp{--motd-conf} Set the message-of-the-day conference of the server to the specified conference. Since version 1.9 of lyskomd the @code{set-info} call can be used to do this. @item @samp{--kom-news-conf} Set the news-about-lyskom conference of the server to the specified conference. Since version 1.9 of lyskomd the @code{set-info} call can be used to do this. @item @samp{--motd-of-kom} Set the message of the day of the server to the specified text. Since version 1.9 of lyskomd the @code{set-info} call can be used to do this. @end table @node Reporting Options @subsection Reporting Options These options control reporting of information about the database. @table @asis @item @samp{-s} or @samp{--print-statistics} Gather statistics about the lengths of texts. A table containing the frequence of all lengths that are currently present is printed. @item @samp{-t} or @samp{--list-text-no} Print ``Checking @var{text-no}'' for every text that examined. @b{Warning:} This produces lots of output. @end table @node DBCK Notes @section Notes for DBCK The messages ``Conference @var{conf-no} has a bad Text-list. Starts with 0'' and ``Person @var{pers-no} has created @var{num} conferences, not @var{num} (as said in person-stat).'' are normal. If you get them when you specify the @samp{-g} option, let @code{dbck} repair them and run @samp{dbck -g} again. @node DBCK Files @section Files Used by dbck dbck uses the same files as @code{lyskomd} (@xref{(lyskomd)}.) All file names can be changed in the server configuration file. @xref{(lyskomd)Parameters}. @table @file @item /usr/lyskom Default value of @code{Prefix}. The default of this value is set at compile time, but it can be changed in the server configuration file. @xref{(lyskomd)Parameters}. @item @var{prefix}/db/lyskomd-data Half of the database: all status information. @item @var{prefix}/db/lyskomd-texts The other half of the database: the actual texts. @item @var{prefix}/db/lyskomd-backup A backup copy of @file{lyskomd-data}. Never, ever delete this file unless you know what you are doing, or you may lose the entire data base. Most of the time this is the only complete database file! @end table @node DBCK Bugs @section Known Bugs @itemize @bullet @item Should have an unlock database option. @item Does not check that the data file and text file are consistent. @end itemize @c ====================================================================== @c ====================================================================== @c == == @c == SPLITKOMDB REFERENCE == @c == == @c ====================================================================== @c ====================================================================== @node splitkomdb @chapter Backups with splitkomdb The files created by @code{lyskomd} can become huge. They are more or less constantly modified, so if you do incremental backups the entire @code{lyskomd} database will be saved each time you perform a backup. The @code{splitkomdb} program improves the situation. @menu * splitkomdb basics:: How splitkomdb works. * Invoking splitkomdb:: splitkomdb reference. * splitkomdb example:: A typical setup of splitkomdb. * splitkomdb restore:: How to restore the database. @end menu @ifnottex @node splitkomdb basics @section splitkomdb theory of operation @end ifnottex The @code{splitkomdb} program attempts to split the current database in several files, some of which will not be modified the next time you run @code{splitkomdb}. An incremental backup will thus only have to save some of the files created by @code{splitkomdb}. @code{splitkomdb} can be run while @code{lyskomd} is running. It will use the last completely saved snapshot, and make a copy of it in the directory specified by the @code{Backup export directory:} parameter (default: @file{/usr/lyskom/exportdb/}). There are two recommended ways to make backups of a @code{lyskomd} database: @itemize @bullet @item Run @code{splitkomdb} before each backup, and instruct the backup program to backup @file{/usr/lyskom/exportdb}, but ignore @file{/usr/lyskom/db}. Run @samp{splitkomdb -f} before each full backup. @item Backup all the files in @file{/usr/lyskom/db}. @end itemize The first alternative needs more disk for the copy in @file{/usr/lyskom/exportdb} and is slightly more complex to set up; the second alternative saves the entire database each time you make a backup. @node Invoking splitkomdb @section Invoking splitkomdb @example splitkomdb [-c config-file] [-f] splitkomdb -v | -V @end example @code{splitkomdb} creates a splitted copy of the most recent database snapshot in the directory specified by the @code{Backup export directory:} parameter. If @samp{-v} or @samp{-V} is specified, @code{splitkomdb} prints its version number and exits. The @samp{-f} option is a hint to @code{splitkomdb} that the next backup will be a full backup. When this option is used, @code{splitkomdb} might rewrite all the output files in a way so that future invocations of @code{splitkomdb} will create a little incremental data as possible. The optional @var{config-file} argument can be used to specify the server configuration file. @xref{Server Configuration File}. @node splitkomdb example @section Typical splitkomdb setup A typical way to run @code{splitkomdb} is from @code{cron}. A @file{crontab} entry might look like this: @example 50 2 26 * * /sw/lyskom/bin/splitkomdb -f 10 3 * * * /sw/lyskom/bin/splitkomdb @end example This entry assumes that backups are started at 03:30. It gives @code{splitkomdb} 20 minutes to run. (That should be enough for most sites, but please check what is appropriate for your site.) Full backups are run on the 26th of each month. On those days, at 02:50, @samp{splitkomdb -f} is run. @node splitkomdb restore @section Restoring the database from splitkomdb The splitted format of the database is currently very simple: @table @file @item exportdb/lyskomd-texts-base.backup This file contains the first part of @code{Text file:} (default: @file{db/lyskomd-texts}). This file is created when the @samp{-f} option is given to @code{splitkomdb}. @item lyskomd-texts-tail.backup This file contains the rest of @code{Text file:}. It is always recreated when @code{splitkomdb} is run. @item lyskomd-data.backup This file contains a copy of @code{Data file:} (default: @file{db/lyskomd-data}). If @code{Data file:} file wasn't clean when @code{splitkomdb} was run, @code{Backup file:} (default: @file{db/lyskomd-backup}) will be copied instead. @end table This means that it is easy to restore the database from the splitted database. If you use the default paths, all you have to do is run these commands: @example $ cd /usr/lyskom/exportdb $ cat lyskomd-texts-base.backup \ > lyskomd-texts-tail.backup \ > > ../db/lyskomd-texts $ cp lyskomd-data.backup ../db/lyskomd-data @end example Future versions of @code{splitkomdb} may use a different format. It may even evolve into something so complex that you need a program to recreate the database. Always check the current documentation for information on how to restore the database. @c ====================================================================== @c ====================================================================== @c == == @c == LYSKOMD HACKING GUIDE == @c == == @c ====================================================================== @c ====================================================================== @node Hacking @chapter Hacking lyskomd @menu * The Database:: * Adding Configuration Parameters:: How to add configuration options. * Adding Asynchronous Messages:: Adding protocol A messages. * Adding a New Protocol Request:: Adding protocol A calls. * Adding New Input Types:: * Adding New Result Types:: * Modifying Output Types:: * Adding Aux-Item Types:: Adding predefined aux item types. * Modifying Stored Types:: Modifying types stored in the DB. * Notes:: Mixed notes. * Debugging and Testing:: How to test and debug the server. * local-to-global:: The Local_to_global structure. @end menu @node The Database @section The Database @c FIXME: ramkomd är död! Länge leve LysKOM! @c FIXME: @c FIXME: Jag har tillsammans med Inge kommit på ett sätt att dels få ner @c FIXME: väntetiden i samband med syncningar till <1 sekund, dels få ner @c FIXME: storleken på serverprocessen till mer rimliga nivåer. Denna lösning @c FIXME: lider dock av det stora problemet att den kräver dubbelt så mycket @c FIXME: diskutrymme som den egentligen behöver. Det gör även ramkomd, så det @c FIXME: är ingen försämring i det avseendet. Dock är detta bara en temporär @c FIXME: lösning i väntan på ldb. @c FIXME: @c FIXME: Varför spara allt på en gång? Varför inte spara en liten del av filen @c FIXME: i taget, och utföra några atomiska anrop mellan varje liten @c FIXME: delsynkning? Ungefär så tänkte jag när jag kom på följande schema för @c FIXME: hur man kan göra det hela bättre än det är nu. @c FIXME: @c FIXME: Den databas som ligger på fil är en ögonblickbild (snapshot) av det @c FIXME: som finns i LysKOM. Så är det i ramkomd; så blir det i diskomd. @c FIXME: (Bättre namn, någon? lyskomd tycker jag är reserverat för den version @c FIXME: som har en riktig cache&ldb.) I ramkomd skrivs allt ut på disk @c FIXME: samtidigt. I diskomd minns man bara vad som skall sparas, och sparar @c FIXME: bara en bit i taget. @c FIXME: @c FIXME: I ramkomd finns allt inne i ram-minnet (i teorin. I praktiken är det @c FIXME: mesta utswappat - något som märks varje gång det är dags att synca!). @c FIXME: I diskomd ligger det mesta på disk. I minnet finns dels det som har @c FIXME: använts nyligen, dels det som är ändrat och ännu ej syncat. Diskomd @c FIXME: har alltid minst en, ofta två, databasfiler öppna: @c FIXME: @c FIXME: Fil A Senaste kompletta fil. @c FIXME: Fil B Fil under uppbyggnad. @c FIXME FIXME: Fil Z Näst senast kompletta fil @c FIXME FIXME: (den här gick att kopiera en gång i @c FIXME FIXME: tiden, även om A inte går att kopiera @c FIXME FIXME: just nu.) @c FIXME: @c FIXME: (Dessutom textmassefilen, precis som ramkomd nuförtiden.) @c FIXME: @c FIXME: Så till detaljerna: @c FIXME: @c FIXME: Det finns tre typer av objekt som berörs av den här ändringen: @c FIXME: Text_stat, Person och Conference. Jag använder Person som ett exempel @c FIXME: nedan. @c FIXME: @c FIXME: I ram-cache.c finns en array @c FIXME: @c FIXME: Person *pers_arr[ MAX_CONF ]; @c FIXME: @c FIXME: Den byts mot @c FIXME: @c FIXME: Cache_node *pers_arr[ MAX_CONF ]; @c FIXME: @c FIXME: typedef struct cache_node @{ @c FIXME: Bool exists; @c FIXME: Bool exists_b; @c FIXME: Bool dirty; /* Är *ptr modifierad? */ @c FIXME: void *snap_shot; @c FIXME: void *ptr; @c FIXME: off_t pos; @c FIXME: off_t pos_b; @c FIXME: struct cache_node *lru_link; @c FIXME: int lock_cnt; @c FIXME: @} Cache_node; @c FIXME: @c FIXME: @c FIXME: @c FIXME: UPPSTART @c FIXME: @c FIXME: När servern startas scannar den igenom datafilen (fil A) och fyller i @c FIXME: fältet exists till TRUE/FALSE och pos till att peka på början av det @c FIXME: ställe i filen där data ligger. Övriga fält sätts till FALSE/NULL/0. @c FIXME: @c FIXME: @c FIXME: @c FIXME: CACHE_GET_PERSON @c FIXME: @c FIXME: När ovanliggande rutiner vill läsa en person händer följande: @c FIXME: @c FIXME: !exists Returnera NULL @c FIXME: ptr != NULL Lägg noden först i lru_link. Returnera ptr. @c FIXME: snap_shot != NULL Kopiera snap_shot till ptr. Lägg noden först i @c FIXME: lru_link. Returnera ptr. @c FIXME: annars Läs in från fil A, sätt ptr till den inlästa @c FIXME: structen, lägg noden först i lru_link, @c FIXME: returnera ptr. @c FIXME: @c FIXME: @c FIXME: MARK_PERSON_AS_CHANGED @c FIXME: @c FIXME: När något har ändrats sätts dirty-flaggan till TRUE. @c FIXME: @c FIXME: @c FIXME: CREATE_PERSON @c FIXME: @c FIXME: Sätt exists=TRUE, dirty=TRUE, ptr och lru. @c FIXME: @c FIXME: @c FIXME: DELETE_PERSON @c FIXME: @c FIXME: Sätt exists=FALSE. ptr=NULL. Troligtvis error om lock_cnt != 0. @c FIXME: @c FIXME: @c FIXME: @c FIXME: THROW-OUT @c FIXME: @c FIXME: För att inte diskomd ska bli för stor slängs saker ut ur cachen. @c FIXME: Algoritm: tag första elementet i lru_list. Om dirty==FALSE och @c FIXME: ptr!=NULL och lock_cnt==0 så frigör ptr. Upprepa tills antalet noder @c FIXME: med ptr!=NULL och dirty==FALSE är mindre än antalet "rena" @c FIXME: element man vill ha inne i minnet. (Smutsiga element slängs @c FIXME: aldrig ut.) @c FIXME: @c FIXME: @c FIXME: LOCK_PERSON @c FIXME: @c FIXME: Öka lock_cnt. @c FIXME: @c FIXME: @c FIXME: UNLOCK_PERSON @c FIXME: @c FIXME: Minska lock_cnt. @c FIXME: @c FIXME: @c FIXME: PRE-SYNC @c FIXME: @c FIXME: Utsparningen till fil sker i tre steg. Först sveper man över alla @c FIXME: Cache_noder. För alla som har dirty=TRUE gör man följande: @c FIXME: @c FIXME: if ( lock_cnt == 0 ) @{ @c FIXME: snap_shot = ptr; (Pekartilldelning, ej kopiering.) @c FIXME: ptr = NULL; @c FIXME: Ta bort ptr ur lru-kedjan. @c FIXME: @} else @{ @c FIXME: snap_shot = copy(ptr); @c FIXME: @} @c FIXME: @c FIXME: dirty = FALSE; @c FIXME: @c FIXME: För _alla_ noder görs dessutom följande: @c FIXME: @c FIXME: b_exists==exists; @c FIXME: @c FIXME: @c FIXME: SYNC @c FIXME: @c FIXME: Steg två utförs en liten bit i taget. Till exempel så skulle @c FIXME: man kunna spara en person efter varje atomiskt anrop, eller så. @c FIXME: @c FIXME: b_exists==FALSE? Sätt pos_b. Skriv "@@\n" till fil B. @c FIXME: Är snap_shot != NULL? Sätt pos_b. Skriv ut innehållet i snap_shot @c FIXME: till fil B. @c FIXME: dirty==FALSE && ptr!=NULL Skriv ut innehållet i ptr till fil B. @c FIXME: annars: Kopiera från fil A till fil B. (Eftersom man @c FIXME: vet både var blocket börjar och slutar kan @c FIXME: man kopiera blocket utan att bry sig om vad @c FIXME: som står i det -> väldigt lite CPU går åt). @c FIXME: @c FIXME: @c FIXME: @c FIXME: @c FIXME: POST-SYNC @c FIXME: @c FIXME: När alla Person:er har hanterats som i SYNC ovan är det dags för det @c FIXME: tredje steget. Då går man igenom alla Cache_node:er och gör följande: @c FIXME: @c FIXME: pos = pos_b; @c FIXME: file_b = FALSE; @c FIXME: free(snap_shot); @c FIXME: snap_shot = NULL; @c FIXME: @c FIXME: Fil B som man förut hade öppen för skrivning öppnar man i stället för @c FIXME: läsning som fil A. @c FIXME: @c FIXME: @c FIXME: @c FIXME: @c FIXME: ANMÄRKNINGAR @c FIXME: @c FIXME: Innehållet i snap_shot är alltid "smutsigt" jämfört med innehållet i @c FIXME: fil A. Det som snap_shot pekar på finns aldrig med i lru-kedjan. @c FIXME: @node Adding Configuration Parameters @section Adding Configuration Parameters Make sure that you really understand what you want to configure. Think it over again. Find a good, descriptive name for it. Decide what values the parameter can be set to. Integers? Booleans? Document the parameter in @file{lyskomd.texi}. Add a field to @code{struct kom_par} in @file{param.h}. Add it to @code{parameters[]} in @file{server-config.c}. See @file{conf-file.h} and maybe @file{conf-file.c} for information on this structure. Make sure that the parameter is used instead of any previous hard-coded value. Make sure that @code{dbck} can cope with it. @node Adding Asynchronous Messages @section Adding Asynchronous Messages @table @asis @item @file{async.h} Add the message in @code{typedef enum @{ @} Async}. Make sure that @code{ay_dummy_last} is one more than any other message. If the message is to be sent by default, which is @emph{not} recommended, place its number into @code{ASYNC_DEFAULT_MESSAGES}. @item @file{prot-a-send-async.c} @itemx @file{prot-a-send-async.h} Write a function that sends the message. This function is responsible for writing the message to a particular connection and for ensuring that the message is not sent to clients who do not want it. Make sure that the second argument to @code{async_header} really is the number of elements being sent. Arrays count as two elements: the item count and the elements. @item @file{send-async.c} @itemx @file{send-async.h} Write a function that sends the message to appropriate clients. This function is responsible for checking that async messages should be sent at all, for each client check if it allowed to see the message and ensure that the protocol specified by the connection is appropriate. The send function should either take a @code{struct connection *} as an argument and send the message to that connection, or loop over all connections. Most send functions take a connection pointer; the looping is dealt with elsewhere. @item Make sure that the message is sent in appropriate places. @item Document the message type in @file{Protocol-A.texi}. @end table @menu * Function Templates for prot-a-send-async.c:: * Function Templates for send-async.c:: @end menu @node Function Templates for prot-a-send-async.c @subsection Function Templates for prot-a-send-async.c This is what a typical function in @file{prot-a-send-async.c} should look like. This function is responsible for checking that the client is accepting the message and writing the message itself to the connection. @example void prot_a_async_@var{something}(Connection *cptr, @var{parameters}) @{ ASYNC_CHECK_ACCEPT(cptr, ay_@var{something}); async_header(cptr, @var{num_tokens}, ay_@var{something}); /* Output the body of the message */ async_trailer(cptr); @} @end example @node Function Templates for send-async.c @subsection Function Templates for send-async.h This is what a typical function in @file{send-async.c} should look like. This function is responsible for sending the message to all connections that are appropriate, not sending it if the server is not supposed to send messages at all, and for checking that the protocol specified by the client is one the server knows. @example void async_@var{something}( @var{parameters} ) @{ Connection *cptr; Session_no i = 0; if (!param.send_async_messages) return; while ((i = traverse_connections(i)) != 0) @{ cptr = get_conn_by_number(i); switch(cptr->protocol) @{ case 0: /* No protocol specified yet */ break; case 'A': /* Check that connection is logged on. We might want to check other things here too, such as if the connection is allowed to see the message */ if (cptr->username_valid == TRUE) prot_a_async_@var{something}(cptr, @var{parameters}); break; default: restart_kom("async_@var{something}(): bad protocol.\n"); break; @} @} @} @end example Template for a function that sends to a single connection: @example void async_@var{something}(struct connection *cptr, @var{parameters}) @{ if (!param.send_async_messages) return; switch(cptr->protocol) @{ case 0: /* No protocol specified yet */ break; case 'A': /* Check that connection is logged on. We might want to check other things here too, such as if the connection is allowed to see the message */ if (cptr->username_valid == TRUE) prot_a_async_@var{something}(cptr, @var{parameters}); break; default: restart_kom("async_@var{something}(): bad protocol.\n"); break; @} @} @end example @node Adding a New Protocol Request @section Adding a New Protocol Request Before doing anything, think again. Make sure that the protocol request is needed, is in line with the rest of the protocol, behaves the way people want it to, and that everyone involved agrees that it is a good idea. @enumerate @item Document the request in @file{Protocol-A.texi} @item Declare the function in @file{include/services.h} @item Declare the function @emph{last} in @file{server/fncdef.txt}. It should be given a call number one higher than the currently existing highest contiguous call number. @item If the function takes an input parameter of a new type, changes need to be made in several places. @xref{Adding New Input Types}. @item If the function takes too many parameters of type @code{num}, @code{string} or @code{c_string}, the definition of @code{Connection} in @file{server/connection.h} has to be changed. @item If the function has an output parameter of a new type, changes need to be made in several plaves. @xref{Adding New Result Types}. @item Write the function in a suitable place in the server directory. @item Write tests for the new function in @file{server/testsuite/lyskomd.0}. Write one file for testing the functionality. Write tests in @file{01.exp} (behavior when the client is not logged on) and @file{03.exp} (normal behavior.) @item Run the testsuite to make sure nothing old has been broken. @end enumerate Every request handler should call the @code{CHK_CONNECTION} macro before doing anything else. This ensures that the @code{active_connection} variable is non-NULL. The only time when this might not be the case is if the request handler is not called in response to a client request. This should never happen, but might if someone gets careless. If your function should not be available before the user is logged in, call the CHK_LOGIN macro after doing @code{CHK_CONNECTION}. Take care returning errors to the client. Previous versions of @samp{lyskomd} leaked secret information through error returns. For example, the following code leaks information: @example Success service(Conf_no conf_no, Text_no text_no) @{ Conference *conf_stat; Text_stat *text_stat; CHK_LOGIN(FAILURE); GET_C_STAT(conf_stat, conf_no, FAILURE); GET_T_STAT(text_stat, text_no, FAILURE); if (access_perm(conf_no, conf_stat, active_connection <= none)) @{ err_stat = conf_no; kom_errno = KOM_UNDEF_CONF; return FAILURE; @} if (!text_read_access(active_connection, text_no, text_stat)) @{ err_stat = text_no; kom_errno = KOM_NO_SUCH_TEXT; return FAILURE; @} /* Permissions checked. Do the deed. */ return OK; @} @end example This request can be used to gain precise information on which conferences and texts exist in the system. If an unprivileged user makes a request for any conference and readable text, and the user receives a @samp{KOM_NO_SUCH_TEXT} error, the user can deduce that the conference exists, but is secret. If the user makes a request for a conference known to be secret and a text known not to be readable (either secret or deleted), and the user receives a @samp{KOM_UNDEF_CONF} error, the user can deduce that the text does exist. To avoid traps like these, do permission checks for objects immediately after attempting to get them from the database. See also: @itemize @bullet @item @ref{Adding New Input Types} @item @ref{Adding New Result Types} @item @ref{Modifying Output Types} @end itemize @node Adding New Input Types @section Adding New Input Types Changes need to be made in the following files: @table @file @item Protocol-A.texi Document the new type. @item server/call-switch.awk The new type has to be added to the cascaded ifs that translate the type name to code that points to the appropriate field in a @code{Connection} structure. @item server/prot-a-parse-arg-c.awk The new type has to be added to the cascaded ifs that create the argument list parser. @item server/connections.h The definition of @code{Connection} must be extended with a field where the parse value can be stored. Don't even think about trying to reuse an existing field. It's more trouble than it's worth. @item server/connection.c Free the contents of the field in @code{free_parsed}. @item server/prot-a.c Free the contents of the field in @code{prot_a_destruct}. @item server/internal-connections.c Initialize the contents of the field in @code{init_connection}. @end table @node Adding New Result Types @section Adding New Result Types Changes need to be made in the following files: @table @file @item Protocol-A.texi Document the new type. @item server/prot-a.c Add a line in the @code{prot_a_reply} switch that calls the correct output function. @item server/connections.h Add the type in @code{Res_type}and @code{Result_holder}. @item server/prot-a-output.c @itemx server/prot-a-output.h Write a function that outputs the new type to a connection. Use the existing functions as templates. @end table @node Modifying Output Types @section Modifying Output Types When you modify an existing type you have to rename the old version of the type since it will still be used in existing calls. The convention hs previously been to rename @var{Something} to @var{Something}@samp{_old}, but the preferred method is to append an underscore and the protocol version in which the current version of the type was introduced. For example, if the type @samp{Gazonk} was introduced in protocol version 11, and a new version is to be introduced in protocol version 15, the current @samp{Gazonk} structure is renamed to @samp{Gazonk_11}. This is to avoid names like @samp{Something_older}, @samp{Something_oldest} and @samp{Something_Truly_Ancient}. Changes need to be made to the following files: @table @file @item Protocol-A.texi Document the new type in the appropriate section. Rename the existing type in the type documentation and in all calls that return it. Be thorough! @item fncdef.txt Rewrite all calls that use the modified type so they use the old version of the type. @item prot-a.c Modify the current line in @code{prot_a_reply} for the existing version of the type, and add a new line for the new version of the type. @item connections.h Modify the existing entry in @code{Res_type} and @code{Result_holder}, if necessary. This should only be necessary if the server uses both a new and old type internally, which is not recommended. Add new entries for the new version of the type. @item prot-a-output.h @itemx prot-a-output.c Rename the existing output routing according to the new name of the type. Write a new output routine for the new version of the type. @item memory.c If there are functions for the type in @code{memory.c}, make sure that your new type is initialized, cleared and copied in an appropriate manner. @end table If the type you modify is stored in the database, make sure it gets saved properly. @xref{Modifying Stored Types}. @node Adding Aux-Item Types @section Adding Aux-Item Types @enumerate @item Document the new type in Protocol-A.texi @item Write a definition of the new type in @file{run-support/aux-items.conf}. @item Some tests in at least @file{server/testsuite/lyskomd.0/01.exp}, @file{server/testsuite/lyskomd.0/03.exp} and @file{server/testsuite/lyskomd.0/18.exp} will fail when new predefined aux-items are added. Fix the tests. @item Write test cases for the new aux-item. If the aux item can be set on a letterbox, do so in @file{server/testsuite/lyskomd.0/03.exp} where the comment containing @samp{AUXITEM} says to do so. More complex aux-items should have more tests written for them. @file{server/testsuite/lyskomd.0/20.exp} might provide some inspiration. @item If the new type requires add, delete or undelete triggers that do not already exist, declare the trigger functions in @file{aux-items.c} and add them to the @code{aux_item_triggers} array in the same file. @item If the new type is so complex that is cannot be fully defined in @file{aux-items.conf}, then add it to the @code{compiled_aux_items} array in @file{aux-items.c}. Note that this functionality has not been tested until someone actually adds one of these beasts, so watch your step. @end enumerate @node Modifying Stored Types @section Modifying Stored Types If you want to modify an existing type that is stored in the database, think again. Can the job be done with aux-items instead? Is it really necessary? Be very, very careful when doing this. You have to make sure that the type as sent in old calls and async messages is not changed in any way. You have to make sure that the type can be stored to and read from the database. @enumerate @item Document the changes in Protocol-A.texi if the change is visible in the protocol. @item Bump the database version number by one for the next release of the server. @item Write a function in @file{ram-output.c} to output the new format. Update all old functions in @file{ram-output.c} that are database version dependent so that they can deal with the new database format. @item Write a function in @file{ram-parse.c} to read the new format. Update all old functions in @file{ram-parse.c} that are database version dependent so that they can deal with the new database format. @item Set the default database format in @file{ram-parse.c} and @file{ram-output.c}. The variables to change are @code{input_format} and @file{output_format}, respectively. @item Don't forget to update the functions in @file{memory.c} @end enumerate @menu * Template for ram-output.c:: * Template for ram-parse.c:: @end menu @node Template for ram-output.c @subsection Template for ram-output.c For types that can be output in several different formats, use the following templates for them. You have to be able to output in all formats, or @code{dbck} will be unable to convert between formats. @example static void foutput_@var{something}_0(FILE *fp, @var{something} *o) @{ /* Output version 0 of @var{something} */ @} static void foutput_@var{something}_1(FILE *fp, @var{something} *o) @{ /* Output version 1 of @var{something} */ @} static void foutput_info_2(FILE *fp, @var{something} *o) @{ /* Output version 2 of @var{something} */ @} void foutput_@var{something}(FILE *fp, @var{something} *o) @{ switch(output_format) @{ case 0: foutput_@var{something}_0(fp, info); break; case 1: foutput_@var{something}_1(fp, info); break; case 2: foutput_@var{something}_2(fp, info); break; default: restart_kom("unknown database format: %d", output_format); break; @} @} @end example Note that if two versions are the same, only write one functions. For example, if version 0 and version 1 are the same, only write an @code{foutput_@var{something}_0} function and call it from both case 0 and case 1. @node Template for ram-parse.c @subsection Template for ram-parse.c @example static Success fparse_@var{something}_0(FILE *fp, @var{something} *o) @{ /* Parse version 0 */ return OK; @} static Success fparse_@var{something}_1(FILE *fp, @var{something} *o) @{ /* Parse verson 1 */ return OK; @} static Success fparse_@var{something}_2(FILE *fp, @var{something} *o) @{ /* Parse verson 2 */ return OK; @} extern Success fparse_@var{something}(FILE *fp, @var{something} *o) @{ if ( fparse_long_errors != 0 ) @{ log("fparse_info(): fparse_long_errors == %d on entry. Reset.\n", fparse_long_errors); fparse_long_errors = 0; @} switch (input_format) @{ case 0: return fparse_@var{something}_0(fp, o); break; case 1: return fparse_@var{something}_1(fp, o); break; case 2: return fparse_@var{something}_2(fp, o); break; default: restart_kom("unknown input format: %d\n", input_format); return FAILURE; break; @} @} @end example Note that if two versions are the same, only write one functions. For example, if version 0 and version 1 are the same, only write an @code{fparse_@var{something}_0} function and call it from both case 0 and case 1. @node Notes @section Hacking Notes @menu * Parsing Bit Fields:: How to parse bit fields properly. * Membership Notes:: How members and memberships are handled. * Linking Pairs of Aux Items:: How to link pairs of aux items. * Notes for fncdef.txt:: Format of the fncdef.txt file. * Traversing Connections:: How to traverse connections in lyskomd. @end menu @node Parsing Bit Fields @subsection Parsing Bit Fields The parser for a bit field parameter type should be very tolerant of the length of the token. Anything from a single bit and up should be permitted. The parser should use default values for bits that are not provided and ignore extra bits. Here is a model function: @example void prot_a_parse_bitfield(Connection *client, Bitfield *res) @{ String token; String_size len; token = prot_a_get_token(client); len = s_strlen(token); if (len <= 0) longjmp(parse_env, ISC_PROTOCOL_ERR); init_bitfield(res); switch (len = s_strlen(token)) @{ default: case 8: res->bit_8 = token.string[7]; case 7: res->bit_7 = token.string[6]; case 6: res->bit_6 = token.string[5]; case 5: res->bit_5 = token.string[4]; case 4: res->bit_4 = token.string[3]; case 3: res->bit_3 = token.string[2]; case 2: res->bit_2 = token.string[1]; case 1: res->bit_1 = token.string[0]; @} @} @end example The function gets the token, checks the sanity of the length, then initialized the result to its default values. Then it does a switch on all token lengths that are equal to or smaller than the number of bits the server knows about. The fall-through ensures that all bits in the token are read. @node Membership Notes @subsection Membership Notes The @code{position} field in the membership is @emph{not} stored. It has to be set every time a membership is requested for transmission to the client. @node Linking Pairs of Aux Items @subsection Linking Pairs of Aux Items Sometimes two aux items need to work in tandem. The first instance of this was the FAQ and FAQ-for-conference items. The FAQ item contains the text number of a text that is a FAQ for a conference. The FAQ-for-conference item contains the conference for which a text is a FAQ. This is needed so that deletion of the text properly removes the aux-item on the conf (plus, it's nice to be able to see that a text is a FAQ.) The @code{linked_item} field in the Aux_item structure is for linking items. The linking must be managed through the use of triggers. This field is not visible in the protocol. It is saved in the database. It is not possible to have more than one link per item. Please remember the following points. @itemize @bullet @item The target of a link should have a link back. All links need to go both ways. @item In the add trigger for one end, create the other end of the link and set the @code{linked_item} field in both items. Don't forget to mark the objects at both ends as changed. @item Deletion and undeletion of the other side of the link will be managed automatically. You don't need delete and undelete triggers simply to destroy the other side of a link. @item Don't kill the server because one end is missing. It is possible for the administrator to remove an item manually. Log a message and continue working. @end itemize @node Notes for fncdef.txt @subsection Notes for fncdef.txt The fncdef.txt file is used to define the RPC functions. Each line consists of the call number, the return type of the call, the parameters and the output types of the call. Some examples: @example 10 number create_conf_old c_string (param.conf_name_len) conf_type 12 success lookup_name c_string (param.conf_name_len) : conf_list @end example The first line defines a call named @code{create_conf_old} that takes two arguments, a string that can only be as long as @code{param.conf_name_len} and a @code{conf_type}. It returns a number to the client. If the service call returns -1, the server will return an error. The @code{create_conf_old} call has RPC number 10. The second line defines a call named @code{lookup_name} that takes a string argument that can be no longer than @code{param.conf_name_len}, and returns a @code{conf_list}. The service call returns a @code{Success}. If it does not return @code{OK}, the server will return an error. The @code{lookup_name} call has RPC number 12. @subsubsection Scripts That Use fncdef.txt The following scripts operate on @file{fncdef.txt}. If you make modifications to the format of @file{fncdef.txt}, you have to update these scripts. @table @file @item call-switch.awk Generates @file{call-switch.incl}, which is included by @file{connections.c} @item com-h.awk Generates @file{com.h}, which is included by several files. @item fnc-def-init.awk Generates @file{fnc-def-init.incl}, which is included by @file{connections.c}. @item prot-a-is-legal-fnc.awk Generates @file{prot-a-is-legal-fnc.incl}, which is included by @file{prot-a.c} @item prot-a-parse-arg-c.awk @itemx prot-a-parse-arg-h.awk Generates @file{prot-a-parse-arg.c} and @file{prot-a-parse-arg.h}. @end table @node Traversing Connections @subsection Traversing Connections Since session 0 is interpreted as the currently active session by get_conn_by_number it is important to be careful when traversing sessions. Code like this does not work since it will do one iteration through the loop with @code{sess} set to zero. This formerly caused @code{get_conn_by_number} to return @code{NULL}, but now causes it to return the session pointer for the current session. @example for (sess = 0; (sess = traverse_connections(sess)) != 0; ) @{ cptr = get_conn_by_number(sess); ... @} @end example The canonical traversal code looks like this: @example Session_no session = 0; while ((session = traverse_connections(session)) != 0) @{ cptr = get_conn_by_number(session); ... @} @end example This code has @code{session} set to a session number before ever entering the loop. @node Debugging and Testing @section Debugging and Testing We're slowly adding support for debugging and testing lyskomd properly. @menu * The Test Suite:: The lyskomd regression test suite. * Configuration Options:: Debugging options for the configure script. * Coverage Testing:: How to do coverage testing with gcov. * Debug Calls:: Special protocol A calls for testing. @end menu @node The Test Suite @subsection The Test Suite The lyskomd test suite is in src/server/testsuite. Please extend this with additional test cases every time you make modifications to the server. Run the test suite often to make sure that your changes did not break anything. The file config/prot-a.exp contains some support for protocol A. Don't use these functions in test cases. Use them to set up the inital database and for things that have to be done, such as logins and enabling privileges, but that don't need to be tested. Also, don't count on all the code in prot-a.exp to be fully functional. Add new functions to this file as you see fit. The basic structure of a test case is the following: @example source "config/prot-a.exp" read_versions lyskomd_start client_start 0 talk_to client 0 kom_connect "DejaGnu test suite" @i{The test cases} talk_to client 0 kom_logout kom_login 5 [holl "gazonk"] 0 kom_enable 255 send "9999 44 0\n" simple_expect "=9999" client_death 0 lyskomd_death @end example Use the existing test cases as templates. @node Configuration Options @subsection Configuration Options There are several testing and debugging-related configuration options for lyskomd. Some of them also apply to libisc. @table @samp @item --with-purify Build lyskomd with Purify. This currently does not work. @item --with-efence Build lyskomd with Electric Fence for checking buffer overruns. This option does work. @item --with-checker Build lyskomd with Gnu Checker for checking memory accesses, leaks, file descriptors and all kinds of stuff. As of Checker version 0.99.6, Gnu Checker cannot deal with lyskomd. Once built, and this requires modifications to Checker (at least on Linux) it reports spurious errors. Still, the option is here for those who want to try it out. @item --with-gcov Build lyskomd with instrumentation for @code{gcov}. You have to use this option if you want to run @code{gcov} on lyskomd. For @code{gcov} to be effective, you should turn off optimization as well. @item --with-traced-allocations There is some builtin support for detecting memory leaks in lyskomd. Whenever the server exits normally it reports how much memory it still uses to @file{etc/memory-usage}. The count should always be 0. If there is a leak you can use this option to trace it down. See @file{src/server/ram-smalloc.c} for more information. You need gdb and a lot of time to use this option. @item --with-optimization=@var{value} Build lyskomd with the specified level of optimization. Use either numeric values to select the level of optimization, or say @samp{--with-optimization=no} or @samp{--without-optimization} to turn optimization off. @end table @node Coverage Testing @subsection Coverage Testing When you write new code, make sure that it is completely covered by the test suite. Run the lyskomd configure script with the @samp{--with-gcov}, @samp{--with-debug-calls} and @samp{--without-optimization} flags to instrument the server for coverage testing with gcov. If you run configure without the @samp{--without-optimization} option, the server will be compiled with optimizations on. This is fine, but the coverage data from gcov isn't completely reliable since parts of the program may have been optimized out of existance. Recompile everything, then run the test suite. Next do @samp{gcov -f @var{filename}} to compute coverage information for the file @var{filename}. The resulting file @var{filename}@samp{.gcov} shows which lines have been executed, and which haven't been run. Try to get 100% coverage. @node Debug Calls @subsection Debug Calls Run the configure script with @samp{--with-debug-calls} to compile in support for debugging calls in the server. These calls are strictly for making testing easier (or possible.) They are not official, and they may change at any time. @menu * memory-info:: Get information from malloc (1000) * set-marks:: Set the number of marks on a text (1001) * backdate-text:: Change the creation date of a text (1002) @end menu @node memory-info @subsubsection memory-info (DEBUG) Experimential @findex memory-info @example memory-info [1000] ( ) -> (( arena : INT32; ordblks : INT32; smblks : INT32; hblks : INT32; hblkhd : INT32; usmblks : INT32; fsmblks : INT32; uordblks : INT32; fordblks : INT32; keepcost : INT32; )); @end example This call returns the data returned by @code{mallinfo} in the server. See the man page for @code{mallinfo} for explanations of the fields. @node set-marks @subsubsection set-marks (DEBUG) Experimental @findex set-marks @example set-marks [1001] (( text-no : Text_no; no-of-marks : INT32; )) -> ( ); @end example Set the number of marks on text @code{text-no} to @code{no-of-marks}, regardless of how many marks it really has. This call is useful for forcing the database into a state where the number of marks is incorrect in some way. @node backdate-text @subsubsection backdate-text (DEBUG) Experimental @findex backdate-text @example backdate-text [1002] (( text-no : Text_no; seconds : INT32; )) -> ( ); @end example Backdate a text in the server. Change the creation date of text @code{text-no} so it appears to have been created @code{seconds} earlier than it was actually created. This can be used to test the garbage collector. @node local-to-global @section The local-to-global structure The data structure that stores the mapping from local to global text numbers is currently one of the more advanced structures used by lyskomd. @ignore @c FIXME: Translate this @subsection Background The Det sätt som textnummer läggs till har ett antal egenskaper: - Texter läggs hela tiden på bakifrån, aldrig i mitten eller i början. - Numren på de lokala textnumren är konsekutiva, dvs inga hål finns. Sådana hål kan dock uppstå (och uppstår!) när texter tas bort. Lösning ======= Det första man ser när man analyserar innehållet i en mapp, är att det finns långa avsnitt av idel nollor, och långa avsnitt där det inte finns några nollor alls, eller åtminstone väldigt få. Detta antyder alltså att man bör ha en adaptiv datastruktur som anpassar sig till det lokala förhållanden. Vi föreslår alltså följande. Mappen lagras i block (små arrayer). Det finns två sorters block: 1. Glesa block. Glesa block består egentligen av två block. I det ena blocket ligger nycklar (Local_text_no) och i det andra blocket ligger data (Text_no). Inom ett block använder man binärsökning i det ena blocket för att hitta just den Local_text_no man är ute efter. 2. Täta block. Täta block består av ett enda block som innehåller data (Text_no). Man vet vilket lokalt textnummer det första entryt svarar mot. Det kan finnas enstaka lokala nummer i ett tätt block som inte finns -- då innehåller data 0. Blockstorleken är fixerad till t ex 100 entries. (Det verkar som om man tjänar nästan exakt samma antal bytes oavsett om man väljer blockstorlek 50 eller 1000). Ett fullt tätt block innehåller alltid exakt 100 lokala textnummer. Ett fullt glest block innehåller alltid 100 existerande globala textnummer. (Ett glest block tar dubbelt så mycket plats som ett tätt block, eftersom ett glest block ju egentligen består av både ett nyckelblock och ett värdeblock). För att hålla reda på sina block har man en array av block_info: typedef struct block_info { int first_free; int zeroes; Local_text_no start; Local_text_no * key_block; Text_no * value_block; } L2g_block_info; Om key_block == NULL så är det ett tätt block. first_free Fältet first_free visar var i blocket som man kan fylla på med fler värden. Det är 100 för fulla block. För block som inte är fulla pekar det ut det entry i value_block som nästa värde ska hamna i. Det gäller t ex det sista blocket, som fylls på allt eftersom nya inlägg skickas till mötet eller block där texter har tagits bort. zeroes Fältet zeroes används bara för täta block, och räknar antalet nollor i blocket. Om zeroes blir större än 50% av blockstorleken gör man om blocket till ett glest block. Fältet zeroes är en optimering som troligtvis underlättar ihopslagning av block. Det är möjligt att den inte behövs. start Fältet start innehåller numret på det första lokala textnumret i blocket. key_block Fältet key_block är en pekare till blocket med Local_text_no, dvs nycklarna i blocket. Detta fält är NULL om detta är ett tätt block. value_block Fältet value_block är en pekare till blocket med Text_no, dvs värdena i blocket. Förutom detta behövs en struct per möte som håller reda på arrayen med block: typedef struct local_to_global { int num_blocks; int block_size; L2g_block_info * blocks; } Local_to_global; @end ignore @c ====================================================================== @c ====================================================================== @c == == @c == LYSKOMD DATABASE FORMAT == @c == == @c ====================================================================== @c ====================================================================== @ifinfo @node lyskomd Database Specification @chapter lyskomd Database Specification This document specifies the format of the lyskomd database files. The specification is currently incomplete. Only the structure, not the actual data records are documented. @menu * Version 0:: Database used with early versions of lyskomd. * Version 1:: Database used with lyskomd 1.9.0. * Version 2:: Database used with lyskomd 2.0.0. @end menu @end ifinfo @node Version 0 @section Data File Version 0 Version 0 was used by lyskomd versions up to 1.8. @example database : header @II{NL} next-free-num @II{NL} confs persons next-text-num @II{NL} texts ; header : @BB{CLEAN} | @BB{DIRTY} ; next-free-num : @II{INTEGER} ; confs : confs conf | @II{empty} ; conf : empty-record | @BB{+} conf-record @II{NL} ; persons : persons person | @II{empty} ; person : empty-record | @BB{+} person-record @II{NL} ; next-text-num : @II{INTEGER} ; texts : texts text | @II{empty} ; text : empty-record | @BB{+} text-record @BB{NL} ; empty-record : @BB{@@} @BB{NL} ; @end example The number of person and conference records is exactly one less than @II{next-free-num}. The number of text records is exactly one less than @II{next-text-num}. Records are stored sequentially. Conference number 18 is the 18th conference record in the file. This implies that deleted records must be stored as @II{empty-record} records. @II{next-text-num} is the number of the highest text. There are exactly one less than this number of text records in the database. @II{next-free-num} is the number of the highest conference. There are exactly one less than this number of both person and conference records. This implies that if conference N is not a letterbox, then person record N will be an @II{empty-record}. If the header says ``CLEAN'', the database file is complete. If the header says ``DIRTY'', the server has not finished writing it completely. @node Version 1 @section Data File Version 1 Version 1 was used by lyskomd version 1.9. @example database : header @BB{NL} @BB{records} ; header : @BB{CLEAN:00001} | @BB{DIRTY:00001} ; records : records record | @II{empty} ; record : next-free-num | next-text-num | conference | person | info | text | deleted ; next-free-num : @BB{#C} @II{INTEGER} ; next-text-num : @BB{#P} @II{INTEGER} ; conference : @BB{C} @II{integer} conf-record @BB{NL} | @BB{P} @II{integer} person-record @BB{NL} | @BB{T} @II{integer} text-record @BB{NL} | @BB{I} info-record @BB{NL} ; deleted : @BB{-C} @II{integer} @BB{NL} | @BB{-P} @II{integer} @BB{NL} | @BB{-T} @II{integer} @BB{NL} ; @end example The integer in the conference, text and person records is the ID of the record. This implies that records can be in any order. The @II{next-free-num} record is used to store the next available ID for conferences in the system. There may be several of these records in the database. The @II{next-text-num} record is used to store the next available ID for texts in the system. There may be several of these records in the database. A conference or text must have a number lower than the closest @II{next-free-num} or @II{next-text-num} preceding it. The deletion records are used to indicate that an object found earlier in the database has been deleted. The implementation of these in lyskomd 1.9 does not work, and they are not used. The @II{-C} record indicates deletion of a conference. The @II{-P} record indicates deletion of a person. The @II{-T} record indicates deletion of a text. The integer in the deletion record is the ID of the object being deleted. @node Version 2 @section Data File Version 2 Version 2 is used by lyskomd version 2.0. The structure of the data file is similar to version 1. The header has been extended with a timestamp contaning the time when the database file was saved. This timestamp consists of twenty characters, the number of seconds since 00:00:00 GMT January 1, 1970 (a Unix @code{time_t}.) @example database : header @BB{NL} @BB{records} ; header : @BB{CLEAN:00001} @BB{NL} timestamp @BB{NL} | @BB{DIRTY:00001} @BB{NL} timestamp @BB{NL} ; timestamp : digit digit digit digit digit digit digit digit digit digit digit digit digit digit digit digit digit digit digit digit ; ; digit : @BB{0} | @BB{1} | @BB{2} | @BB{3} | @BB{4} | @BB{5} | @BB{6} | @BB{7} | @BB{8} | @BB{9} ; @end example Furthermore several data types have been changed to accommodate additions introduced in version 10 of protocol A. The @II{server-info}, @II{conf-record} and @II{text-record} include information about aux-items (highest-aux-no and aux-item-list.) The @II{conf-record} contains the expire field added to the conf-stat structure. The @II{conf-record} and @II{person-record} records use the new local-to-global structure for storing maps. @contents @bye