TODO

This file attempts to list the things that need to be done on this
server.

Before the next release
=======================

* Showstoppers

** Add a checksum on each object in the database.

** The text-mass file should contain some redundancy. At a minimum, it
   should store the text number and text length of each text, so that
   there is absolutely no risk that the wrong text is displayed. (That
   has actually happened more than once due to faulty hardware or
   operating systems--we should do better.)

** Documentation Issues

*** Remove the text "In all likelihood, the implementation of this flag 
    is screwed up." from lyskomd.texi after checking that
    supervisor-only works properly.

*** Move dbck.texi, hacking.tex and lyskomdb.texi into lyskomd.texi.  
    Two reasons: there are too many manuals, and automake apparently
    doesn't allow more than one file to include version.texi.

*** Document updateLysKOM in lyskomd.texi.
    There should be an "Invoking updateLysKOM" node.
    updateLysKOM should use the "Status file" parameter (and other
    parameters). (DONE)
    Use parameters in src/server/updateLysKOM.c`savecore().
    I will do this --ceder

*** Document komrunning in lyskomd.texi.
    There should be an "Invoking komrunning" node.
    komrunning should be rewritten i C; it only works with BSD-style ps
    programs.
    I am doing this --ceder

*** Fix doc/lyskomdb.texi (Version 1) (see FIXME comments).

*** Dump all little doc files into the texinfo manuals.
    I will do this --DCB

*** Fix the node structure of Protocol-A.texi (3228867).
    I will do this --DCB

*** Document 104=map_created_texts
    I will do this --ceder

*** Update/rewrite README.

        Introduction: 
        Chapter 1 (Requirements): Needs update.
        Chapter 2 (Installing, Quick Version): Needs minor revision.
                   Refers to doc/ADMINISTRATION
        Chapter 3 (Database versions): DONE.
        Chapter 4 (Building): Needs major revisions.
        Chapter 5 (Insallation): Needs writing.

*** The protocol revision section of Protocol-A.info is incomplete.
    I will do this --ceder


** Build Issues

*** Check that texinfo.tex is included in a proper place by automake.
    I will do this --ceder
    Update 1999-04-04: It isn't.  I've written a patch to automake
    that seems to fix the problem and sent it to the maintainers.
    We will probably have to use the patched makeinfo to make the
    release.  --ceder


** Testing

*** Test validation of regexps containing groups. Try "X\\(YZ\\)?X".
    I will do this. --DCB

*** Test that "% No connections left" works.

** Known Bugs

*** Asynchronous messages may not be censored enough. Check all
    messages that contain conf-nos.



** Improvements

*** Use libisc 1.0.
    I will do this --ceder

*** Get rid of mux_printf and doc/mux.proto et c.
    I might do this --ceder

** Fix proper ctor/dtor semantics of everything in memory.c.  _clear
   is an abomination.

** Check the FIXME in cached_conf_exists. I am certain that the
   necessary changes to cached_delete_conf are done, and almost sure
   that no other changes are required (checked where s.exists is set
   to zero for existing confs, and that is only in init_cache and
   cached_delete_conf. See ChangeLog entry for Sat Sep 13
   (simple-cache.c). -- DCB

   But: we found problems with this 1999-03-28, which DCB fixed in his
   copy of the code.  Committed yet?

* High priority, but they can wait until after the next release.

** Actually remove all man pages.  This must wait till after the next release.

** We have to be able to limit creation of certain aux-items to users
   with special privileges. Most of the import/export items need
   this. Put in a new flag in the aux-item-definition file, add it to
   the parser, check it in aux_item_check_add_perm or whatever the
   hell the function was named. Test it. The biggest problem is to
   figure out what permissions should be needed. I suppose it's
   possible that we could let the user specify level and bits in the
   aux-items.conf file, but that would mean mucking more with the
   parser.
   I will do this. --DCB

** Fixa s{kerhetsbuggen vid read_texts et c. Det b|r vara s} att man
   inte f}r veta mappningen s} ofta som nu. Vilka texter som {r l{sta
   b|r ocks} vara mer hemligt. Ett nytt anrop, query_unread(), b|r
   inf|ras. Resultatet {r antalet ol{sat brev och _kanske_
   Local_text_no f|r det h|gsta nummret. 

   (Det var l{nge sen jag skrev det h{r, och jag minns inte riktigt
   vad jag menar, men man borde nog titta p} det h{r s} sm}ningom).


** param.maxqueuedsize and other ISC configuration is ignored.

** Transaction logging, so that we never lose any data.

** dbck should check that all creation_times of Texts are consecutive,
   and that all local add-times of texts in confereces are
   consecutive.

** lyskomd should log a warning if it is travelling backwards trough
   time.

** Fix Nisse's text garbing procedure. See doc/IDEAS for details.
   
** Proof-read Protocol-A.texi.

** Consider implementing aux-infos that are the "reverse" of a Mark,
   of the aux-info faq-text, of a presentation, et c. Once that is
   done, the mark_text field of the Text_stat can be removed. More
   importantly, it can easily be seen which conference a text is a
   presentation for (et c). We also get more redundancy that dbck can
   use. (3490225) 

        I think that we want a more complex mark structure, and have
        both text and marker know about the marks on a particular
        text.

** Consider implementing an aux-info that causes comments of a text,
   or of all texts written by the author, to get a bcc-recpt to the
   authors letterbox. Make sure that this bcc-recpt is not the only
   thing that makes it possible for the person to read the text. 
   (3381984, 3382009)

** Define and implement asynchronous messages for cache inconsitency.

          This is partially done. -- DCB

   One problematic situation is when a new comment is created, and a
   conference that is a recipient of the commented text is not a
   member of the comment.  Any user that is a member of that
   conference will not receive the new_text message, so his cache will
   not be updated.

** Add an improved internal structure for the text_stat list to avoid
   a large fixed-size array.

** Document the mark bits. (6390, 24358, 24444, 40876, 42356, 45110,
   1111605, 2687219)

        Better still, implement a more complex mark structure rather
        than using the current mark number as a bitfield. We need to
        be able to see what marks are set on a given text. -- DCB

** Document the user area.

** Add a separate subject field (a short string) to the Text-stat. 
   Problems: should the subject be restricted to ISO 8859-1?

* Lower priority than the above-mentioned things.

** isc_disable()/isc_enable() should be used, so that a single session
   cannot starve other sessions.  We might need a better mechanism
   that isc_disable()/isc_enable.

** The Info structure should contain statistics about the server
   (uptime, other interesting stuff.) (from 1991)

** difftime is not used everywhere where it should be used. There are
   a few places where '-' is still used to get a diff in seconds.

** Fix something that can replace the user_area.  The elisp-client
   creates a new user-area each time a person creates a permanent
   filter.  What a waste of text numbers!  It would be better to have
   a way to store per-user configuration separately.  It should
   probably be keyed data, but separating it in different sections
   like we do with the user-area really doesn't make much sense.

** Comments should optionally inhibit the garb (3608307).  See IDEAS.

** Do something about how large texts are sent back to the client.  We
   should probably get rid of ISC altogether.  See IDEAS.

** Performance enhancement: Don't emit asynchronous messages
   immediately. Attempt to write, say, 4 KB of data at a time to the
   clients. The server currently makes a lot of small writes.

** Performance enhancement: Don't call gettimeofday() more than once
   per atomic call.

** Introduced aux-items that states that "this text is the
   presentation/motd of conference so-and-so".

** Run the garb at 02:00 (configurable) rather than every 24 hours.

** Some generalized way to query the server for (unread) texts with a
   certain aux-item set. (see 21415 for motivation)

** Write a "lyskomd hackers guide" (requested in 24684).

** Tree-structure for the conferences (41768). 

        Yay! See the IDEAS file for some concrete ideas on how to do
        this. -- DCB.

** Don't include so much information in the Person and Conference
   structures internally in the server. A Conference is (probably)
   used much more often than the large internal lists within it.

   For active conferences, the l2g map is probably used a lot. However
   the created-texts map for a person is not used a lot. The
   read-texts part of memberships get updated a lot. For maximum gains
   we probably have to split off the created-texts map and make is
   possible to read individual memberships from disk. The entire thing
   should be hidden from most of the server by automatically reading
   bits of maps or memberships on demand from disk. High-level
   functions shouldn't have to know about this.

** Improve the data base. Text 250010 contains some ideas for a grand
   redesign.

** A better way to "list active conferences" than to get the
   conference status of every conference and sort them on the
   last-written field. According to 310351 PortaKOM had the command
   "Lista aktiva möten". We should be as good as PortaKOM in every
   aspect. But see text 3604479 which outlines a way to implement this
   in a client in a reasonably efficient way without support from the
   server.

** Stack-based slow searching?  See 1013917 for some ideas.

** 103=local-to-global and 34=get-map should be able to return more
   information when applied to e. g. letterboxes. It should return
   information about all texts that ACTPERS is allowed to read. See
   678026 and 678411 for more thoughts about this.

** Re-run the test in 775202 and see if anything can be improved.

** New version of mark_as_read:

        void mark_as_read(Text_no);

   All examined clients emulate this call with a loop over all
   recipients of the text.

** Text parts, so that we can have both audio and text.

** Keyword support on texts.

** Keyword support on conferences (1112142, 1112200).

** Version numbering on calls? (1441352)

** A possibility to ask for conferences created after a certain date. 
   (1622840). Dubious value; the new conferences should have a
   presentation.

** Add a call to retrieve the highest used conference number, so that
   a client can loop over all conferences/detect when new conferences
   have been created while the client wasn't logged on.

** Add get_next_conf/get_previous_conf calls, to be able to loop
   through the conferences.

** A documented way to store the prefered language in the common area
   of the user area (1632323).

** Check the rules for when you are allowed to change a recpt to a
   cc-recpt. (1901530, 2329667, 2329682). (This may already be
   implemented -- check it.)

** It would be nice if an author could be notified whenever a text
   written by him was commented. 3381984 and 3382009 outlines a way to
   implement that using a special aux-info and automatic adding of a
   bcc-recpt by the server to the new comment.

** There should be a limit on how many texts you can mark as read in a 
   text to avoid the list becoming too long. This bug has a low
   priority. Footnote (1991-01-24): When Inge's Numlist package is
   finished, this won't matter any more. This bug requires no other
   action. Footnote (1999-01-18): When will the package be done?


* Fixed

** Document 103=local_to_global
    DONE --ceder

** Missing test cases in 03.exp
    Calls 100, 101, 102, 103, 104
    I've done 100-102. 103 and 104 left.
    I will do this --ceder
    DONE

** local_to_global isn't tested enough.  Run gcov on it.
	DONE.  We now have 96.39% coverage; only some file error
	       handling is untested.

** Improve the file format for local-to-global.c.  Warning: this will
    cause an incompatible change in the database format.
	DONE

** Fine-tune the Text_mapping data type (3623903, 3624144, 3624179, 3624173).
	DONE: range-begin and range-end added to Text-Mapping.

** Merge all changes done to prot-A.txt into Protocol-A.texi.
	DONE.  prot-A.txt removed.

** Remove the man pages for lyskomd and dbck.
	DONE.  All man pages still exists, but they only refer to the
	Texinfo documentation.  We don't want man pages from earlier
	lyskomd installations to linger around.  The should be
	completely removed in the next release, though.

** Fix dbck so that it can repair Member<->Membership inconsistencies.
        DONE. Needs to be tested though.

** Implement a test suite that tests all calls successfully at least
   once.
        DONE.

** Should the error message be named conference-zero or
   zero-conference? (Protocol-A.texi)
        DONE: conference-zero

** 52=get-unread-confs should ignore conferences where you are a
   passive member.
        DONE.

** Mention explicitly in the protocol spec that the cient cannot send
   "0 *" to the server.
        DONE.

** Protocol-A.texi contains both INT32 and INTEGER. (3228850)
        DONE. Changed all INTEGER to INT32

** Protocol-A.texi uses mailbox and letterbox interchangeably. Use one
   term only, or at least state explicitly that they are the same
   thing.
        DONE. Letterbox used in structures, mailbox in text. It's safe
        to replace all mailboxes with letterbox.

** String-Size is used but not defined in Protocol-A.texi. (3228850)
        DONE. Use INT32 instead

** Session-No is defined twice in Protocol-A.texi. (3228850)
        DONE. Removed second definition.

** Check the unused static function regarding aux-infos.
        DONE.

** ceder thinks that the sender of a BCC recipient group should be
   allowed to see it, so that the sender can see when the recipient
   has read the text. (3380375, 3381048)
        DONE.

** ceder thinks that a BCC recipient group should be visible to
   anybody that may become a member of the BCC recipient (as opposed
   to all members and all administrators), lest everybody starts to be
   passive members of every conference just to be able to see BCC
   recipients. (3380375, 3381048)
        DONE.

** The redirect aux-info needs more documentation. See FIXME comment
   in Protocol-A.texi.
        FIXED.

** Validation of aux-items.
        DONE. Regexp validation done.

** Ability to have a callback function when an aux-item is created,
   removed, changed or resurrected. (This is according to 3053346
   critical. Why?)
        DONE.

** The Membership that is returned by 98=query-read-texts should
   include the position of the Membership.
        DONE. Did the same for get-membership.

** Write a Texinfo manual for lyskomd based on the man pages.  
        DONE. Whee!

** The redirect aux-info needs more documentation. See FIXME comment
   in Protocol-A.texi.
        DONE.


** Should we really retain deleted aux-items indefinitely? (3490203)
        DONE. No, and we don't any more.

** disconnect(0) should disconnect the current session. (806239)
        DONE. Session 0 is now always interpreted as the current
        session.

** Add the dbck reference to the lyskomd manual.
        DONE. There is now a dbck Texinfo manual

** An asynchronous message is sent to everybody when a secret
   conference changes its name. That is an unacceptable leak of
   sensitive information. (990788, 2804846)
        FIXED.

** Check that Async 13 is sent at the proper times (see 79208).
        FIXED.

** Newline terminates requests. Check that a wording similar to that
   in 1509862 is present in Protocol-A.texi.
        FIXED.

** 89=create-person should probably not do an automatic login. (Wasn't
   this discussed in KOM? Yes it was, in 3267591, but there are no
   surviving comments. Did anybody object? /ceder)
        DONE.

** Check change-what-am-i-doing very carefully. According to 915974
   and 916257 there may be a bug in it, but that is pure speculation.
        DISMISSED. Tried sending long, short and invalid strings. No
        dice. Code inspection of functions turn up nothing.

** Should the conference supervisor be able to set the secret bit of a
   membership type? Probably yes. Maybe no.
        DONE. Can't set forbid-secret to true if there are secret
        members. Conference supervisor will have to boot secret
        members or convince them to convert.

** Create a "server hackers guide" and move the contents of
   doc/server.extend (and several other files in the doc directory) to
   it. The "server hackers guide" should probably be an appendix of
   doc/lyskomd.texi.
        DONE.

** Test the interpretation of session 0 as the current session.
        DONE.

*** Integrate doc/Bugrapporter into this document.
        DONE.

** Let the author of a text add other people's texts as footnotes.
   (from 1991)
	REJECT. (ceder & DCB 1999-03-28)

** There is no limit on how much can be queued in a write queue in
   isc. (from 1991, still an issue.)
	DONE -- found to be fixed when the code was inspected 1999-03-28

** get-n-unread as per 1111652.
	REJECT. (ceder & DCB 1999-03-28)

** If would be cool to be able to do finger @kom.lysator.liu.se and
   get something like:

        @finger @lyskom.lysator.liu.se
        
         Wed  2-Jan-91 15:27:08  Up 718:34:00
         3+2 Jobs   Load av   0.20   0.07   0.00
        
         No LysKOM administrator in attendance
        
         Job  Line Activity  User               Where
          22    p1  Reading  ceder              LysKOM internals
          23    p2  Waiting  noppe              Inl{gg }t mig
          24    p3  Reading  pell               Hackers (@) Nanny
        
           1   172  LDB      OPERATOR
           2   173  KOMSTAT  Statistikid f|r statistikinsamling

	REJECT. (ceder & DCB 1999-03-28 -- a client could do this)

** cached_get_garb_nice has nowhere to get garb_nice from. It's not
   cached. 

        I think this is fixed with small_conf_arr. Ceder, what do you
        say? --DCB
	I say it's fixed.  /ceder

** There's a long-standing memory leak, probably in ISC. It appears
   to happen when we queue up lots of junk to clients that have died. 
   Trace what happens if we do isc_oflush and hit the E2BIG bit.

      I tried lots of crap. I think this will have to wait until we
      have a leak detection tool and can run the generic test suite on 
      the code. --DCB

      Even with a leak detection tool no leak was found.  The server
      simply uses a lot of (possibly fragmented) memory.  /ceder 1999-03-28

** Read all texts in LysKOM (-) Systemet, protokollet mm created
   after 1991 (start at text 170527) and incorporate selected
   information in this document.

	DONE.

** Read 45110, 45121, 45284 again and incorporate selected comments in 
   this document.

        45110: Membership stuff. Bellman's idea of sorting using
        anything but priority hasn't taken over the world since 1991. 
        I think we can declare it dead. Membership flags have been
        added. How a membership breaks the read order is a client
        matter. I don't think it anything to do with the server. The
        user area has been implemented. There's another todo item
        about documenting it. Marks and keywords are covered by
        another todo item. We're doing external recipients with
        aux-items. Asynchronous message bit is done.  Server info is
        done. Don't do the mapcar function for this version. We are
        using latex to document the server. I declare this DONE.

        45121: Comments to the above text. Dealt with.

        45284: Also comments. Also dealt with.

	DONE (or REJECTED).

** isc_getnextevent should use a write-set in select.
   (from 1991)
	DONE (apparently)

** Fix case errors in type names etc in the documentation (3267160).
	DONE.

** New version of get-text-stat (et al) where the misc-info-list is
   removed. (1108039).
	REJECT: too much work to be worth it.

* In progress 

** Document aux-items for mail import/export. (3229403)

   3229403 1998-08-17  20:53  /47 rader/ David Byers
   Kommentar till text 3228797 av Jonas S Karlsson (Palmist)
   Mottagare: LysKOM; Utvecklingsgruppens interna möte <876>
   Markerad av 1 person.
   Ärende: 13, 14, 15
   ------------------------------------------------------------
   > Hmm, låter som en pik ;)
   
   Det menar du inte?
   
   Ceder har nog inga beskrivningar som har att göra med import av email.
   Det som ceder frågade om var en aux-item som används för att säta
   emailadress på personer, möten eller servern, inte texter.
   
   Som jag läser ditt mail så tycker du att följande aux-items behövs och
   är tillräckliga:
   
   1. Extern mottagare för inlägg
      Vad innehåller aux-item? En sladdpostadress? Vad används den till?
      Vem använder den?
   
   2. "Reply-to" på möten
      En Reply-To header som skall sättas på mail som exporteras från
      brevlådan. Vem är ansvarig för exporten? Klienten eller en
      exportör? 
   
   3. "From" på möten
      Se 2.
   
   4. CC på möten
      Se 2.
   
   5. Adress till mailinglista
      Kan sättas på möten och innehåller en sladdpostadress dit mail
      skall skickas. Vem är ansvarig för att skicka mail? En exportör
      eller klienten?
   
   6. Diverse mailhuvuden på enstaka importerade inlägg
      Du har nämnt message-id, from, reply-to, to och cc. Jag föreslår
      att man dessutom har ett item som säger att inlägget är ett
      importerat mail och att man sparar alla headerrader i inlägget.
      Det är upp till klienten att besluta om den skall visa dem eller
      inte. 
   
   7. "Spamfilter"
      En sträng som tolkas av importören. Antagligen vill du ha
      klientspecifika aux-items för detta, om du inte definierar ett
      lagom stort och stabilt språk för filtreringen. 
   
   
   Är det här tillräckligt? Om så är fallet, checka ut aux-items.def ur
   lyskomd, skriv in dina definitioner och dokumentera dem i
   Protocol-A.texi.
   (3229403) ------------------------------------------
   

** Fix aux-item 13, 14 and 15 according to 3228006 and 3228045.

   3228006 1998-08-17  14:46  /8 rader/ David Byers
   Mottagare: LysKOM; Utvecklingsgruppens interna möte <838>
   Markerad av 1 person.
   Ärende: 13, 14, 15
   ------------------------------------------------------------
   13 och 14 ser bra ut. Jag tycker du tar bort parentesen i slutet av
   beskrivningen av vad 13 betyder när den är satt på ett möte.
   
   Eventuellt skulle man kunna säga på nummer 13 att innehållet får bestå
   av två saker separerade med space. Det första är i så fall en
   identifierare. Följande identifierare är definierade: "list-address"
   och "subscription-address". Eller nåt. Jag vet inte. Skriv något åt
   det här hållet om du tycker det verkar OK.
   (3228006) ------------------------------------------
   Kommentar i text 3228045 av ceder (Per Cederqvist)
   
   3228045 1998-08-17  14:52  /10 rader/ ceder (Per Cederqvist)
   Kommentar till text 3228006 av David Byers
   Mottagare: LysKOM; Utvecklingsgruppens interna möte <839>
   Markerad av 1 person.
   Ärende: 13, 14, 15
   ------------------------------------------------------------
   Jag skulle nog nästan hellre stoppa in en ny aux-item som är
   moderatoradress.  Eller rent av låta list-address och
   subscription-address vara två nya aux-items, och bara tillåta nummer
   13 på brevlådor.  Då blir 13 en persons emailaddress.
   
   Hmm.  För ett möte som "Idonex" vill man kanske stoppa dit en aux-item
   som är något typ "sales@idonex.se".  Renast blir nog om 13 står för
   emailadresser som hör till en fysisk person, juridisk person, eller
   liknande.  (Filosofisk fråga: är en mailinglista "något liknande"
   eller något väsensskillt?)
   (3228045) ------------------------------------------



Local variables:
mode: outline
End: