Skip to content
Snippets Groups Projects
Select Git revision
  • 81ea278251d9e8efc99e70f76151f5686e73bd01
  • master default
  • dbck-q-n-d-link
  • foutput-text_stat-override
  • generations
  • text-stat-sha256
  • use-nettle
  • import-nettle
  • refactor-cached_get_text
  • refactor-cached_get_text-part-2
  • add-text_store
  • introduce-generation_position
  • remove-reclamation
  • dbfile-temp-filenames
  • sstrdup
  • dbfile_open_read-check-magic
  • adns_dist
  • liboop_dist
  • search
  • isc
  • dbdbckmultiplechoice
  • last.cvs.revision
  • 2.1.2
  • 2.1.1
  • 2.1.0
  • adns_1_0
  • liboop_0_9
  • 2.0.7
  • search_bp
  • 2.0.6
  • 2.0.5
  • isc_1_01
  • Protocol-A-10.4
  • 2.0.4
  • 2.0.3
  • 2.0.2
  • 2.0.1
  • 2.0.0
  • isc_1_00
  • isc_merge_1999_05_01
  • isc_merge_1999_04_21
41 results

IDEAS

Blame
  • IDEAS 10.01 KiB
    
    
    	THIS FILE IS NO LONGER USED
    	THIS FILE IS NO LONGER USED
    	THIS FILE IS NO LONGER USED
    
    
    We now use the Bugzilla installation at Lysator to handle the bugs and
    feature requests of lyskomd.  See the following:
    
    	http://bugzilla.lysator.liu.se/
    
    The Bugzilla URL:s below point out the relevant entries.  The ideas
    were copied to Bugzilla during October 2001.
    
    
    	THIS FILE IS NO LONGER USED
    	THIS FILE IS NO LONGER USED
    	THIS FILE IS NO LONGER USED
    
    
    
    * Open Secret Conferences (mirar)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=127>
    
      It would be nice to be able to set permissions on a conference so
      that it is considered open to all with write permissions and closed
      or secret to all others.
    
    
    * Better Text Garbing (nisse@lysator.liu.se)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=82>
    
      Keep texts that have comments newer than N days, where N is a
      parameter of the conference.
    
      The parameter N has been implemented (keep_commented). The garb
      has to be changed (this is also in the TODO file.) It might be
      desirable to look even deeper into the comment chain so we keep
      *chains* that have additions newer than N days.
    
    
    * Chop up texts before sending (byers@lysator.liu.se)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=105>
    
      We aren't too good at handling large texts. The problem is that we
      start off by reading the entire thing into memory from disk, then
      shove it all onto an ISC queue. For big texts that means we sit and
      wait for I/O of the text, then we sit and wait while we queue it all 
      up, and while we're doing that, we actually wait for the server to
      write the garbage to the pipe! 
    
      It would be nice to be able to spread the work around.
    
      I think the trick would be to not read everything from disk at once.
      We want to read a bit, send it, read some more, and send that. We
      add a function to ISC that lets us push an ISC packet onto the ISC
      queue that contains no data, just an instruction to call a callback.
      That callback reads some more data from the disk and queues that in
      ISC too.
    
      When we want to output a text, we start by enqueueing a callback to
      read the second bit of the text. Then we read the first bit and
      enqueue that. When we call isc_flush, isc will start by calling the
      callback. It will enqueue a callback to read the third bit from
      disk, then read the second bit from disk and enqueue that. Then we
      return to ISC and let ISC write the first bit to the socket. The
      result is that the ISC queue will contain block N of the text, then
      a callback to get N+2, then block N+1.
    
      The reason for doing it this way is that if we can get the callback
      to do the reads asynchronously, we can have ISC writing a bit of
      data to the socket while we are reading the next bit from disk. If
      we can do async reads, when ISC gets to the callback, the queue will 
      contain a callback for block N+1, then block N of the text. The
      callback will enqueue a callback for block N+2, then initiate an
      async read for block N+1 and return immediately. The server will
      write block N to the socket concurrently with the read of block N+1.
      When the async read returns, its handler (callback, signal handler,
      whatever) enqueues the data in the ISC queue. This should result in
      the server spending much less time in IO wait.
    
      If we can't do async reads, then we might as well just put the
      callback for block N+1 after the data for block N. Async reads are
      available under Solaris, but not in Linux 2.0. Regrettably the
      interface is probably vendor-dependent.
    
      A potential problem is that we need to make *sure* that the text
      file is not cleaned when a text is deleted. That is not a problem in 
      the current server, but could be in the future. If we have a pending 
      request for text N, and someone deletes it before we can send the
      whole thing, we have to be able to continue sending it.
    
      Changes need to be made to ISC (typed queue elements, call the
      callback); prot_a_output and mux.c (need to output texts in a
      special way); plus we have to write the callbacks. This should be
      totally doable.
    
      NEWS: The aio_ interface is not supported by Linux, at least not
      yet. It is, however, a Posix 4 extension, not just a Solaris thing.
      On Linux we'd have to use poll or select to do asynchronous IO. Is
      it possible to use ISC for this as well?
    
    	note: it is impossible to use select to do asynchronous I/O
            against files.  select always considered data from a file to
    	be immediately availabe.  /ceder
    
      ceder says: adding support for aio_* to ISC is probably a good
      idea.  It should be possible to do it in a way that is useful even
      on platforms that don't have aio_*.  The idea then is to make sure
      that we never read more than one block from the file until we have
      written it to the client.  Having a 8 KiB buffer where chunks are
      copied from file to client, from file to client, is a lot better
      than reading 1 MiB from file, chopping it up in small blocks, and
      storing them i an ISC queue.
    
    
    
    
    * Sync the database without blocking requests (byers@lysator.liu.se)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=128>
    
      Currently the database is synced in three phases. Phase one creates
      a snapshot of all changed objects by traversing the internal arrays
      and moving the data pointer to the snapshot pointer in the cache
      nodes. The second phase writes objects to disk. The third phase
      frees the snapshot data from memory. The first and last phase block
      the server completely.
    
      Rewrite the sync algorithm like this. Assume that all objects in the 
      database can be ordered, regardless of type. The server can be in
      one of two states: syncing or not syncing.
    
      When the server starts syncing, open the target data file and write
      the database header to the file. Write kom_info to the database. Set 
      next_to_sync to zero.
    
      Periodically, sync a few objects. Examine object next_to_sync. If it 
      has been saved to disk, do nothing. If it has a snapshot, write the
      snapshot to disk. If it is in memory, write it to disk from memory.
      If it is on disk, copy it from the old data file. Clear the
      snapshot. Clear the dirty flag. Clear the saved-to-disk flag.
      Increase next_to_sync and repeat this step until a fixed number of
      objects have been saved.
    
      When all objects are done, set the state of the server to not
      syncing, set the current data file to the one just synced.
    
      While the server is syncing, access to objects needs to be done
      carefully. If an object is requested that has not been synced,
      create a snapshot of it before returning it, or write it to disk
      immediately. If an object that has been synced but is not in memory
      is requested, then it must be read from the new data file. 
    
      Tricky stuff: if something is added while we are syncing, it should
      not be written to disk. If something is deleted while we are syncing 
      is should be written to disk anyway.
    
      The basic idea here is to make sure that the database file contains
      a copy of the database as it was when the sync was started, no
      matter what happens with it afterwards. We have like an index;
      everything before the index has been synced and can be treated as if 
      we are not syncing at all. Everything beyond the frontier may be
      unsynced, and we need to make sure that it is retained unchanged
      until it gets written to disk.
    
    
    * Hierarchical Conferences (byers@lysator.liu.se)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=110>
    
      LysKOM conferences should be organized hierarchically.
    
      Give each conference a parent conference pointer and a children
      conference list. Internally, prepend the parent ID to each
      conference name, so that we can have two conferences with the same
      name contained below different parent conferences. The prepending
      should not be visible to the client.
    
      Permissions in a conference are based on the permissions in the
      conference closest in child-parent order to the requested conference 
      in which the person is a member (whew!) Some examples: conference A
      is rd_prot and conference B is not. Persons who are members of A can 
      see anything in A and B. Persons who are members in B can see
      anything in B, but not in A. To persons who are members of neither,
      B behaves as if it were rd_prot, since its parent conference is.
    
      It has to be possible to do name lookup on a subtree of the
      conference tree.
    
      The small_conf_stat structure should probably mirror the tree
      structure and contain permission information about conferences.
      Don't forget that we want to be able to do sequential traversal of
      all elements in a subtree.
    
      This should require surprisingly small changes in the server. Mainly 
      small_conf_stat, name lookup and storage, and permissions checks.
    
    
    * Garbage Collecting Conferences (byers@lysator.liu.se)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=129>
    
      Conferences have an expire field. Conferences that have not
      contained texts for that many days should be removed automatically
      by the garbage collector or by dbck.
    
    
    * Mark Types (byers@lysator.liu.se)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=96>
    
      Marks should be fixed. Marks should have flags and stuff. Texts
      should know who has marked them. There's a text about this somewhere 
      in LysKOM.
    
    
    * Multibyte character compatibility (byers@lysator.liu.se)
    
      Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=99>
    
      The server should be capable of operating in multi-byte
      environments. We'll have to be able to specify a coding system and
      charset for conference and person names. We'll have to make sure the 
      regexp engine can deal with multibyte characters. We'll need a new
      method of separating the subject line from the text body. The name
      matching will have to be more flexible. For eight-bit characters
      we'll need a way of specifying a collate table. For multibyte
      characters, maybe we'll have to be able to specify a procedure to
      determine equivalence of two characters. 
    
    
    
    Local variables:
    mode: outline
    paragraph-separate: "[ \t]*$"
    outline-regexp: "[*]+"
    end: