Select Git revision
Per Cederqvist authored
IDEAS 10.01 KiB
THIS FILE IS NO LONGER USED
THIS FILE IS NO LONGER USED
THIS FILE IS NO LONGER USED
We now use the Bugzilla installation at Lysator to handle the bugs and
feature requests of lyskomd. See the following:
http://bugzilla.lysator.liu.se/
The Bugzilla URL:s below point out the relevant entries. The ideas
were copied to Bugzilla during October 2001.
THIS FILE IS NO LONGER USED
THIS FILE IS NO LONGER USED
THIS FILE IS NO LONGER USED
* Open Secret Conferences (mirar)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=127>
It would be nice to be able to set permissions on a conference so
that it is considered open to all with write permissions and closed
or secret to all others.
* Better Text Garbing (nisse@lysator.liu.se)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=82>
Keep texts that have comments newer than N days, where N is a
parameter of the conference.
The parameter N has been implemented (keep_commented). The garb
has to be changed (this is also in the TODO file.) It might be
desirable to look even deeper into the comment chain so we keep
*chains* that have additions newer than N days.
* Chop up texts before sending (byers@lysator.liu.se)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=105>
We aren't too good at handling large texts. The problem is that we
start off by reading the entire thing into memory from disk, then
shove it all onto an ISC queue. For big texts that means we sit and
wait for I/O of the text, then we sit and wait while we queue it all
up, and while we're doing that, we actually wait for the server to
write the garbage to the pipe!
It would be nice to be able to spread the work around.
I think the trick would be to not read everything from disk at once.
We want to read a bit, send it, read some more, and send that. We
add a function to ISC that lets us push an ISC packet onto the ISC
queue that contains no data, just an instruction to call a callback.
That callback reads some more data from the disk and queues that in
ISC too.
When we want to output a text, we start by enqueueing a callback to
read the second bit of the text. Then we read the first bit and
enqueue that. When we call isc_flush, isc will start by calling the
callback. It will enqueue a callback to read the third bit from
disk, then read the second bit from disk and enqueue that. Then we
return to ISC and let ISC write the first bit to the socket. The
result is that the ISC queue will contain block N of the text, then
a callback to get N+2, then block N+1.
The reason for doing it this way is that if we can get the callback
to do the reads asynchronously, we can have ISC writing a bit of
data to the socket while we are reading the next bit from disk. If
we can do async reads, when ISC gets to the callback, the queue will
contain a callback for block N+1, then block N of the text. The
callback will enqueue a callback for block N+2, then initiate an
async read for block N+1 and return immediately. The server will
write block N to the socket concurrently with the read of block N+1.
When the async read returns, its handler (callback, signal handler,
whatever) enqueues the data in the ISC queue. This should result in
the server spending much less time in IO wait.
If we can't do async reads, then we might as well just put the
callback for block N+1 after the data for block N. Async reads are
available under Solaris, but not in Linux 2.0. Regrettably the
interface is probably vendor-dependent.
A potential problem is that we need to make *sure* that the text
file is not cleaned when a text is deleted. That is not a problem in
the current server, but could be in the future. If we have a pending
request for text N, and someone deletes it before we can send the
whole thing, we have to be able to continue sending it.
Changes need to be made to ISC (typed queue elements, call the
callback); prot_a_output and mux.c (need to output texts in a
special way); plus we have to write the callbacks. This should be
totally doable.
NEWS: The aio_ interface is not supported by Linux, at least not
yet. It is, however, a Posix 4 extension, not just a Solaris thing.
On Linux we'd have to use poll or select to do asynchronous IO. Is
it possible to use ISC for this as well?
note: it is impossible to use select to do asynchronous I/O
against files. select always considered data from a file to
be immediately availabe. /ceder
ceder says: adding support for aio_* to ISC is probably a good
idea. It should be possible to do it in a way that is useful even
on platforms that don't have aio_*. The idea then is to make sure
that we never read more than one block from the file until we have
written it to the client. Having a 8 KiB buffer where chunks are
copied from file to client, from file to client, is a lot better
than reading 1 MiB from file, chopping it up in small blocks, and
storing them i an ISC queue.
* Sync the database without blocking requests (byers@lysator.liu.se)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=128>
Currently the database is synced in three phases. Phase one creates
a snapshot of all changed objects by traversing the internal arrays
and moving the data pointer to the snapshot pointer in the cache
nodes. The second phase writes objects to disk. The third phase
frees the snapshot data from memory. The first and last phase block
the server completely.
Rewrite the sync algorithm like this. Assume that all objects in the
database can be ordered, regardless of type. The server can be in
one of two states: syncing or not syncing.
When the server starts syncing, open the target data file and write
the database header to the file. Write kom_info to the database. Set
next_to_sync to zero.
Periodically, sync a few objects. Examine object next_to_sync. If it
has been saved to disk, do nothing. If it has a snapshot, write the
snapshot to disk. If it is in memory, write it to disk from memory.
If it is on disk, copy it from the old data file. Clear the
snapshot. Clear the dirty flag. Clear the saved-to-disk flag.
Increase next_to_sync and repeat this step until a fixed number of
objects have been saved.
When all objects are done, set the state of the server to not
syncing, set the current data file to the one just synced.
While the server is syncing, access to objects needs to be done
carefully. If an object is requested that has not been synced,
create a snapshot of it before returning it, or write it to disk
immediately. If an object that has been synced but is not in memory
is requested, then it must be read from the new data file.
Tricky stuff: if something is added while we are syncing, it should
not be written to disk. If something is deleted while we are syncing
is should be written to disk anyway.
The basic idea here is to make sure that the database file contains
a copy of the database as it was when the sync was started, no
matter what happens with it afterwards. We have like an index;
everything before the index has been synced and can be treated as if
we are not syncing at all. Everything beyond the frontier may be
unsynced, and we need to make sure that it is retained unchanged
until it gets written to disk.
* Hierarchical Conferences (byers@lysator.liu.se)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=110>
LysKOM conferences should be organized hierarchically.
Give each conference a parent conference pointer and a children
conference list. Internally, prepend the parent ID to each
conference name, so that we can have two conferences with the same
name contained below different parent conferences. The prepending
should not be visible to the client.
Permissions in a conference are based on the permissions in the
conference closest in child-parent order to the requested conference
in which the person is a member (whew!) Some examples: conference A
is rd_prot and conference B is not. Persons who are members of A can
see anything in A and B. Persons who are members in B can see
anything in B, but not in A. To persons who are members of neither,
B behaves as if it were rd_prot, since its parent conference is.
It has to be possible to do name lookup on a subtree of the
conference tree.
The small_conf_stat structure should probably mirror the tree
structure and contain permission information about conferences.
Don't forget that we want to be able to do sequential traversal of
all elements in a subtree.
This should require surprisingly small changes in the server. Mainly
small_conf_stat, name lookup and storage, and permissions checks.
* Garbage Collecting Conferences (byers@lysator.liu.se)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=129>
Conferences have an expire field. Conferences that have not
contained texts for that many days should be removed automatically
by the garbage collector or by dbck.
* Mark Types (byers@lysator.liu.se)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=96>
Marks should be fixed. Marks should have flags and stuff. Texts
should know who has marked them. There's a text about this somewhere
in LysKOM.
* Multibyte character compatibility (byers@lysator.liu.se)
Bugzilla: <URL:http://bugzilla.lysator.liu.se/show_bug.cgi?id=99>
The server should be capable of operating in multi-byte
environments. We'll have to be able to specify a coding system and
charset for conference and person names. We'll have to make sure the
regexp engine can deal with multibyte characters. We'll need a new
method of separating the subject line from the text body. The name
matching will have to be more flexible. For eight-bit characters
we'll need a way of specifying a collate table. For multibyte
characters, maybe we'll have to be able to specify a procedure to
determine equivalence of two characters.
Local variables:
mode: outline
paragraph-separate: "[ \t]*$"
outline-regexp: "[*]+"
end: