1 TODO for 1.2 And Beyond
2 =======================
4 NOTE: Any item that doesn't have (***DONE) in it, isn't done yet. The
5 (***TESTING NEEDED) means that the item has been done but not yet properly
8 NOTE: A TODO entry does not mean that it is ever going to be done. Some
9 of the entries may be just ideas, good, bad or ugly. If you want to work
10 on some of the TODO entries simply let us know about it by dropping a note
11 to silc-devel mailing list or appear on 'silc' channel on SILCNet.
17 o Create apps/tutorial containing various Toolkit API tutorials.
19 o The Toolkit split. The Toolkit is to be splitted in parts. How many
20 parts and what the parts are isn't decided yet. Each part is a separate
21 software package. Current thinking is of the following:
23 SILC Toolkit SILC protocol, client and server library
24 SILC Runtime Toolkit runtime library
25 SILC Crypto Toolkit crypto, asn1, math, skr, pgp, etc.
27 The rationale for this is of course that other than SILC projects
28 might like to use the various libraries SILC Toolkit provides, but
29 naturally they don't want the bloat of SILC protocol related stuff.
31 The Runtime library in SILC Toolkit is a general purpose runtime library,
32 like Glib and APR are. The runtime library is to be developed further
33 to provide alternative to Glib and APR.
35 The Crypto library in SILC Toolkit is a general purpose crypto library
36 providing pretty nice APIs compared to many other crypto libraries,
37 especially OpenSSL. The Crypto library is to be developed further
38 to include support for OpenPGP, X.509 and SSH2.
44 o SILC_PACKET_FLAG_ACK support. Implement ACK packet and packet payload
47 o All payload encoding routines should take SilcStack as argument.
49 o Remove SilcCommandCb from silccommand.h.
51 o All payload test routines into lib/silccore/tests/.
54 lib/silcclient, The Client Library
55 ==================================
57 o Giving WHOIS for nick that doesn't exist should remove any same
58 named entries from the client cache.
60 o peer-to-peer private messages
62 o Private message key request notification to application. See XXX in
65 o in JOIN notify handle resolving that timedout. Currently the user is
66 never joined the channel if this happens. What to do if message is
67 received from user that hasn't been resolved/joined?
69 o Message ACKing support.
71 o in /cmode and /cumode with +r, maybe the public key and private key
72 could be just some "string", which would then match to "string.pub" and
75 o If the SILC Events (see below) are implemented, perhaps client library
76 should provide events so that application developer has a choice of
77 developing the SILC app with callbacks or with events.
80 Runtime library, lib/silcutil/
81 ==============================
83 o Fix universal time decoding (doesn't accept all formats) in silctime.c.
85 o Add functions to manipulate environment variables.
87 SilcBool silc_setenv(const char *variable, const char *value);
88 const char *silc_getenv(const char *variable);
89 SilcBool silc_clearenv(const char *variable);
91 o Add functions to loading shared/dynamic object symbols (replaces the
92 SIM library (lib/silcsim) and introduces generic library). Add this
93 to lib/silcutil/silcdll.[ch].
95 SilcDll silc_dll_load(const char *object_path, SilcDllFlags flags);
96 void silc_dll_close(SilcDll dll);
97 void *silc_dll_getsym(SilcDll dll, const char *symbol);
98 const char *silc_dll_error(SilcDll dll);
100 o Add directory opening/traversing functions
102 o silc_getopt routines
104 o silc_hash_table_replace -> silc_hash_table_set. Retain support for
105 silc_hash_table_replace as macro.
107 o The SILC Event signals. Asynchronous events that can be created,
108 connected to and signalled. Either own event routines or glued into
111 SilcTask silc_schedule_task_add_event(SilcSchedule schedule,
112 const char *event, ...);
113 SilcBool silc_schedule_event_connect(SilcSchedule schedule,
115 SilcTaskCallback event_callback,
117 SilcBool silc_schedule_event_signal(SilcSchedule schedule,
118 const char *event, ...);
121 silc_schedule_task_add_event(schedule, "connected",
125 silc_schedule_event_connect(schedule, "connected", connected_cb, ctx);
126 silc_schedule_event_signal(schedule, "connected", integer, buf,
128 SILC_TASK_CALLBACK(connected_cb)
130 FooCtx ctx = context;
135 va_start(args, context);
136 integer = va_arg(args, SilcUInt32);
137 buf = va_arg(args, SilcBuffer);
142 Problems: Events would be SilcSchedule specific, and would not work on
143 multi-thread/multi-scheduler system. The events should be copyable
144 between schedulers. Another problem is the signal delivery. Do we
145 deliver them synchronously possibly from any thread to any other thread
146 or do we deliver them through the target schedulers. If we use the
147 schedulers then signalling would be asynchronous (data must be
148 duplicated and later freed) which is not very nice.
150 o If the event signals are added, the SILC_PARAM_* stuff needs to be
151 moved from silcbuffmt.h to silctypes.h or something similar.
153 o In case the SILC Events are done we shall create a new concept of
154 parent and child SilcSchedule's. When new SilcSchedule is created a
155 parent can be associated to it. This association could be done either
156 directly by the parent or by any other children. This way the signals
157 would in effect be global and would reach all children schedulers.
159 This relationship would be associative only. The schedulers are still
160 independent and run independently from each other. All schedulers
161 would be linked and could be accessed from any of the schedulers.
162 It should be possible to retrieve the parent and enumate all children
163 from any of the schedulers.
165 SilcSchedule silc_schedule_init(int max_tasks, void *app_context,
166 SilcSchedule parent);
167 SilcSchedule silc_schedule_get_parent(SilcSchedule schedule);
169 o Additional scheduler changes: optimize silc_schedule_wakeup. Wakeup
170 only if the scheduler is actually waiting something. If it is
171 delivering tasks wakeup is not needed.
173 o Structured log messages to Log API. Allows machine readable log
174 messages. Would allow sending of any kind of data in a log message.
176 o Base64 to an own API
180 o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least.
181 SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled
184 o silc_stringprep to non-allocating version.
186 o SilcStack aware SilcHashTable.
188 o SilcStack aware SilcDList.
190 o Thread pool API. Add this to lib/silcutil/silcthread.[ch].
192 typedef void (*SilcThreadPoolFunc)(SilcSchedule schedule,
195 /* Allocate thread pool with at least `min_threads' and at most
196 `max_threads' many threads. If `stack' is non-NULL all memory
197 is allocated from the `stack'. If `start_min_threads' is TRUE
198 this will start `min_threads' many threads immediately. */
199 SilcThreadPool silc_thread_pool_alloc(SilcStack stack,
200 SilcUInt32 min_threads,
201 SilcUInt32 max_threads,
202 SilcBool start_min_threads);
204 /* Free thread pool. If `wait_unfinished' is TRUE this will block
205 and waits that all remaining active threads finish before freeing
207 void silc_thread_pool_free(SilcThreadPool tp, SilcBool wait_unfinished);
209 /* Run `run' function with `run_context' in one of the threads in the
210 thread pool. Returns FALSE if the thread pool is being freed. If
211 there are no free threads left in the pool this will queue the
212 the `run' and will call it once a thread becomes free.
214 If `completion' is non-NULL it will be called to indicate completion
215 of the `run' function. If `schedule' is non-NULL the `completion'
216 will be called through the scheduler in the main thread. If it is
217 NULL the `completion' is called directly from the thread after the
218 `run' has returned. */
219 SilcBool silc_thread_pool_run(SilcThreadPool tp,
220 SilcSchedule schedule,
221 SilcThreadPoolFunc run,
223 SilcThreadPoolFunc completion,
224 void *completion_context);
226 /* Modify the amount of maximum threads of the pool. */
227 void silc_thread_pool_set_max_threads(SilcThreadPool tp,
228 SilcUInt32 max_threads);
230 /* Returns the amount of maximum size the pool can grow. */
231 SilcUInt32 silc_thread_pool_num_max_threads(SilcThreadPool tp);
233 /* Returns the amount of free threads in the pool currently. */
234 SilcUInt32 silc_thread_pool_num_free_threads(SilcThreadPool tp);
236 /* Stops all free and started threads. The minumum amount of threads
237 specified to silc_thread_pool_alloc always remains. */
238 void silc_thread_pool_purge(SilcThreadPool tp);
240 o Fast mutex implementation. Fast rwlock implementation. Mutex and
241 rwlock implementation using atomic operations.
243 o Compression routines are missing. The protocol supports packet
244 compression thus it must be implemented. SILC Zip API must be
247 o Add new functions to SilcStack API in lib/silcutil/silcstack.[ch]. Add
248 silc_stack_[set|get]_alignment. It defines the default alignment used
249 when allocating memory from stack. It can be used to specify special
250 alignments too when needed (such as for hardware devices like crypto
251 accelerators). Move also the low level silc_stack_malloc and
252 silc_stack_realloc from silcstack_i.h to silcstack.h. Remove the
253 _ua unaligned memory allocation routines. Remove unaligned memory
254 allocation possibility.
256 o Add '%@' format to silc_snprintf functions. It marks for external
257 rendering function of following type:
259 /* Snprintf rendering function. The `data' is rendered into a string
260 and allocated string is returned. If NULL is returned the
261 rendering is skipped and ignored. If the returned string does
262 not fit to the destination buffer it may be truncated. */
263 typedef char *(*SilcSnprintfRender)(void *data);
265 It can work like following:
267 char *id_renderer(void *data)
270 id_to_str(tmp, sizeof(tmp), (SilcID *)data);
274 silc_snprintf(buf, sizeof(buf), "Client ID %@", id_renderer, client_id);
276 (o Generic SilcStatus or SilcResult that includes all possible status and
277 error conditions, including those of SILC protocol. Though, the SILC
278 protocol related status (currently in silcstatus.h) cannot be in
279 runtime library) maybe
281 (o SILC specific socket creation/closing routines to silcnet.h, wrappers
282 to all send(), recv(), sendto() etc. Bad thing is that we'd have to
283 define all socket options, sockaddrs, etc.) maybe
288 lib/silcutil/symbian/
289 =====================
291 o Something needs to be thought to the logging globals as well,
292 like silc_debug etc. They won't work on EPOC. Perhaps logging
293 and debugging is to be disabled on EPOC. The logging currently works
294 by it cannot be controlled, same with debugging.
297 SFTP Library, lib/silcsftp/
298 ===========================
300 o Read prefetch (read-ahead, reading ahead of time). Maybe if this can
304 SKR Library, lib/silcskr/
305 =========================
307 o Add fingerprint as search constraint.
309 o Add OpenPGP support. Adding, removing, fetching PGP keys. (Keyring
312 o Add support for importing public keys from a directory and/or from a
313 file. Add support for exporting the repository (different formats for
314 different key types?).
316 o Change the entire silc_skr_find API. Remove SilcSKRFind and just simply
317 add the find constraints as variable argument list to silc_skr_find, eg:
319 silc_skr_find(skr, schedule, callback, context,
320 SILC_SKR_FIND_PUBLIC_KEY, public_key,
321 SILC_SKR_FIND_COUNTRY, "FI",
322 SILC_SKR_FIND_USAGE, SILC_SKR_USAGE_AUTH,
325 NULL argument would be ignored and skipped.
327 o Add OR logical rule in addition of the current default AND, eg:
329 // Found key(s) MUST have this public key AND this country.
330 silc_skr_find(skr, schedule, callback, context,
331 SILC_SKR_FIND_RULE_AND,
332 SILC_SKR_FIND_PUBLIC_KEY, public_key,
333 SILC_SKR_FIND_COUNTRY, "FI",
336 // Found key(s) MUST have this public key OR this key context
337 silc_skr_find(skr, schedule, callback, context,
338 SILC_SKR_FIND_RULE_OR,
339 SILC_SKR_FIND_PUBLIC_KEY, public_key,
340 SILC_SKR_FIND_CONTEXT, key_context,
343 o SilcStack to SKR API.
346 Crypto Library, lib/silccrypt/
347 ==============================
351 o Add fingerprint to SilcSILCPublicKey and retrieval to silcpk.h, and
352 possibly to silcpkcs.h.
354 /* Return fingerprint of the `public_key'. Returns also the algorithm
355 that has been used to make the fingerprint. */
356 const unsigned char *
357 silc_pkcs_get_fingerprint(SilcPublicKey public_key,
358 const char **hash_algorithm,
359 SilcUInt32 *fingerprint_len);
361 o Change SILC PKCS API to asynchronous, so that accelerators can be used.
362 All PKCS routines should now take callbacks as argument and they should
363 be delivered to SilcPKCSObject and SilcPKCSAlgorithm too.
365 /* Signature computation callback */
366 typedef void (*SilcPKCSSignCb)(SilcBool success,
367 const unsigned char *signature,
368 SilcUInt32 signature_len,
371 /* Signature verification callback */
372 typedef void (*SilcPKCSVerifyCb)(SilcBool success, void *context);
374 /* Encryption callback */
375 typedef void (*SilcPKCSEncryptCb)(SilcBool success,
376 const unsigned char *encrypted,
377 SilcUInt32 encrypted_len,
380 /* Decryption callback */
381 typedef void (*SilcPKCSDecryptCb)(SilcBool success,
382 const unsigned char *decrypted,
383 SilcUInt32 decrypted_len,
386 Either add new _async functions or add the callbacks to existing API
387 and if the callback is NULL then the API is not async and if provided
388 it may be async. For example;
390 SilcBool silc_pkcs_sign(SilcPrivateKey private_key,
391 unsigned char *src, SilcUInt32 src_len,
392 unsigned char *dst, SilcUInt32 dst_size,
394 SilcBool compute_hash, SilcHash hash,
395 SilcPKCSSignCb async_sign,
396 void *async_sign_context);
398 (if this is done then there's no reason why the buffers in the
399 callbacks cannot be the ones user gives here) or allow only async:
401 SilcBool silc_pkcs_sign(SilcPrivateKey private_key,
402 unsigned char *src, SilcUInt32 src_len,
403 SilcBool compute_hash, SilcHash hash,
404 SilcPKCSSignCb async_sign,
405 void *async_sign_context);
409 SilcBool silc_pkcs_sign_async(SilcPrivateKey private_key,
410 unsigned char *src, SilcUInt32 src_len,
411 SilcBool compute_hash, SilcHash hash,
412 SilcPKCSSignCb async_sign,
413 void *async_sign_context);
415 o Change PKCS Algorithm API to take SilcPKCSAlgorithm as argument to
416 encrypt, decrypt, sign and verify functions. We may need to for exmaple
417 check the alg->hash, supported hash functions. Maybe deliver it also
418 to all other functions in SilcPKCSAlgorithm to be consistent.
420 o Add DSS support. Take implementation from Tom or make it yourself.
422 o Implement the defined SilcDH API. The definition is in
423 lib/silccrypt/silcdh.h. Make sure it is asynchronous so that it can
424 be accelerated. Also take into account that it could use elliptic
429 o All cipher, hash, hmac etc. allocation routines should take their name
430 in as const char * not const unsigned char *.
433 SILC Accelerator Library
434 ========================
436 o SILC Accelerator API. Provides generic way to use different kind of
437 accelerators. Basically implements SILC PKCS API so that SilcPublicKey
438 and SilcPrivateKey can be used but they call the accelerators.
440 Something in the lines of (preliminary):
442 /* Register accelerator to system. Initializes the accelerator. */
443 Varargs are optional accelerator specific init parameteres. */
444 SilcBool silc_acc_register(SilcAccelerator acc, ...);
446 silc_acc_register(softacc, "min_threads", 2, "max_threads", 16, NULL);
448 /* Unregister accelerator. Uninitializes the accelerator. */
449 SilcBool silc_acc_unregister(const SilcAccelerator acc);
451 /* Return list of the registered accelerators */
452 SilcDList silc_acc_get_supported(void);
454 /* Find existing accelerator. `name' is accelerator's name. */
455 SilcAccelerator silc_acc_find(const char *name);
457 /* Return accelerator's name */
458 const char *silc_acc_get_name(SilcAccelerator acc);
460 /* Accelerate `public_key'. Return accelerated public key. */
461 SilcPublicKey silc_acc_public_key(SilcAccelerator acc,
462 SilcPublicKey public_key);
464 /* Accelerate `private_key'. Returns accelerated private key. */
465 SilcPrivateKey silc_acc_private_key(SilcAccelerator acc,
466 SilcPrivateKey private_key);
468 /* Return the underlaying public key */
469 SilcPublicKey silc_acc_get_public_key(SilcAccelerator acc,
470 SilcPublicKey public_key);
472 /* Return the underlaying private key */
473 SilcPrivateKey silc_acc_get_private_key(SilcAccelerator acc,
474 SilcPrivateKey private_key);
476 typedef struct SilcAcceleratorObject {
477 const char *name; /* Accelerator's name */
478 SilcBool (*init)(va_list va); /* Initialize accelerator */
479 SilcBool (*uninit)(void); /* Uninitialize accelerator */
480 const SilcPKCSAlgorithm *pkcs; /* Accelerated PKCS algorithms */
481 const SilcDHObject *dh; /* Accelerated Diffie-Hellmans */
482 const SilcCipherObject *cipher; /* Accelerated ciphers */
483 const SilcHashObject *hash; /* Accelerated hashes */
484 const SilcHmacObject *hmac; /* Accelerated HMACs */
485 const SilcRngObject *rng; /* Accelerated RNG's */
486 } *SilcAccelerator, SilcAcceleratorStruct;
488 Allows accelerator to have multiple accelerators (cipher, hash etc)
489 and multiple different algorithms and implementations (SHA-1, SHA-256 etc).
491 SilcPublicKey->SilcSILCPublicKey->RsaPublicKey accelerated as:
492 SilcPublicKey->SilcAcceleratorPublicKey->SilcSoftAccPublicKey->
493 SilcPublicKey->SilcSILCPublicKey->RsaPublicKey
495 silc_acc_public_key creates SilcPublicKey and SilcAcceleratorPublicKey
496 and acc->pkcs->import_public_key creates SilcSoftAccPublicKey.
498 o Implement software accelerator. It is a thread pool system where the
499 public key and private key operations are executed in threads.
501 const struct SilcAcceleratorObject softacc =
503 "softacc", softacc_init, softacc_uninit,
504 softacc_pkcs, NULL, NULL, NULL, NULL
507 /* Called from silc_acc_private_key */
508 int silc_softacc_import_private_key(void *key, SilcUInt32 key_len,
509 void **ret_private_key)
511 SilcSoftAccPrivateKey prv = silc_calloc(1, sizeof(*prv));
512 prv->pkcs = acc->pkcs;
513 prv->private_key = key;
514 *ret_private_key = prv;
517 (o Symmetric key cryptosystem acceleration? They are always sycnhronouos
518 even with hardware acceleration so the crypto API shouldn't require
525 o Import TFM. Talk to Tom to add the missing functions. Use TFM in
526 client and client library, but TMA in server, due to the significantly
527 increased memory consumption with TFM, and the rare need for public
528 key operations in server.
530 We want TFM's speed but not TFM's memory requirements. Talk to Tom
531 about making the TFM mp dynamic just as it is in LTM.
533 o The SILC MP API function must start returning indication of success
534 and failure of the operation.
536 o Do SilcStack support for silc_mp_init, silc_mp_init_size and other
537 any other MP function (including utility ones) that may allocate
540 o All utility functions should be made non-allocating ones.
543 SILC XML Library, lib/silcxml/
544 ==============================
546 o SILC XML API (wrapper to expat). Look at the expat API and simplify
547 it. The SILC XML API should have at most 8-10 API functions. It should
548 be possible to create full XML parser with only one function. And, it
549 should be possible to have a function that is able to parse an entire
550 XML document. It should also have a parser function to be able to
551 parse a stream of XML data (SilcStream). It MUST NOT have operations
552 that require multiple function calls to be able to execute that one
553 operation (like creating parser).
556 lib/silcske/silcske.[ch]
557 ========================
559 o Ratelimit to UDP/IP transport for incoming packets.
565 o Negative integer encoding is missing, add it.
567 o SILC_ASN1_CHOICE should perhaps return an index what choice in the
568 choice list was found. Currently it is left for caller to figure out
569 which choice was found.
571 o SILC_ASN1_NULL in decoding should return SilcBool whether or not
572 the NULL was present. It's important when it's SILC_ASN1_OPTIONAL
573 and we need to know whether it was present or not.
579 o OpenPGP certificate support, allowing the use of PGP public keys
586 o SSH2 public key/private key support, allowing the use of SSH2 keys
593 o PKIX implementation
599 o Deprecate the old server. Write interface for the new lib/silcserver
600 server library. The interface should work on Unix/Linux systems.
602 o Consider deprecating also the old config file format and use XML
603 istead. This should require SILC XML API implementation first.
605 o The configuration must support dynamic router and server connections.
606 The silcd must work without specifying any servers or routers to
609 o The configuration must support specifying whether the server is
610 SILC Server or SILC Router. This should not be deduced from the
611 configuration as it was in < 1.2.
613 o The configuration must support specifying the ciphers and hmacs and
614 their order so that user can specify which algorithms take preference.
620 o Rewrite the entire server. Deprecate apps/silcd as the main server
621 implementation and create lib/silcserver/. It is a platform
622 independent server library. The apps/silcd will merely provide a
623 a simple interface for the library.
625 o Write the SILC Server library extensively using SILC FSM.
627 o Server library must support multiple networks. This means that one
628 server must be able to create multiple connections that each reach
629 different SILC network. This means also that all cache's etc. must
630 be either connection-specific or network-specific.
632 o Library must support dynamic router and server connections. This means
633 that connections are create only when they are needed, like when someone
634 says JOIN foo@foo.bar.com or WHOIS foobar@silcnet.org.
636 o Library must support server-to-server connections even though protocol
637 prohibits that. The responder of the connection should automatically
638 act as a router. The two servers create an own, isolated, SILC network.
639 To be used specifically with dynamic connections.
641 o Library must support multiple threads and must be entirely thread safe.
643 o Library must have support for SERVICE command.
645 o The server must be able to run behind NAT device. This means that
646 Server ID must be based on public IP instead of private IP.
648 o The following data must be in per-connection context: client id cache,
649 server id cache, channel id cache, all statistics must be
652 o The following data must be in per-thread context: command context
653 freelist/pool, pending commands, random number generator.
655 o Do inccoming packet processing in an own FSM thread in the
656 server-threads FSM. Same as in client library.
658 o Reference count all Silc*Entry structures.
660 Some issues that must be kept in mind from 1.0 and 1.1 silcd's:
662 o The SERVER_SIGNOFF notify handing is not optimal, because it'll
663 cause sending of multiple SIGNOFF notify's instead of the one
664 SERVER_SIGNOFF notify that the server received. This should be
665 optimized so that the only SERVER_SIGNOFF is sent and not
666 SIGNOFF of notify at all (using SIGNOFF takes the idea about
667 SERVER_SIGNOFF away entirely).
669 o Another SERVER_SIGNOFF opt/bugfix: Currently the signoff is
670 sent to a client if it is on same channel as the client that
671 signoffed. However, the entire SERVER_SIGNOFF list is sent to
672 the client, ie. it may receive clients that was not on the
673 same channel. This is actually against the specs. It must be
674 done per channel. It shouldn't receive the whole list just
675 because one client happened to be on same channel.
677 o If client's public key is saved in the server (and doing public key
678 authentication) then the hostname and the username information could
679 be taken from the public key. Should be a configuration option!
681 o Add a timeout to handling incoming JOIN commands. It should be
682 enforced that JOIN command is executed only once in a second or two
683 seconds. Now it is possible to accept n incoming JOIN commands
684 and process them without any timeouts. THis must be employed because
685 each JOIN command will create and distribute the new channel key
686 to everybody on the channel.
688 o Related to above. If multiple JOINs are received in sequence perhaps
689 new key should be created only once, if the JOINs are handeled at the same
690 time. Now we create multiple keys and never end up using them because
691 many JOINs are processed at the same time in sequence. Only the last
692 key ends up being used.