Commit Graph

93 Commits

Author SHA1 Message Date
Oswald Buddenhagen
9261897629 don't record newuid in the sync state
this value is only ever used to find just pushed messages by TUID, so we
can simply use the UIDNEXT value from before we started pushing - and of
course, we need to record that in the journal. it makes no sense to log
the new value after completing a search, as there won't be a next search
before we push the next messages.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
96be183acb rename sync_vars_t::uidnext => newuid & fix comment
the purpose of this variable is to hold the UIDNEXT value from before
we started pushing new messages, i.e., the minimal uid we can expect
them to have.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
d7eae525bd fix TrashRemoteNew copy direction 2012-09-22 17:35:39 +02:00
Oswald Buddenhagen
35851f133b add option to control amount of fsync()ing 2012-09-15 15:28:15 +02:00
Oswald Buddenhagen
49223b2df2 avoid that a system crash can cause messages to be propagated twice
fdatasync() the journal after creating the pair record and recording
the TUID, but before the message propagation actually starts.

all other writes to the journal are not flushed, as they will at worst
cause some unnecessary network traffic without visible effect.
2012-09-15 15:28:15 +02:00
Oswald Buddenhagen
df6c3b64b7 avoid that a system crash can clobber the sync state file
make sure that the new state is committed to disk before overwriting the
old version - by default meta data is committed first, so we may end up
with no valid state at all otherwise.
2012-09-15 13:25:50 +02:00
Oswald Buddenhagen
f11504aa07 update copyrights
make the wrapper's help string also mention copyrights pertaining only
to the actual syncer, as this is the only string many people will ever
see.
2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
d4c786823d replace FSF address with something more ... contemporary 2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
6d49c343fc use a hash table for message => sync record lookup
this removes the pathological O(<number of sync records> * <number of
new messages>) case at the cost of being a bit more cpu-intensive (but
O(<number of all messages>)) for old messages.
2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
dfd7516b9a introduce ability to flatten the hierarchy of Stores 2012-09-01 21:15:52 +02:00
Oswald Buddenhagen
343f16771a don't crash when select() on master fails synchronously
svars->drv[S] would not be initialized yet, so cancel_sync() would
crash.
2012-09-01 21:15:08 +02:00
Oswald Buddenhagen
28cccf4b35 fix error handling of invalid SyncState *
when we find that the store is incompatible with in-store sync state,
we want to fail the whole channel. however, we must not claim that the
store died, otherwise it won't be disposed of properly.
2012-09-01 21:15:08 +02:00
Oswald Buddenhagen
9c86ec3442 employ alternative scheme to finding messages by TUID
instead of SEARCHing every single message (which is slow and happens to
be unreliabe with M$ Exchange 2010), just FETCH the new messages from
the mailbox - the ones we just appended will be amongst them.
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
b4cef554fc clearer debug msg 2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
7c815538ab fix line wrapping before info messages
unless an info message is explictly marked as a continuation, it must
terminate any pending line (typically the progress information) first.

debug output is not affected, as it is mutually exclusive with info
output, and no debug lines are left unterminated outside clear scopes.
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
6b3b6f12bb centralize flushing of unfinished debug lines 2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
d2bed4990d unify error reporting
- introduce sys_error() and use it instead of perror() and
  error(strerror()) in all expected error conditions
- perror() is used only for "something's really wrong with the system"
  kind of errors
- file names, etc. are quoted if they are not validated yet, so e.g. an
  empty string becomes immediately obvious
- improve and unify language
- add missing newlines
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
584e51ed7d docs
- insert "separator comments" between driver entry points
- document driver API
- document sync_vars_t parts that are stored in the sync state header
2012-09-01 16:03:35 +02:00
Oswald Buddenhagen
e5d323cc47 rely on the maildir's existence with "SyncState *"
now that we open the box first, we know that it will exist at this
point.
2012-09-01 16:03:35 +02:00
Oswald Buddenhagen
05fd0b9970 split out drv->load() from drv->select() 2012-09-01 16:02:50 +02:00
Oswald Buddenhagen
424e0e7221 make callbacks return early when canceling
even after driver->cancel() the store may complete commands successfully.
return early in this case, so we don't attempt to continue syncing.
2012-07-30 01:21:32 +02:00
Oswald Buddenhagen
ea951a697f fix error paths wrt sync drivers, take 2
synchronous error codes which are passed through callbacks aren't a
particularly good idea, after all: latest when the callback does stuff
which does not concern the caller, the return code becomes ambiguous.
instead, protect the sync_vars object with a refcount when invoking
driver functions from loops, as the callbacks they call could invalidate
the object and we would have no way of knowing that the loop should be
aborted prematurely. the upcoming async imap driver will also need a
refcount to protect the cancelation marker of the imap socket dispatcher
loop.
2012-07-30 01:21:32 +02:00
Oswald Buddenhagen
ec8f440383 don't call cancel() repeatedly on a store
erroring command replies will trickle in even after canceling
2012-07-30 01:21:32 +02:00
Oswald Buddenhagen
b0bbd23512 replace DRV_STORE_BAD with a separate bad_callback()
that way we don't have to piggy-back (possibly asynchronous) fatal
errors to particular commands.

internally, the drivers still use synchronous return values as well,
so they don't try to access the invalidated store after calling back.
2012-07-30 01:21:31 +02:00
Oswald Buddenhagen
6d86e5347e don't access free'd memory in cancel_sync()
as it happens, the 1st round *may* trash svars - if we get the
cancelation request after the slave store has already died.
2012-07-29 12:26:38 +02:00
Oswald Buddenhagen
9554026443 make drv->cancel()'s callback have no status code
this function is not going to actually execute any commands, so it
makes no sense for the callback to have a status code.
2012-07-29 12:25:24 +02:00
Oswald Buddenhagen
57444e9df9 don't decode aux pointer on DRV_CANCELED
the aux data was already free()d by the error callback by the time we
get this status code.
2012-07-22 20:19:20 +02:00
Oswald Buddenhagen
7bab2d6d94 de-duplicate error paths
makes the code more compact. yay for gotos.
2012-07-07 18:00:47 +02:00
Oswald Buddenhagen
61d98c5a1d fix a bunch of warnings 2012-07-07 18:00:47 +02:00
Oswald Buddenhagen
058d01f179 don't compare find_old_done with find_new_total
this didn't have any effect as no async drivers currently exist.
2011-03-27 11:54:48 +02:00
Oswald Buddenhagen
296ac0364c add CR after TUID during LF => CRLF conversion 2011-03-27 11:54:02 +02:00
Oswald Buddenhagen
8df1ebaf40 fix (another) out-of-bounds access in CRLF conversion
if the header contained no CRs but the body (or the post-TUID part of
the header) did, the TUID insertion would add an excess CR, thus
overflowing the buffer by one byte.
2011-03-27 11:54:02 +02:00
Oswald Buddenhagen
d94dadbaeb fix line ending conversion logic
imap may very well store messages with LF line endings. only RFC2822
requires CRLF.
consequently, preserve the line endings as much as possible unless the
mailbox format does not support it (this would be the case for unix mbox
- i actually have no idea about maildir).
2010-02-06 10:49:57 +01:00
Oswald Buddenhagen
09dfddb36b some more error reporting relating malformed messages 2010-02-06 10:49:06 +01:00
Oswald Buddenhagen
92914b37cc deal with UIDVALIDITY of 0 properly.
-REF: 20080822094543.GA3528@ugly.local
2008-08-23 07:54:00 +00:00
Oswald Buddenhagen
023d3ee577 fix error paths wrt sync drivers 2006-12-09 10:39:30 +00:00
Oswald Buddenhagen
9b657a46a0 fix bug in newline conversion causing buffer overflows.
this leads to segfaults and has some security impact.
2006-08-10 07:01:02 +00:00
Oswald Buddenhagen
21abb22c98 seen messages are eligible for expiration even if they are recent in the
mailbox.
2006-05-27 12:43:03 +00:00
Oswald Buddenhagen
168e5f3282 make the driver model, sync_chans() and sync_boxes() fully async.
async drivers to follow ...
2006-03-21 20:03:21 +00:00
Oswald Buddenhagen
bdcc285403 unscrew lf=>crlf conversion and tuid insertion 2006-03-21 17:50:57 +00:00
Oswald Buddenhagen
16eaf903db ok, mismerging and not running the reg-tests is lame. unscrew expunging
again.
2006-03-21 17:40:31 +00:00
Oswald Buddenhagen
c7903f8003 don't enter trash loop if not trashing at all. also, move expunge
message where it belongs. not adding info("trashing"), as it will be
replaced in a moment anyway.
2006-03-21 16:03:09 +00:00
Oswald Buddenhagen
b5d70aa596 async merge: aggregate most variables of main() & sync_boxes() in
main_vars_t resp. sync_vars_t.
also some minor var renames, whitespace, comments.
2006-03-21 15:53:43 +00:00
Oswald Buddenhagen
72a2d4b690 info() about opening of stores 2006-03-21 10:30:45 +00:00
Oswald Buddenhagen
3e3cf3ac9a update copyrights 2006-03-20 20:16:22 +00:00
Oswald Buddenhagen
d7126dca5e "fprintf( stderr," => "error(". new functions debugn() and infon()
for messages with missing newline; warn() and error() act upon this.
2006-03-19 11:29:12 +00:00
Oswald Buddenhagen
fbbb86738b factor out box selection from sync_boxes to avoid code dupe 2006-02-12 11:42:46 +00:00
Oswald Buddenhagen
bc39f10a1e lock the sync state open the journal before opening the master. this is
a bit ugly for the "SyncState *" case, as we have to create a directory
without making it a maildir right away. however, this makes the code
quite a bit simpler to understand and simpler to parallelize.
2006-02-11 20:28:45 +00:00
Oswald Buddenhagen
5224b5bc9f don't commit state file when a fatal error occurs 2006-02-11 20:02:06 +00:00
Oswald Buddenhagen
630a04ad3e unbelieveable, but close() can actually fail 2006-02-11 19:52:53 +00:00