so far, we ensured that propagation _into_ a store completes before
expunging it, but not that propagation _from_ it completes. this way we
could end up expunging the source messages before the changes reached
the target, which could mess up resuming after an interruption.
this doesn't really fix anything under current conditions, as so far
only Maildir driver functions that modify known messages can cause
concurrent expunges to be detected, and we don't call any of these
before the so far unchecked loops.
so far, we checked for M_DEAD only in loops over messages. but we should
have checked srec->msg uses as well. this would make the code a mess, so
instead call back from the drivers when messages are expunged, so we can
reset the pointers.
the only case where this really matters so far is the flag setting loop,
which may cause the concurrent expunge of not yet handled messages to be
detected by the maildir driver.
all printing of auxiliary callback arguments must be conditional on the
command having actually succeeded. this affected fetch_msg() most, which
outright crashed due to a null pointer deref.
to fix this for good, we automate the generation of the status printing
and checking. as a side effect, this fixes the fetch_msg() callback not
printing the status at all.
amends 4cc5ad5a.
this tests only the common case of the far side being async - adding
100% instead of 50% to the runtime of the test to cover a corner case
didn't seem worth it.
instead of delaying the callback, delay the actual driver call. this is
in line with how the IMAP driver would behave, as since commit 6c08f568
it queues the socket writes (the network upstream latency goes on top,
but that doesn't alter the result).
amends 4423a932.
- print on which side we're upgrading a placeholder. as a side effect,
this moves some magic out of upgrade_srec().
- don't use past tense for something that is only about to happen
- don't print status of every sync entry affected by journal replay -
the entry load already prints it, and relevant operations print the
new status. also, it was inconsistent with not printing the entry's
old flags.
wrap make_flags() into fmt_flags() which returns a (struct-wrapped)
string, so the calls can be inlined into the printf statements, without
reserving buffers.
we locally force optimization, so copy elision is always done, as debug
builds would otherwise suffer a somewhat unreasonable performance hit.
we optimized the case where the string would be equal even without
upper-casing, but for the much more common case where the strings differ
even after upper-casing, this was just an additional conditional.
we used to pass all initially loaded messages to match_tuids(), which
could be quite some when syncing old messages. as lost TUIDs result in
O(n^2) behavior, this could have a serious performance impact.
this is meant primarily for use with a redirected output, where no
progress counters are shown. but it would be weird not to have it with
tty output as well, so it replaces the counters after completion.
REFMAIL: 87bl2cgt6u.fsf@curie.anarc.at
speak more accurately of redundancy, not conflict. these are fatal
errors at all only because the user likely meant something else than
they typed, so we force them to think again.
this moves the channel iteration & synchronization code from main(),
with all its dependencies.
then it is "re-threaded" to be more directly driven by the driver
callbacks (like sync_boxes() is), rather than being a weird state
machine.
while the code is moved, localize many variables, and use an enum
instead of #defines for the states.
it makes no sense to trash the placeholders, and in fact the common
case is that they are deleted due to being replaced by the full
message.
a separate S_PURGED state needed to be added, as S_PURGE needs to be
reset after setting F_DELETED (so the operation doesn't count as still
pending after journal replay), yet trashing needs an indicator. logging
is now done via a separate command, as piggy-backing it on flag updates
just makes things less legible for no benefit.
this is mostly academical, as trashing being done on the side where
placeholders reside is rather unlikely.
... as otherwise these messages would be just lost.
the assumption is that opposite-side trashing is used only for locally
generated messages whose size we control. it's also more consistent with
same-side trashing, where even oversized messages would be trashed.
the exclusion was broken anyway, as we failed to query the size of old
messages, particularly after 70bad661.
messages for which only a placeholder was propagated should be treated
as not propagated, as otherwise the actual contents will be lost when
only not propagated messages are trashed.
amends 70bad661.
this is a lot more legible, and makes it possible to insert values in
the middle without churn.
i didn't find a way to do this with the pre-processor, so we now have
another code generator.
we now use the $< make variable, which requires gmake on netbsd < 9.0,
and possibly other systems with an ancient make.
this makes config+data file "sets" relocatable, which is useful for
testing.
this is technically a gratuitous backwards incompatible behavior
change, but to the degree that anyone uses relative paths at all, they
almost certainly rely on PWD being set up such that they won't see a
difference.
there are fallbacks to the old locations for compatibility.
the new locations use "isync" instead of "mbsync", which is preparation
for renaming the executable back in v1.6.
- wrap flow-controlled statements that contain blocks into blocks
themselves
- wrap bodies of do-while()s into blocks
- use braces on 'else' symmetrically (this obviously has a cascading
effect, so this patch touches lots of lines)
- attach braces
unavoidably, the rules are sometimes broken around #ifdef-ery.
instead of specifying the mailboxes and sync state verbatim, use a
format which deals only with "subjects" (but no UIDs), and specifies the
whole state for each subject on a single line (exceptions prove the
rule).
the dumpers don't try to re-create the abstraction, as that's deemed
to be an unreasonable effort.
while rewriting most of the test data anyway, move it to the bottom of
the file, which is a more natural location for it.
don't abort the comparison if continuing makes sense, and try to be more
specific about the problems.
we give up if messages are excessive/missing or the subject is wrong,
as that touches upon the rather complex problem of diff optimization.
this avoids the ugly and error-prone repeated reading of the state
after a failure.
cmpbox() had to be made non-destructive on the box state.
readchan() had to be created.
parse the test data into hierarchical structures instead of using it in
its raw form. this is semantically cleaner and allows us to change the
input format more easily.
it doesn't really add any value when resuming after an interruption, as
incomplete propagations are perfectly expected, and are recovered from.
but it does make sense in the non-UIDPLUS path, where not finding just
stored messages indicates a problem.
so move the reporting from match_tuids() to msgs_found_new(), and remove
a redundant warning from the latter.
amends 6577bf3e.
while at it, add/fix some licenses/copyrights/comments:
- it makes no sense to have a GPL exception in scripts
- ted did not contribute to the man page
- tst_timers is not part of the mbsync executable
- explicitly put the build system under GPL and add copyrights
manual:
- explain what "rename on move" really means
- reword "remote" to "opposite" to make it less confusing
(possibly renaming TrashRemoteNew left as an exercise for later)
- mention example mbsyncrc
- consistently capitalize Store/Channel/Group where they refer to the
respective configuration entities
- emphasize that SyncState may need a trailing slash (as we do for Path)
- fix missing mention of global usage/default for some options
example mbsyncrc:
- add big fat note that empty lines matter
- stop demoing deprecated options
- point out that CertificateFile is optional
REFMAIL: 877dd11jb3.fsf@angela.anarc.at
the messages are trashed in mailbox (and thus UID) order, and in
practice we expect the operations to complete in order. however, if
older messages need to be trashed after a journal replay, and we get
interrupted again, the next replay would produce an unsorted array,
and thus break the binary search.
amends 2bba9b9.
there isn't really a reason for that; DEBUG_CRASH is quite unlike the
other DEBUG_ flags.
note that the DEBUG_*_ALL flags are not checked, because they always
come with their corresponding less verbose flag anyway.
copy-pasto; it certainly wasn't meant to respond to --debug-sync.
the problem was barely noticeable, as the maildir driver's only debugs
are in the rarely triggered rescan path, apart from the flags usually
being used en bloc anyway.
amends 0e1f8f9a.
